Skip to main content Accessibility help

On the entropy for semi-Markov processes

  • Valerie Girardin (a1) and Nikolaos Limnios (a2)


The aim of this paper is to define the entropy of a finite semi-Markov process. We define the entropy of the finite distributions of the process, and obtain explicitly its entropy rate by extending the Shannon–McMillan–Breiman theorem to this class of nonstationary continuous-time processes. The particular cases of pure jump Markov processes and renewal processes are considered. The relative entropy rate between two semi-Markov processes is also defined.


Corresponding author

Postal address: Mathématiques, Campus II, Université de Caen, BP 5186, 14032 Caen, France. Email address:
∗∗ Postal address: Laboratoire de Mathématiques Appliquées, Université de Technologie de Compiègne, BP 20529, 60205 Compiègne Cedex, France.


Hide All
[1] Albert, A. (1962). Estimating the infinitesimal generator of a continuous time finite state Markov process. Ann. Math. Statist. 38, 727753.
[2] Bad Dumitrescu, M. (1988). Some informational properties of Markov pure-jump processes. Cas. Pestovani Mat. 113, 429434.
[3] Breiman, L. (1958). The individual ergodic theorem of information theory. Ann. Math. Statist. 28, 809811.
[4] Breiman, L. (1960). Correction to: the individual ergodic theorem of information theory. Ann. Math. Statist. 31, 809810.
[5] Gut, A. (1988). Stopped Random Walks, Limit Theorems and Applications. Springer, New York.
[6] Limnios, N. and Oprişan, G. (2001). Semi-Markov Processes and Reliability. Birkhauser, Boston, MA.
[7] Mcmillan, M. (1953). The basic theorems of information theory. Ann. Math. Statist. 24, 196219.
[8] Moore, E. H., and Pyke, R. (1968). Estimation of the transition distributions of a Markov renewal process. Ann. Inst. Statist. Math. 20, 411424.
[9] Perez, A. (1964). Extensions of Shannon—McMillan's limit theorem to more general stochastic processes. In Trans. Third Prague Conf. Inf. Theory, Statist. Decision Functions, Random Processes, Publishing House of the Czechoslovak Academy of Science, Prague, pp. 545574.
[10] Pinsker, M. S. (1964). Information and Information Stability of Random Variables and Processes. Holden-Day, San Francisco, CA.
[11] Shannon, C. E. (1948). A mathematical theory of communication. Bell Syst. Tech. J. 27, 379423, 623—656.
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Journal of Applied Probability
  • ISSN: 0021-9002
  • EISSN: 1475-6072
  • URL: /core/journals/journal-of-applied-probability
Please enter your name
Please enter a valid email address
Who would you like to send this to? *


MSC classification


Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed