Hostname: page-component-7479d7b7d-c9gpj Total loading time: 0 Render date: 2024-07-12T07:01:11.170Z Has data issue: false hasContentIssue false
  • English
  • Français

Scoring probability forecasts for point processes: the entropy score and information gain

Part of: Stochastic processes Inference from stochastic processes

Published online by Cambridge University Press:  14 July 2016

Daryl J. Daley  and
David Vere-Jones
Daryl J. Daley
Affiliation:
Centre for Mathematics and Its Applications, Mathematical Sciences Institute, Australian National University, Canberra, ACT 0200, Australia. Email address: daryl.daley@anu.edu.au
David Vere-Jones
Affiliation:
School of Mathematical and Computing Sciences, Victoria University of Wellington, PO Box 600, Wellington, New Zealand. Email address: david.vere-jones@mcs.vuw.ac.nz
Get access Rights & Permissions [Opens in a new window]

Abstract

The entropy score of an observed outcome that has been given a probability forecast p is defined to be –log p. If p is derived from a probability model and there is a background model for which the same outcome has probability π, then the log ratio log(p/π) is the probability gain, and its expected value the information gain, for that outcome. Such concepts are closely related to the likelihood of the model and its entropy rate. The relationships between these concepts are explored in the case that the outcomes in question are the occurrence or nonoccurrence of events in a stochastic point process. It is shown that, in such a context, the mean information gain per unit time, based on forecasts made at arbitrary discrete time intervals, is bounded above by the entropy rate of the point process. Two examples illustrate how the information gain may be related to realizations with a range of values of ‘predictability'.

Keywords

Entropy scoreprobability gaininformation gain

MSC classification

Primary: 60G25: Prediction theory
Secondary: 62M20: Prediction
Type
Part 6. Stochastic processes
Information
Journal of Applied Probability , Volume 41 , Issue A: Stochastic Methods and their Applications , 2004 , pp. 297 - 312
Copyright
Copyright © Applied Probability Trust 2004 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Abramowitz, M. and Stegun, I. A., (eds) (1992). Handbook of Mathematical Functions. Dover, New York. (Ninth Dover printing of 10th Government Printing Office printing, with corrections.) Google Scholar
Akaike, H. (1973). Information theory and an extension of the maximum likelihood principle. In 2nd Internat. Symp. Information Theory , eds Petrov, B. N. and Csáki, F., Akadémiai Kiadó, Budapest, pp. 261–281.Google Scholar
Akaike, H. (1974). A new look at the statistical model identification. IEEE Trans. Automatic Control 19, 716–723.CrossRefGoogle Scholar
Aki, K. (1989). Ideal probabilistic earthquake prediction. Tectonophysics 169, 197–198.Google Scholar
Bremaud, P. (1981). Point Processes and Queues: Martingale Dynamics. Springer, New York.CrossRefGoogle Scholar
Csiszár, I. (1969). On generalized entropy. Studia Sci. Math. Hung. 4, 401–419.Google Scholar
Daley, D. J. and Vere-Jones, D. (1988). An Introduction to The Theory of Point Processes. Springer, New York.Google Scholar
Daley, D. J. and Vere-Jones, D. (2003). An Introduction to The Theory of Point Processes , Vol. I, Elementary Theory and Methods , 2nd edn. Springer, New York.Google Scholar
Fritz, J. (1969). Entropy of point processes. Studia Sci. Math. Hung. 4, 389–399.Google Scholar
Fritz, J. (1973). An approach to the entropy of point processes. Periodica Math. Hung. 3, 73–83.CrossRefGoogle Scholar
Grigelionis, B. (1974). The representation by stochastic integrals of square integrable martingales. Litovsk. Mat. Sb. 14, 53–69, 233-234 (in Russian).Google Scholar
Grigelionis, B. (1975). Stochastic point processes and martingales. Litovsk. Mat. Sb. 15, 101–114, 227-228 (in Russian).Google Scholar
Imoto, M. (2000). A quality factor of earthquake probability models in terms of mean information gain. Zesin 53, 79–81.CrossRefGoogle Scholar
Jacod, J. and Shiryaev, A. N. (1987). Limit Theorems for Stochastic Processes. Springer, Berlin.CrossRefGoogle Scholar
Kagan, Y. Y. and Knopoff, L. (1977). Earthquake risk prediction as a stochastic process. Phys. Earth Planet. Interiors 14, 97–108.CrossRefGoogle Scholar
Kallenberg, O. (1986). Random Measures , 4th edn. Akademie Verlag, Berlin.Google Scholar
Liptser, R. S. and Shiryaev, A. N. (2001). Statistics of Random Processes , Vol. II, Applications (Appl. Math. 5), 2nd edn. Springer, Berlin.Google Scholar
Macchi, O. (1975). The coincidence approach to stochastic point processes. Adv. Appl. Prob. 7, 83–122.CrossRefGoogle Scholar
McFadden, J. (1965). The entropy of a point process. J. Soc. Ind. Appl. Math. 13, 988–994.Google Scholar
Papangelou, F. (1978). On the entropy rate of stationary point processes. Z. Wahrscheinlichkeitsth. 44, 191–211.Google Scholar
Rényi, A. (1959). On the dimension and entropy of probability distributions. Acta Math. Acad. Sci. Hung. 10, 193–215.CrossRefGoogle Scholar
Rényi, A. (1960). Dimension, entropy and information. In Trans. 2nd Prague Conf. Inf. Theory , Publishing House of the Czechoslovak Academy of Science, Prague, pp. 545–556.Google Scholar
Rudemo, M. (1964). Dimension and entropy for a class of stochastic processes. Magyar Tud. Akad. Mat. Kutató Int. Közl. 9, 73–87.Google Scholar
Vere-Jones, D. (1998). Probability and information gain for earthquake forecasting. Comput. Seismol. 30, 248–263.Google Scholar