Hostname: page-component-7479d7b7d-767nl Total loading time: 0 Render date: 2024-07-13T13:19:59.935Z Has data issue: false hasContentIssue false
  • English
  • Français

On sets of countable non-negative matrices and Markov decision processes

Published online by Cambridge University Press:  01 July 2016

Douglas P. Kennedy
Douglas P. Kennedy*
Affiliation:
University of Cambridge
Get access Rights & Permissions [Opens in a new window]

Abstract

Consider a set S of countable non-negative matrices satisfying the property that for any two indices i, j, for some n ≧ 1 there are matrices M1, M2, · · ·, Mn in S with (M1M2 · · · Mn)ij >0. For non-negative vectors x set Tx = supM∈SMx, where the supremum is taken separately in each coordinate. Assume that for each x with Tx finite in each coordinate there is a matrix in S which achieves the supremum simultaneously for all coordinates. With these two assumptions on S, the R-theory for a countable irreducible matrix is extended to the operator T. The results are used to consider the existence of stationary optimal policies for Markov decision processes with multiplicative rewards.

Keywords

NON-NEGATIVE MATRICESR-THEORYPERRON–FROBENIUS THEORYDYNAMIC PROGRAMMINGMARKOV DECISION PROCESSES
Type
Research Article
Information
Advances in Applied Probability , Volume 10 , Issue 3 , September 1978 , pp. 633 - 646
Copyright
Copyright © Applied Probability Trust 1978 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

[1] Bather, J. (1973) Optimal decision procedures for finite Markov chains, II. Adv. Appl. Prob. 5, 521–540.CrossRefGoogle Scholar
[2] Bellman, R. (1956) On a class of quasi-linear equations. Canadian J. Maths 8, 198–202.CrossRefGoogle Scholar
[3] Bellman, R. (1957) Dynamic Programming. Princeton University Press, Princeton, N.J. Google ScholarPubMed
[4] Hordijk, A. (1974) Dynamic Programming and Markov Potential Theory. Mathematisch Centrum, Amsterdam.Google Scholar
[5] Howard, P. A. and Matheson, J. E. (1972) Risk-sensitive Markov decision processes. Management Sci. 18, 356–369.CrossRefGoogle Scholar
[6] Mandl, P. and Seneta, E. (1969) The theory of non-negative matrices in a dynamic programming problem. Austral. J. Statist. 11, 85–96.CrossRefGoogle Scholar
[7] Seneta, E. (1973) Non-Negative Matrices. Allen and Unwin, London.Google Scholar
[8] Vere-Jones, D. (1967) Ergodic properties of non-negative matrices I. Pacific J. Maths 22, 361–386.CrossRefGoogle Scholar