Hostname: page-component-cd9895bd7-dzt6s Total loading time: 0 Render date: 2024-12-22T07:14:10.715Z Has data issue: false hasContentIssue false
  • English
  • Français

Adaptive control of M/M/1 queues—continuous-time Markov decision process approach

Published online by Cambridge University Press:  14 July 2016

Lam Yeh  and
L. C. Thomas
Lam Yeh*
Affiliation:
National University of Singapore
L. C. Thomas*
Affiliation:
University of Manchester
*
∗ Postal address: Department of Mathematics, National University of Singapore, Kent Ridge, Singapore 0511.
∗∗ Postal address: Department of Decision Theory, University of Manchester, Manchester M13 9PL, U.K.
Get access Rights & Permissions [Opens in a new window]

Abstract

By considering continuous-time Markov decision processes where decisions can be made at any time, we show in the case of M/M/1 queues with discounted costs that there exists a monotone optimal policy among all the regular policies.

Keywords

CONTINUOUS-TIME MARKOV DECISION PROCESSESINCOMPLETE GAMMA FUNCTIONMONOTONE OPTIMAL POLICYREGULAR POLICY
Type
Research Papers
Information
Journal of Applied Probability , Volume 20 , Issue 2 , June 1983 , pp. 368 - 379
Copyright
Copyright © Applied Probability Trust 1983 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Collins, E. J. (1981) Markov Decision Processes with Reference to Queueing Theory. , University of Manchester.Google Scholar
Crabill, T. B. (1972) Optimal control of a service facility with variable exponential service time and constant arrival rate. Management Sci. 18, 560–566.Google Scholar
Dunford, N. and Schwartz, J. T. (1958) Linear Operators, Part 1. Interscience, New York.Google Scholar
Erdelyi, A. Magnus, W., Oberhetlinger, F. and Tricomi, F. G. (1953) Higher Transcendental Functions, Vol. 2. McGraw-Hill, New York.Google Scholar
Horduk, A. and Van Der Duyn Schouten, F. A. (1980) Weak convergence of decision processes. In Recent Developments in Markov Decision Processes, ed. Hartley, R., Thomas, L. C. and White, D. J., Academic Press, London.Google Scholar
Karlin, S. (1960) Dynamic inventory policy with varying stochastic demands. Management Sci. 6, 231–258.Google Scholar
Yeh, Lam (1981) Adaptive Control of Queues. , University of Manchester.Google Scholar
Lippman, S. A. (1975) Applying a new device in the optimization of exponential queueing systems. Operat. Res. 23, 687–710.Google Scholar
Mitchell, B. (1973) Optimal service-rate selection in an M/G/1 queue. SIAM J. Appl. Math. 24, 19–35.Google Scholar
Papachristos, S. (1975) Adaptive Dynamic Programming in Inventory Control. , University of Manchester.Google Scholar
Scarf, H. (1959) Bayes solution of the statistical inventory problem. Ann. Math. Statist. 30, 490–508.Google Scholar
Strauch, R. E. (1966) Negative dynamic programming. Ann. Math. Statist. 37, 871–890.Google Scholar
Van Der Duyn Schouten, F. A. (1979) Markov Decision Processes with Continuous Time Parameter. Mathematical Center, Amsterdam.Google Scholar
Yadin, M. and Zacks, S. (1978) Adaptation of the service capacity in a queueing system which is subjected to a change in the arrival rate at unknown epoch. Adv. Appl. Prob. 10, 666–681.CrossRefGoogle Scholar