Skip to main content Accessibility help

Bias optimal admission control policies for a multiclass nonstationary queueing system

  • Mark E. Lewis (a1), Hayriye Ayhan (a2) and Robert D. Foley (a2)


We consider a finite-capacity queueing system where arriving customers offer rewards which are paid upon acceptance into the system. The gatekeeper, whose objective is to ‘maximize’ rewards, decides if the reward offered is sufficient to accept or reject the arriving customer. Suppose the arrival rates, service rates, and system capacity are changing over time in a known manner. We show that all bias optimal (a refinement of long-run average reward optimal) policies are of threshold form. Furthermore, we give sufficient conditions for the bias optimal policy to be monotonic in time. We show, via a counterexample, that if these conditions are violated, the optimal policy may not be monotonic in time or of threshold form.


Corresponding author

Postal address: Department of Industrial and Operations Engineering, University of Michigan, 1205 Beal Avenue, Ann Arbor, MI 48109-2117, USA. Email address:
∗∗ Postal address: School of Industrial and Systems Engineering, Georgia Institute of Technology, Atlanta, GA 30332-0205, USA.


Hide All
[1]. Altman, E. and Stidham, S. Jr (1995). Optimality of monotone policies for two-action Markovian decision processes, with applications to control of queues with delayed information. Queueing Systems 21, 267291.
[2]. Dutta, P. K. (1991). What do discounted optima converge to? A theory of discount rate asymptotics in economic models. J. Econom. Theory 55, 6494.
[3]. Fu, M. C., Marcus, S. I., and Wang, I-J. (2000). Monotone optimal policies for a transient queueing staffing problem. Operat. Res. 48, 327331.
[4]. Ghoneim, H. A. and Stidham, S. Jr (1985). Control of arrivals to two queues in series. Europ. J. Operat. Res. 21, 399409.
[5]. Haviv, M., and Puterman, M. L. (1998). Bias optimality in controlled queueing systems. J. Appl. Prob. 35, 136150.
[6]. Helm, W. E., and Waldmann, K.-H. (1984). Optimal control of arrivals to multi-server queues in a random environment. J. Appl. Prob. 21, 602615.
[7]. Hernández-Lerma, O., and Lasserre, J. B. (1999). Further Topics on Discrete-Time Markov Control Processes. Springer, New York.
[8]. Heyman, D. P. (1968). Optimal operating policies for M/G/1 queueing systems. Operat. Res. 16, 362382.
[9]. Lewis, M. E. (1998). Bias optimality in a two-class nonstationary queueing system. Doctoral Thesis, Georgia Institute of Technology.
[10]. Lewis, M. E. (2001). Average optimal policies in a controlled queueing system with dual admission control. J. Appl. Prob. 38, 369385.
[11]. Lewis, M. E., and Puterman, M. L. (2000). A note on bias optimality in controlled queueing systems. J. Appl. Prob. 37, 300305.
[12]. Lewis, M. E., and Puterman, M. L. (2001). A probabilistic analysis of bias optimality in unichain Markov decision processes. IEEE Trans. Automatic Control 46, 96100.
[13]. Lewis, M. E., and Puterman, M. L. (2001). Bias optimality. In Handbook of Markov Decision Processes: Methods and Applications, eds Feinberg, E. and Shwartz, A., Kluwer, Dordrecht, pp. 89111.
[14]. Lewis, M. E., Ayhan, H., and Foley, R. D. (1999). Bias optimality in a queue with admission control. Prob. Eng. Inf. Sci. 13, 309327.
[15]. Lippman, S. A. (1975). Applying a new device in the optimization of exponential queueing systems. Operat. Res. 23, 687712.
[16]. Lippman, S. A., and Ross, S. M. (1971). The streetwalker's dilemma: a job shop model. SIAM J. Appl. Math. 20, 336342.
[17]. Massey, W. A., and Whitt, W. (1996). Stationary-process approximations for the nonstationary Erlang loss model. Operat. Res. 44, 976983.
[18]. Massey, W. A., and Whitt, W. (1997). Peak congestion in multi-service systems with slowly varying arrival rates. QUESTA 25, 157172.
[19]. Miller, B. L. (1969). A queueing reward system with several customer classes. Manag. Sci. 16, 234245.
[20]. Reiman, M. I. (1989). Asymptotically optimal trunk reservation for large trunk groups. In Proc. 28th Conf. Decision Control (Tampa, FL, December 1989), IEEE, New York, pp. 25362541.
[21]. Stidham, S. Jr (1985). Optimal control of admission to a queueing system. IEEE Trans. Automatic Control 30, 705713.
[22]. Veinott, A. F. Jr (1966). On finding optimal policies in discrete dynamic programming with no discounting. Ann. Math. Statist. 37, 12841294.
[23]. Veinott, A. F. Jr (1974). Markov decision chains. In Studies in Optimization (Studies Math. 10), eds Dantzig, G. B. and Eaves, B. C., Mathematical Association of America, Washington, pp. 124159.
[24]. Yadin, M., and Naor, P. (1963). Queueing systems with a removable service station. Operat. Res. Quart. 14, 393405.


MSC classification

Bias optimal admission control policies for a multiclass nonstationary queueing system

  • Mark E. Lewis (a1), Hayriye Ayhan (a2) and Robert D. Foley (a2)


Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed.