Skip to main content Accessibility help

Sensitivity analysis for Markov reward structures until entrance times

  • N. M. van Dijk (a1) and H. Korezlioglu (a2)


This work presents an estimate of the error on a cumulative reward function until the entrance time of a continuous-time Markov chain into a set, when the infinitesimal generator of this chain is perturbed. The derivation of an error bound constitutes the first part of the paper while the second part deals with an application where the time until saturation is considered for a circuit switched network which starts from an empty state and which is also subject to possible failures.


Corresponding author

Postal address: Department of Econometrics, University of Amsterdam, Roeterstraat 11, 1018 WB, Amsterdam, The Netherlands
∗∗ Postal address: Ecole Nationale Supérieure des Télécommunications, 46 rue Barrault, 75634 Paris Cedex 13, France. Email address:


Hide All
[1] Baccelli, B. and Brémaud, P. (1993). Virtual customers in sensitivity and light traffic analysis via Campbell's formula for point processes. Adv. Appl. Prob. 25, 221234.
[2] Brémaud, P., and Vasquez-Abad, J. (1992). On the pathwise computation of derivatives with respect to the rate of a point process: the phantom RPA method. Queueing Systems 10, 249270.
[3] Cao, X. R. (1992). A new method of performance sensitivity analysis for non-Markovian queueing networks. Queueing Systems 10, 313350.
[4] Decreusefond, L. (1998). Perturbation analysis and Malliavin calculus. Ann. Appl. Prob. 8, 496523.
[5] van Dijk, N. M. (1988). Simple bounds for queueing systems with breakdowns. Perf. Eval. 8, 117128.
[6] van Dijk, N. M. (1990). On the importance of bias-terms for error bounds and comparison results. In Proc. 1st Int. Conf. on Numerical Solutions of Markov Chains, Raleigh, pp. 640654.
[7] van Dijk, N. M. (1991). Transient error bound analysis for continuous-time Markov reward structures. Perf. Eval. 13, 147158.
[8] van Dijk, N. M. (1992). Approximate uniformization for continuous-time Markov chains with an application to performability analysis. Stoch. Proc. Appl. 40, 339357.
[9] van Dijk, N. M., and Puterman, M. L. (1988). Perturbation theory for Markov reward processes with applications to queueing systems. Adv. Appl. Prob. 20, 7989.
[10] van Dijk, N. M. and van der Wal, J. (1989). Simple bounds and monotonicity results for multi-server exponential tandem queues. Queuing Systems 4, 119.
[11] Glasserman, P. (1992). Gradient Estimation via Perturbation Analysis. Kluwer, Dordrecht.
[12] Glynn, P. W. (1987). Likelihood ratio gradient simulation: an overview. Proc. 1987 Winter Simulation Conf., pp. 366375.
[13] Grassi, V., and Donatiello, L. (1992). Sensitivity analysis of performability. Perf. Eval. 14, 227239.
[14] Grassmann, W. (1990). Finding transient solutions in Markovian event systems through randomization. In Proc. 1st Int. Conf. on Numerical Solutions of Markov Chains, Raleigh, pp. 375395.
[15] Heidelberger, P., and Goyal, A. (1988). Sensitivity analysis of continuous-time Markov chains using uniformization. In Computer Performance and Reliability, North-Holland, Amsterdam, pp. 93104.
[16] Hinderer, K. (1978). On approximate solutions of finite-stage dynamic programs. In Dynamic Programming and its Applications, ed. Puterman, M. Academic Press, New York, pp. 298318.
[17] Ho, Y. C., and Cao, X. R. (1983). Perturbation analysis and optimization of queueing networks. J. Optim. Theory Appl. 40, 559582.
[18] Karlin, S., and Taylor, H. M. (1981). A Second Course in Stochastic Processes. Academic Press, New York.
[19] Kemeny, J. G., Snell, J. L., and Knapp, A. W. (1966). Denumerable Markov Chains. Van Nostrand, New York.
[20] Liu, Z., and Nain, P. (1991). Sensitivity results in open, closed mixed product form queueing networks. Perf. Eval. 13, 237251.
[21] Meyer, C. D. Jr. (1980). The condition of a finite Markov chain and perturbation bounds for the limiting probabilities. SIAM J. Alg. Discrete Methods 1, 273283.
[22] Reibman, A., Trivedi, K. S., and Smith, R. (1989). Markov and Markov reward model transient analysis: an overview of numerical approaches. Eur. J. Operat. Res. 40, 257267.
[23] Reiman, M. I., and Weiss, A. (1989). Sensitivity analysis via likelihood ratios. Operat. Res. 37, 830844.
[24] Rubinstein, R. (1989). Sensitivity analysis and performance extrapolation for computer simulation models. Operat. Res. 37, 7281.
[25] Schweitzer, P. J. (1968). Perturbation theory and finite Markov chains. J. Appl. Prob. 5, 401413.
[26] Shantikumar, J. G., and Yao, D. D. (1987). Stochastic monotonicity of the queue lengths in closed queueing networks. Operat. Res. 35, 583588.
[27] Stoyan, D. (1983). Comparison Methods for Queues and other Stochastic Models. John Wiley, New York.
[28] Suri, R. (1983). Robustness of queueing network formulas. J. Assoc. Comp. Mach. 30, 564594.
[29] Tsoucas, P., and Walrand, J. (1989). Monotonicity of throughput in non-Markovian networks. J. Appl. Prob. 26, 134141.
[30] Whitt, W. (1978). Approximations of dynamic programs I. Math. Operat. Res. 3, 231243.


MSC classification

Sensitivity analysis for Markov reward structures until entrance times

  • N. M. van Dijk (a1) and H. Korezlioglu (a2)


Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed