Skip to main content Accessibility help

Large-scale join-idle-queue system with general service times

  • S. Foss (a1) and A. L. Stolyar (a2)


A parallel server system with n identical servers is considered. The service time distribution has a finite mean 1 / μ, but otherwise is arbitrary. Arriving customers are routed to one of the servers immediately upon arrival. The join-idle-queue routeing algorithm is studied, under which an arriving customer is sent to an idle server, if such is available, and to a randomly uniformly chosen server, otherwise. We consider the asymptotic regime where n → ∞ and the customer input flow rate is λn. Under the condition λ / μ < ½, we prove that, as n → ∞, the sequence of (appropriately scaled) stationary distributions concentrates at the natural equilibrium point, with the fraction of occupied servers being constant at λ / μ. In particular, this implies that the steady-state probability of an arriving customer waiting for service vanishes.


Corresponding author

* Postal address: Department of Actuarial Mathematics and Statistics, Heriot-Watt University, Edinburgh EH14 4AS, UK. Email address:
** Postal address: Department of Industrial and Enterprise Systems Engineering, University of Illinois at Urbana-Champaign, 104 S. Mathews Avenue, Office 201C, Urbana, IL 61801, USA. Email address:


Hide All
[1] Badonnel, R. and Burgess, M. (2008). Dynamic pull-based load balancing for autonomic servers. In Network Operations and Management Symposium, NOMS 2008, IEEE, pp. 751754.
[2] Billingsley, P. (1995). Probability and Measure, 3rd edn. John Wiley, New York.
[3] Bramson, M., Lu, Y. and Prabhakar, B. (2012). Asymptotic independence of queues under randomized load balancing. Queueing Systems 71, 247292.
[4] Bramson, M., Lu, Y. and Prabhakar, B. (2013). Decay of tails at equilibrium for FIFO join the shortest queue networks. Ann. Appl. Prob. 23, 18411878.
[5] Eschenfeldt, P. and Gamarnik, D. (2015). Join the shortest queue with many servers. The heavy traffic asymptotics. Preprint. Available at
[6] Lu, Y. et al. (2011). Join-idle-queue: a novel load balancing algorithm for dynamically scalable web services. Performance Evaluation 68, 10561071.
[7] Mitzenmacher, M. (2001). The power of two choices in randomized load balancing. IEEE Trans. Parallel Distributed Systems 12, 10941104.
[8] Mukherjee, D., Borst, S. C., van Leeuwaarden, J. S. H. and Whiting, P. A. (2016). Universality of load balancing schemes on the diffusion scale. J. Appl. Prob. 53, 11111124.
[9] Stolyar, A. L. (2015). Pull-based load distribution in large-scale heterogeneous service systems. Queueing Systems 80, 341361.
[10] Stolyar, A. L. (2017). Large-scale heterogeneous service systems with general packing constraints. Adv. Appl. Prob. 49, 6183.
[11] Stolyar, A. L. (2017). Pull-based load distribution among heterogeneous parallel servers: the case of multiple routers. Queueing Systems 85, 3165.
[12] Vvedenskaya, N. D., Dobrushin, R. L. and Karpelevich, F. I. (1996). A queueing system with a choice of the shorter of two queues—an asymptotic approach. Problems Information Transmission 32, 1527.
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Journal of Applied Probability
  • ISSN: 0021-9002
  • EISSN: 1475-6072
  • URL: /core/journals/journal-of-applied-probability
Please enter your name
Please enter a valid email address
Who would you like to send this to? *


MSC classification


Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed