Universality of power-of-d load balancing in many-server systems

Debankur Mukherjee, Sem C. Borst, Johan S. H. van Leeuwaarden, Philip A. Whiting

Research output: Contribution to journalArticlepeer-review

15 Downloads (Pure)


We consider a system of N parallel single-server queues with unit exponential service rates and a single dispatcher where tasks arrive as a Poisson process of rate λ(N). When a task arrives, the dispatcher assigns it to a server with the shortest queue among d(N) randomly selected servers (1≤d(N)≤N). This load balancing strategy is referred to as a JSQ(d(N)) scheme, noting that it subsumes the celebrated Join-the-Shortest Queue (JSQ) policy as a crucial special case for d(N) = N. We construct a stochastic coupling to bound the difference in the queue length processes between the JSQ policy and a JSQ(d(N)) scheme with an arbitrary value of d(N). We use the coupling to derive the fluid limit in the regime where λ(N)/N→λ<1 as N→∞ with d(N)→∞, along with the associated fixed point. The fluid limit turns out not to depend on the exact growth rate of d(N) and in particular coincides with that for the JSQ policy. We further leverage the coupling to establish that the diffusion limit in the critical regime where (N−λ(N))/√N→β>0 as N→∞ with d(N)/(√N√log(N))→∞ corresponds to that for the JSQ policy. These results indicate that the optimality of the JSQ policy can be preserved at the fluid level and diffusion level while reducing the overhead by nearly a factor O(N) and O(√N−/log(N)), respectively.
Original languageEnglish
Pages (from-to)265-292
Number of pages28
JournalStochastic Systems
Issue number4
Publication statusPublished - Dec 2018

Bibliographical note

Copyright the Author(s) 2018. Version archived for private and non-commercial use with the permission of the author/s and according to publisher conditions. For further rights please contact the publisher.


  • load balancing
  • power-of-d scheme
  • join the shortest queue
  • stochastic coupling
  • functional limit theorems
  • fluid limit
  • diffusion limit
  • many-server asymptotics


Dive into the research topics of 'Universality of power-of-d load balancing in many-server systems'. Together they form a unique fingerprint.

Cite this