Skip to main content
SpringerLink
Log in

WCFS: a new framework for analyzing multiserver systems

  • Published:
Queueing Systems Aims and scope Submit manuscript

Abstract

Multiserver queueing systems are found at the core of a wide variety of practical systems. Many important multiserver models have a previously-unexplained similarity: identical mean response time behavior is empirically observed in the heavy traffic limit. We explain this similarity for the first time. We do so by introducing the work-conserving finite-skip (WCFS) framework, which encompasses a broad class of important models. This class includes the heterogeneous M/G/k, the Limited Processor Sharing policy for the M/G/1, the Threshold Parallelism model and the Multiserver-Job model under a novel scheduling algorithm. We prove that for all WCFS models, scaled mean response time \(E[T](1-\rho )\) converges to the same value, \(E[S^2]/(2E[S])\), in the heavy-traffic limit, which is also the heavy traffic limit for the M/G/1/FCFS. Moreover, we prove additively tight bounds on mean response time for the WCFS class, which hold for all load \(\rho \). For each of the four models mentioned above, our bounds are the first known bounds on mean response time.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Notes

  1. This assumption is defined in Sect. 2.3.

  2. The data was published in a scaled form [15]. We rescale the data so the smallest job in the trace uses one normalized CPU.

References

  1. Nathuji, R., Isci, C., Gorbatov, E.: Exploiting platform heterogeneity for power efficient data centers. In: Fourth International Conference on Autonomic Computing (ICAC’07), p. 5 (2007)

  2. Mars, J., Tang, L., Hundt, R.: Heterogeneity in “homogeneous’’ warehouse-scale computers: a performance opportunity. IEEE Comput. Arch. Lett. 10(2), 29–32 (2011)

    Article  Google Scholar 

  3. Cho, H.-D., Engineer, P.D.P., Chung, K., Kim, T.: Benefits of the big. LITTLE architecture. EETimes, (2012)

  4. Yashkov, S., Yashkova, A.: Processor sharing: a survey of the mathematical theory. Autom. Remote Control 68(9), 1662–1731 (2007)

    Article  Google Scholar 

  5. Nuyens, M., Van Der Weij, W.: Monotonicity in the limited processor sharing queue. Resource 4, 7 (2008)

    Google Scholar 

  6. Telek, M., Van Houdt, B.: Response time distribution of a class of limited processor sharing queues. SIGMETRICS Perform. Eval. Rev. 45(3), 143–155 (2018)

    Article  Google Scholar 

  7. Zhang, J., Zwart, B.: Steady state approximations of limited processor sharing queues in heavy traffic. Queueing Syst. 60(3), 227–246 (2008)

    Article  Google Scholar 

  8. Gupta, V., Harchol-Balter, M.: Self-adaptive admission control policies for resource-sharing systems. SIGMETRICS Perform. Eval. Rev. 37(1), 311–322 (2009)

    Article  Google Scholar 

  9. Delimitrou, C., Kozyrakis, C.: Quasar: Resource-efficient and QoS-aware cluster management. In: Proceedings of the 19th International Conference on Architectural Support for Programming Languages and Operating Systems. ASPLOS ’14, pp. 127–144 (2014)

  10. Peng, Y., Bao, Y., Chen, Y., Wu, C., Guo, C.: Optimus: an efficient dynamic resource scheduler for deep learning clusters. In: Proceedings of the Thirteenth EuroSys Conference. EuroSys ’18 (2018)

  11. Maguluri, S.T., Srikant, R., Ying, L.: Stochastic models of load balancing and scheduling in cloud computing clusters. In: 2012 Proceedings IEEE Infocom, pp. 702–710. IEEE, Orlando (2012)

  12. Feitelson, D.G., Rudolph, L., Schwiegelshohn, U.: Parallel job scheduling—a status report. In: Workshop on Job Scheduling Strategies for Parallel Processing, pp. 1–16. Springer, New York (2004)

  13. Srinivasan, S., Kettimuthu, R., Subramani, V., Sadayappan, P.: Characterization of backfilling strategies for parallel job scheduling. In: Proceedings of International Conference on Parallel Processing Workshop, pp. 514–519 (2002)

  14. Carastan-Santos, D., De Camargo, R.Y., Trystram, D., Zrigui, S.: One can only gain by replacing easy backfilling: a simple scheduling policies case study. In: 2019 19th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID), pp. 1–10 (2019)

  15. Tirmazi, M., Barker, A., Deng, N., Haque, M.E., Qin, Z.G., Hand, S., Harchol-Balter, M., Wilkes, J.: Borg: the next generation. In: Proceedings of the Fifteenth European Conference on Computer Systems. EuroSys ’20 (2020)

  16. Grosof, I., Harchol-Balter, M., Scheller-Wolf, A.: Stability for two-class multiserver-job systems. arXiv preprint arXiv:2010.00631 (2020)

  17. Grosof, I., Harchol-Balter, M., Scheller-Wolf, A.: WCFS: a new framework for analyzing multiserver systems (2022). https://doi.org/10.48550/ARXIV.2109.12663

  18. Loulou, R.: Multi-channel queues in heavy traffic. J. Appl. Probab. 10(4), 769–777 (1973)

    Article  Google Scholar 

  19. Köllerström, J.: Heavy traffic theory for queues with several servers. I. J. Appl. Probab. 11(3), 544–552 (1974)

    Article  Google Scholar 

  20. Köllerström, J.: Heavy traffic theory for queues with several servers. II. J. Appl. Probab. 16(2), 393–401 (1979)

    Article  Google Scholar 

  21. Kingman, J.: Some inequalities for the queue GI/G/1. Biometrika 49(3/4), 315–324 (1962)

    Article  Google Scholar 

  22. Gamarnik, D., Momčilović, P.: Steady-state analysis of a multiserver queue in the Halfin-Whitt regime. Adv. Appl. Probab. 40(2), 548–577 (2008)

    Article  Google Scholar 

  23. Aghajani, R., Ramanan, K.: The limit of stationary distributions of many-server queues in the Halfin–Whitt regime. Math. Oper. Res. 45(3), 1016–1055 (2020)

    Article  Google Scholar 

  24. Dai, J., Dieker, A., Gao, X.: Validity of heavy-traffic steady-state approximations in many-server queues with abandonment. Queueing Syst. 78(1), 1–29 (2014)

    Article  Google Scholar 

  25. Goldberg, D.A., Li, Y.: Simple and explicit bounds for multi-server queues with universal 1/(1-rho) scaling. arXiv preprint arXiv:1706.04628 (2017)

  26. Efrosinin, D.V., Rykov, V.V.: On performance characteristics for queueing systems with heterogeneous servers. Autom. Remote Control 69(1), 61–75 (2008)

    Article  Google Scholar 

  27. Alves, F., Yehia, H., Pedrosa, L., Cruz, F., Kerbache, L.: Upper bounds on performance measures of heterogeneous M/M/c queues. Math. Probl. Eng. 2011 (2011)

  28. Efrosinin, D., Stepanova, N., Sztrik, J., Plank, A.: Approximations in performance analysis of a controllable queueing system with heterogeneous servers. Mathematics 8(10), 1803 (2020)

    Article  Google Scholar 

  29. Lin, W., Kumar, P.: Optimal control of a queueing system with two heterogeneous servers. IEEE Trans. Autom. Control 29(8), 696–703 (1984)

    Article  Google Scholar 

  30. Van Harten, A., Sleptchenko, A.: On Markovian multi-class, multi-server queueing. Queueing Syst. 43(4), 307–328 (2003)

    Article  Google Scholar 

  31. Boxma, O.J., Deng, Q., Zwart, A.P.: Waiting-time asymptotics for the M/G/2 queue with heterogeneous servers. Queueing Syst. 40(1), 5–31 (2002)

    Article  Google Scholar 

  32. Keaogile, T., Fatai Adewole, A., Ramasamy, S.: Geo (\(\lambda \))/Geo (\(\mu \))+ G/2 queues with heterogeneous servers operating under FCFS queue discipline. Am. J. Appl. Math. Stat 3(2), 54–58 (2015)

    Google Scholar 

  33. Sani, S., Daman, O.A.: The M/G/2 queue with heterogeneous servers under a controlled service discipline: stationary performance analysis. IAENG Int. J. Appl. Math. 45(1) (2015)

  34. Ramasamy, S., Daman, O.A., Sani, S.: An M/G/2 queue where customers are served subject to a minimum violation of FCFS queue discipline. Eur. J. Oper. Res. 240(1), 140–146 (2015)

    Article  Google Scholar 

  35. Zhang, J., Dai, J.G., Zwart, B.: Law of large number limits of limited processor-sharing queues. Math. Oper. Res. 34(4), 937–970 (2009)

    Article  Google Scholar 

  36. Zhang, J., Dai, J.G., Zwart, B.: Diffusion limits of limited processor sharing queues. Ann. Appl. Probab. 21(2), 745–799 (2011)

    Article  Google Scholar 

  37. Harchol-Balter, M.: Performance Modeling and Design of Computer Systems: Queueing Theory in Action. Cambridge University Press, Cambridge (2013)

    Book  Google Scholar 

  38. Berg, B., Dorsman, J.-P., Harchol-Balter, M.: Towards optimality in parallel scheduling. Proc. ACM Meas. Anal. Comput. Syst. 1(2) (2017)

  39. Berg, B., Harchol-Balter, M.: Optimal scheduling of parallel jobs with unknown service requirements. In: Handbook of Research on Methodologies and Applications of Supercomputing, pp. 18–40. IGI Global, Hershey (2021)

  40. Berg, B., Harchol-Balter, M., Moseley, B., Wang, W., Whitehouse, J.: Optimal resource allocation for elastic and inelastic jobs. In: Proceedings of the 32nd ACM Symposium on Parallelism in Algorithms and Architectures. SPAA ’20, pp. 75–87 (2020)

  41. Brill, P.H., Green, L.: Queues in which customers receive simultaneous service from a random number of servers: a system point approach. Manag. Sci. 30(1), 51–68 (1984)

    Article  Google Scholar 

  42. Rumyantsev, A., Morozov, E.: Stability criterion of a multiserver model with simultaneous service. Ann. Oper. Res. 252(1), 29–39 (2017)

    Article  Google Scholar 

  43. Hong, Y., Wang, W.: Sharp zero-queueing bounds for multi-server jobs (2021)

  44. Ghaderi, J.: Randomized algorithms for scheduling VMs in the cloud. In: IEEE INFOCOM 2016—The 35th Annual IEEE International Conference on Computer Communications, pp. 1–9 (2016)

  45. Psychas, K., Ghaderi, J.: Randomized algorithms for scheduling multi-resource jobs in the cloud. IEEE/ACM Trans. Netw. 26(5), 2202–2215 (2018)

    Article  Google Scholar 

  46. Psychas, K., Ghaderi, J.: On non-preemptive VM scheduling in the cloud. Proc. ACM Meas. Anal. Comput. Syst. 1(2), 35–13529 (2017)

    Article  Google Scholar 

  47. Maguluri, S.T., Srikant, R.: Scheduling jobs with unknown duration in clouds. IEEE/ACM Trans. Netw. 22(6), 1938–1951 (2014)

    Article  Google Scholar 

  48. Baccelli, F., Foss, S.: On the saturation rule for the stability of queues. J. Appl. Probab. 32(2), 494–507 (1995)

    Article  Google Scholar 

  49. Foss, S., Konstantopoulos, T.: An overview of some stochastic stability methods. J. Oper. Res. Soc. Jpn. 47(4), 275–303 (2004)

    Google Scholar 

  50. Miyazawa, M.: Rate conservation laws: a survey. Queueing Syst. 15(1), 1–58 (1994)

    Article  Google Scholar 

  51. Grosof, I., Scully, Z., Harchol-Balter, M.: SRPT for multiserver systems. Perform. Eval. 127–128, 154–175 (2018)

    Article  Google Scholar 

  52. Sigman, K., Yao, D.D.: Finite moments for inventory processes. Ann. Appl. Probab. 4, 765–778 (1994)

    Article  Google Scholar 

  53. Scheller-Wolf, A.: Finite moment conditions for stationary content processes with applications to fluid models and queues. PhD thesis, Columbia University (1996)

Download references

Funding

Funding was provided by National Science Foundation (Grant Nos. CMMI-1938909, CSR-1763701) and a Google 2020 Faculty Research Award.

Author information

Authors and Affiliations

  1. Computer Science Department, Carnegie Mellon University, 5000 Forbes Ave, Pittsburgh, PA, 15213, USA

    Isaac Grosof & Mor Harchol-Balter

  2. Tepper School of Business, Carnegie Mellon University, 5000 Forbes Ave, Pittsburgh, PA, 15213, USA

    Alan Scheller-Wolf

Authors
  1. Isaac Grosof
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mor Harchol-Balter
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alan Scheller-Wolf
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Isaac Grosof.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Grosof, I., Harchol-Balter, M. & Scheller-Wolf, A. WCFS: a new framework for analyzing multiserver systems. Queueing Syst 102, 143–174 (2022). https://doi.org/10.1007/s11134-022-09848-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11134-022-09848-6

Keywords

Mathematics Subject Classification

Search

Navigation