Skip to main content
SpringerLink
Log in

Work Scheduling in Cloud Network Based on Deep Q-LSTM Models for Efficient Resource Utilization

  • Research
  • Published:
Journal of Grid Computing Aims and scope Submit manuscript

Abstract

Edge computing has emerged as an innovative paradigm, bringing cloud service resources closer to mobile consumers at the network's edge. This proximity enables efficient processing of computationally demanding and time-sensitive tasks. However, the dynamic nature of the edge network, characterized by a high density of devices, diverse mobile usage patterns, a wide range of applications, and sporadic traffic, often leads to uneven resource distribution. This imbalance hampers system efficiency and contributes to task failures. To overcome these challenges, we propose a novel approach known as the DRL-LSTM approach, which combines Deep Reinforcement Learning (DRL) with Long Short-Term Memory (LSTM) architecture. The primary objective of the DRL-LSTM approach is to optimize workload planning in edge computing environments. Leveraging the capabilities of DRL, this approach effectively handles complex and multidimensional workload planning problems. By incorporating LSTM as a recurrent neural network, it captures and models temporal dependencies in sequential data, enabling efficient workload management, reduced service time, and enhanced task completion rates. Additionally, the DRL-LSTM approach integrates Deep-Q-Network (DQN) algorithms to address the complexity and high dimensionality of workload scheduling problems. Through simulations, we demonstrate that the DRL-LSTM approach outperforms alternative approaches regarding service time, virtual machine (VM) utilization, and the rate of failed tasks. The integration of DRL and LSTM enables the process to effectively tackle the challenges associated with workload planning in edge computing, leading to improved system performance. The proposed DRL-LSTM approach offers a promising solution for optimizing workload planning in edge computing environments. Combining the power of Deep Reinforcement Learning, Long Short-Term Memory architecture, and Deep-Q-Network algorithms facilitates efficient resource allocation, reduces service time, and increases task completion rates. It holds significant potential for enhancing the overall performance and effectiveness of edge computing systems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Availability of Data and Material

The datasets used and/or analyzed during the current study are available from the corresponding author upon reasonable request.

References

  1. Zheng, T., Wan, J., Zhang, J., Jiang, C.: Deep reinforcement learning-based workload scheduling for edge computing. J. Cloud Comput. 11(1), 3 (2022)

    Article  Google Scholar 

  2. Zhou, G., Tian, W., Buyya, R.: Deep reinforcement learning-based methods for resource scheduling in cloud computing: A review and future directions. arXiv preprint arXiv:2105.04086 (2021)

  3. Adhikari, M., Amgoth, T., Srirama, S.N.: A survey on scheduling strategies for workflows in cloud environment and emerging trends. ACM Comput. Surv. (CSUR). 52(4), 1–36 (2019)

    Article  Google Scholar 

  4. Zheng, W., Zhou, Y., Liu, S., Tian, J., Yang, B., ..., Yin, L.: A deep fusion matching network semantic reasoning model. Appl. Sci. 12(7), (2022)

  5. Wu, Q., Zhou, M., Zhu, Q., Xia, Y., Wen, J.: MOELS: Multiobjective evolutionary list scheduling for cloud workflows. IEEE Trans. Autom. Sci. Eng. 17(1), 166–176 (2020)

    Article  Google Scholar 

  6. Index, C.G.C., 2018. Forecast and methodology, 2016–2021 white paper. Updated: February, 1.

  7. Zheng, W., Yin, L.: Characterization inference based on joint-optimization of multi-layer semantics and deep fusion matching network. PeerJ Comput. Sci. (2022)

  8. Kardani-Moghaddam, S., Buyya, R., Ramamohanarao, K.: Adrl: A hybrid anomaly-aware deep reinforcement learning-based resource scaling in clouds. IEEE Trans. Parallel Distrib. Syst. 32(3), 514–526 (2020)

    Article  Google Scholar 

  9. Rjoub, G., Bentahar, J., Wahab, O.A.: BigTrustScheduling: Trust-aware big data task scheduling approach in cloud computing environments. Futur. Gener. Comput. Syst. 110, 1079–1097 (2020)

    Article  Google Scholar 

  10. Wang, Y., Han, X., Jin, S.: MAP based modeling method and performance study of a task offloading scheme with time-correlated traffic and VM repair in MEC systems. Wireless Netw. (2022)

  11. Tong, Z., Chen, H., Deng, X., Li, K., Li, K.: A scheduling scheme in the cloud computing environment using deep Q-learning. Inf. Sci. 512, 1170–1191 (2020)

    Article  Google Scholar 

  12. Ilager, S., Ramamohanarao, K., Buyya, R.: Thermal prediction for efficient energy management of clouds using machine learning. IEEE Trans. Parallel Distrib. Syst. 32(5), 1044–1056 (2020)

    Article  Google Scholar 

  13. Zhang, J., Tang, Y., Wang, H., Xu, K.: ASRO-DIO: Active subspace random optimization based depth inertial odometry. IEEE Trans. Robot. 1–13 (2022)

  14. Yuan, H., Yang, B.: System dynamics approach for evaluating the interconnection performance of cross-border transport infrastructure. J. Manage. Eng. 38(3), (2022)

  15. Monge, D.A., Pacini, E., Mateos, C., Alba, E., Garino, C.G.: CMI: An online multi-objective genetic autoscaler for scientific and engineering workflows in cloud infrastructures with unreliable virtual machines. J. Netw. Comput. Appl. 149, 102464 (2020)

    Article  Google Scholar 

  16. Li, J., Deng, Y., Sun, W., Li, W., Li, R., Li, Q., ..., Liu, Z.: Resource orchestration of cloud-edge–based smart grid fault detection. ACM Trans. Sen. Netw. 18(3), (2022)

  17. Priya, V., Kumar, C.S., Kannan, R.: Resource scheduling algorithm with load balancing for cloud service provisioning. Appl. Soft Comput. 76, 416–424 (2019)

    Article  Google Scholar 

  18. Lu, S., Ban, Y., Zhang, X., Yang, B., Liu, S., Yin, L., Zheng,: Adaptive control of time delay teleoperation system with uncertain dynamics. Front. Neurorobot. (2022)

  19. Li, B., Zhou, X., Ning, Z., Guan, X., Yiu, K.: C, Dynamic event-triggered security control for networked control systems with cyber-attacks: A model predictive control approach. Inf. Sci. 612, 384–398 (2022)

    Article  Google Scholar 

  20. Zhang, H., Mi, Y., Fu, Y., Liu, X., Zhang, Y., Wang, J., ..., Tan, J.: Security defense decision method based on potential differential game for complex networks. Comput. Secur. 129, (2023)

  21. Ren, H., Wang, Y., Xu, C., Chen, X.: Smig-rl: An evolutionary migration framework for cloud services based on deep reinforcement learning. ACM Trans. Internet Technol. (TOIT) 20(4), 1–18 (2020)

    Article  Google Scholar 

  22. Zhang, X., Pan, W., Scattolini, R., Yu, S., Xu, X.: Robust tube-based model predictive control with Koopman operators. Automatica. 137, (2022)

  23. Bianchini, R., Fontoura, M., Cortez, E., Bonde, A., Muzio, A., Constantin, A.M., Moscibroda, T., Magalhaes, G., Bablani, G., Russinovich, M.: Toward ml-centric cloud platforms. Commun. ACM 63(2), 50–59 (2020)

    Article  Google Scholar 

  24. Ilager, S., Muralidhar, R., Buyya, R.: Artificial intelligence (ai)-centric management of resources in modern distributed computing systems. In: 2020 IEEE Cloud Summit, (pp. 1–10). IEEE (2020, October)

  25. Rodrigues, T.K., Suto, K., Nishiyama, H., Liu, J., Kato, N.: Machine learning meets computation and communication control in evolving edge and cloud: Challenges and future perspective. IEEE Commun. Surv. Tutor. 22(1), 38–67 (2020)

    Article  Google Scholar 

  26. Han, S., Ding, H., Zhao, S., Ren, S., Wang, Z., Lin, J., ..., Zhou, S.: Practical and robust federated learning with highly scalable regression training. IEEE Trans. Neural Netw. Learn. Syst. (2023)

  27. Yao, Y., Shu, F., Li, Z., Cheng, X., Wu, L.: Secure transmission scheme based on joint radar and communication in mobile vehicular networks. IEEE Trans. Intell. Transport. Syst. (2023)

  28. Zheng, Y., Lv, X., Qian, L., Liu, X.: An optimal BP neural network track prediction method based on a GA–ACO hybrid algorithm. J. Mar. Sci. Eng. 10(10), (2022)

  29. Zheng, Y., Liu, P., Qian, L., Qin, S., Liu, X., Ma, Y., ..., Cheng, G.: Recognition and depth estimation of ships based on binocular stereo vision. J. Mar. Sci. Eng. 10, (2022)

  30. Karthiban, K., Raj, J.S.: An efficient green computing fair resource allocation in cloud computing using modified deep reinforcement learning algorithm. Soft. Comput. 24(19), 14933–14942 (2020)

    Article  Google Scholar 

  31. Qian, L., Zheng, Y., Li, L., Ma, Y., Zhou, C., ..., Zhang, D.: A New method of inland water ship trajectory prediction based on long short-term memory network optimized by genetic algorithm. Appl. Sci. 12(8), 4073 (2022)

  32. Zhang, X., Fang, S., Shen, Y., Yuan, X., Lu, Z.: Hierarchical velocity optimization for connected automated vehicles with cellular vehicle-to-everything communication at continuous signalized intersections. IEEE Trans. Intell. Transport. Syst. (2023)

  33. Haq, D.Z., Novitasari, D.C.R., Hamid, A., Ulinnuha, N., Farida, Y., Nugraheni, R.D., Nariswari, R., Rohayani, H., Pramulya, R., Widjayanto, A.: Long short-term memory algorithm for rainfall prediction based on El-Nino and IOD data. Procedia Comput. Sci. 179, 829–837 (2021)

    Article  Google Scholar 

  34. Zhao, K., Jia, Z., Jia, F., Shao, H.: Multi-scale integrated deep self-attention network for predicting remaining useful life of aero-engine. Eng. Appl. Artif. Intell. 120, (2023)

  35. Chen, P., Liu, H., Xin, R., Carval, T., Zhao, J., Xia, Y., ..., Zhao, Z.: Effectively detecting operational anomalies in large-scale IoT data infrastructures by using a GAN-Based predictive model. Comput. J. 65(11), 2909–2925 (2022)

  36. Zhou, X., Zhang, L.: SA-FPN: An effective feature pyramid network for crowded human detection. Appl. Intell. 52(11), 12556–12568 (2022)

    Article  Google Scholar 

  37. Xiao, Z., Shu, J., Jiang, H., Lui, J. C. S., Min, G., Liu, J., ..., Dustdar, S.: Multi-objective parallel task offloading and content caching in D2D-aided MEC networks. IEEE Trans. Mob. Comput. (2022)

  38. Dai, X., Xiao, Z., Jiang, H., Alazab, M., Lui, J. C. S., Min, G., ..., Liu, J.: Task offloading for cloud-assisted fog computing with dynamic service caching in enterprise management systems. IEEE Trans. Ind. Inf. 19(1), 662–672 (2023)

  39. Dai, X., Xiao, Z., Jiang, H., Alazab, M., Lui, J. C. S., Dustdar, S., ..., Liu, J.: Task co-offloading for D2D-assisted mobile edge computing in industrial internet of things. IEEE Trans. Ind. Inf. 19(1), 480–490 (2023)

  40. Iang, H., Dai, X., Xiao, Z., Iyengar, A.K.: Joint task offloading and resource allocation for energy-constrained mobile edge computing. IEEE Trans. Mob. Comput. (2022)

  41. Xiao, Z., Shu, J., Jiang, H., Min, G., Chen, H., ..., Han, Z.: Perception task offloading with collaborative computation for autonomous driving. IEEE J. Sel. Areas Commun. 41(2), 457–473 (2023)

  42. Dai, X., Xiao, Z., Jiang, H., Lui, J.C.S.: UAV-assisted task offloading in vehicular edge computing networks. IEEE Trans. Mob. Comput. (2023)

  43. Dai, X., Xiao, Z., Jiang, H., Chen, H., Min, G., Dustdar, S., ..., Cao, J.: A learning-based approach for vehicle-to-vehicle computation offloading. IEEE Internet Things J. 10(8), 7244–7258 (2023)

  44. Peng, Z., Hu, J., Shi, K., Luo, R., Huang, R., Ghosh, B.K., ..., Huang, J.: A novel optimal bipartite consensus control scheme for unknown multi-agent systems via model-free reinforcement learning. Appl. Math. Comput. 369, (2020)

  45. Ma, J., Hu, J.: Safe consensus control of cooperative-competitive multi-agent systems via differential privacy. Kybernetika 58(3), 426–439 (2022)

    MathSciNet  Google Scholar 

  46. Peng, Y., Zhao, Y., Hu, J.: On the role of community structure in evolution of opinion formation: a new bounded confidence opinion dynamics. Inf. Sci. 621, 672–690 (2023)

    Article  Google Scholar 

  47. Min, H., Li, Y., Wu, X., Wang, W., Chen, L., ..., Zhao, X.: A measurement scheduling method for multi-vehicle cooperative localization considering state correlation. Veh. Commun. (2023)

  48. Liang, X., Huang, Z., Yang, S., Qiu, L.: Device-free motion & trajectory detection via RFID. ACM Trans. Embed. Comput. Syst. 17(4), (2018)

  49. Xie, Y., Wang, X., Shen, Z., Sheng, Y., Wu, G.: A two-stage estimation of distribution algorithm with heuristics for energy-aware cloud workflow scheduling. IEEE Trans. Serv. Comput. (2023)

  50. Li, Z., Kong, Y., Jiang, C.: A Transfer Double Deep Q Network Based DDoS Detection Method for Internet of Vehicles. IEEE Trans. Veh. Technol. 72(4), 5317–5331 (2023)

    Article  Google Scholar 

  51. Liu, C., Wu, T., Li, Z., Ma, T., Huang, J.: Robust online tensor completion for IoT streaming data recovery. IEEE Trans. Neural Netw. Learn. Syst. (2022)

  52. Cheng, B., Zhu, D., Zhao, S., Chen, J.: Situation-aware IoT service coordination using the event-driven SOA paradigm. IEEE Trans. Netw. Serv. Manage. 13(2), 349–361 (2016)

    Article  Google Scholar 

Download references

Funding

This work was sponsored in part by National Natural Science Foundation of China (2345678).

Author information

Authors and Affiliations

  1. School of Intelligent and Electronic Engineering, Dalian Neusoft University of Information, Dalian, 116023, China

    Yanli Xing

Authors
  1. Yanli Xing
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Yanli Xing: Conceptualization, Methodology, Formal analysis, Supervision, Writing—original draft, Writing—review & editing.

Corresponding author

Correspondence to Yanli Xing.

Ethics declarations

Ethics Approval and Consent to Participate

Not applicable.

Consent for Publication

Not applicable.

Competing Interests

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xing, Y. Work Scheduling in Cloud Network Based on Deep Q-LSTM Models for Efficient Resource Utilization. J Grid Computing 22, 36 (2024). https://doi.org/10.1007/s10723-024-09746-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10723-024-09746-6

Keywords

Search

Navigation