As the next-generation battery substitute for IoT system, energy harvesting (EH) technology revolutionizes the IoT industry with environmental friendliness, ubiquitous accessibility, and sustainability, which enables various self-sustaining IoT applications. However, due to the weak and intermittent nature of EH power, the performance of EH-powered IoT systems as well as its collaborative routing mechanism can severely deteriorate, rendering unpleasant data package loss during each power failure. Such a phenomenon makes conventional routing policies and energy allocation strategies impractical. Given the complexity of the problem, reinforcement learning (RL) appears to be one of the most promising and applicable methods to address this challenge. Nevertheless, although the energy allocation and routing policy are jointly optimized by the RL method, due to the energy restriction of EH devices, the inappropriate configuration of multi-hop network topology severely degrades the data collection performance. Therefore, this article first conducts a thorough mathematical discussion and develops the topology design and validation algorithm under energy harvesting scenarios. Then, this article develops DeepIoTRouting, a distributed and scalable deep reinforcement learning (DRL)-based approach, to address the routing and energy allocation jointly for the energy harvesting powered distributed IoT system. The experimental results show that with topology optimization, DeepIoTRouting achieves at least 38.71% improvement on the amount of data delivery to sink in a 20-device IoT network, which significantly outperforms state-of-the-art methods.

作为物联网系统的下一代电池替代品，能量收集（EH）技术以其环境友好性、无处不在的可访问性和可持续性彻底改变了物联网行业，使各种自我维持的物联网应用成为可能。然而，由于 EH 电源的微弱和间歇性特性，EH 供电的物联网系统的性能及其协作路由机制可能会严重恶化，在每次电源故障期间都会导致令人不快的数据包丢失。这种现象使得传统的路由策略和能量分配策略变得不切实际。考虑到问题的复杂性，强化学习（RL）似乎是应对这一挑战最有前途和最适用的方法之一。然而，虽然RL方法联合优化了能量分配和路由策略，但由于EH设备的能量限制，多跳网络拓扑的不适当配置严重降低了数据收集性能。因此，本文首先进行了彻底的数学讨论，并开发了能量收集场景下的拓扑设计和验证算法。然后，本文开发了DeepIoTRouting，这是一种基于分布式、可扩展的深度强化学习 (DRL) 的方法，旨在共同解决能量收集驱动的分布式物联网系统的路由和能量分配问题。实验结果表明，通过拓扑优化，DeepIoTRouting在 20 台设备的物联网网络中向接收器传输的数据量至少提高了 38.71%，这明显优于最先进的方法。