Deep Reinforcement Learning (DRL)-based control shows enhanced performance in the management of integrated energy systems when compared with Rule-Based Controllers (RBCs), but it still lacks scalability and generalisation due to the necessity of using tailored models for the training process. Transfer Learning (TL) is a potential solution to address this limitation. However, existing TL applications in building control have been mostly tested among buildings with similar features, not addressing the need to scale up advanced control in real-world scenarios with diverse energy systems. This paper assesses the performance of an online heterogeneous TL strategy, comparing it with RBC and offline and online DRL controllers in a simulation setup using EnergyPlus and Python. The study tests the transfer in both transductive and inductive settings of a DRL policy designed to manage a chiller coupled with a Thermal Energy Storage (TES). The control policy is pre-trained on a source building and transferred to various target buildings characterised by an integrated energy system including photovoltaic and battery energy storage systems, different building envelope features, occupancy schedule and boundary conditions (e.g., weather and price signal). The TL approach incorporates model slicing, imitation learning and fine-tuning to handle diverse state spaces and reward functions between source and target buildings. Results show that the proposed methodology leads to a reduction of 10% in electricity cost and between 10% and 40% in the mean value of the daily average temperature violation rate compared to RBC and online DRL controllers. Moreover, online TL maximises self-sufficiency and self-consumption by 9% and 11% with respect to RBC. Conversely, online TL achieves worse performance compared to offline DRL in either transductive or inductive settings. However, offline Deep Reinforcement Learning (DRL) agents should be trained at least for 15 episodes to reach the same level of performance as the online TL. Therefore, the proposed online TL methodology is effective, completely model-free and it can be directly implemented in real buildings with satisfying performance.

与基于规则的控制器 (RBC) 相比，基于深度强化学习 (DRL) 的控制在综合能源系统的管理方面显示出增强的性能，但由于需要在训练过程中使用定制模型，因此它仍然缺乏可扩展性和泛化性。迁移学习（TL）是解决这一限制的潜在解决方案。然而，现有的 TL 应用在楼宇控制中大多是在具有相似特征的楼宇中进行测试，并未解决在具有不同能源系统的现实场景中扩展高级控制的需求。本文评估了在线异构 TL 策略的性能，并在使用 EnergyPlus 和 Python 的模拟设置中将其与 RBC 以及离线和在线 DRL 控制器进行了比较。该研究测试了 DRL 策略在传导和感应设置中的传输，该策略旨在管理与热能存储 (TES) 相结合的冷水机组。控制策略在源建筑物上进行预训练，并转移到以集成能源系统为特征的各种目标建筑物，包括光伏和电池储能系统、不同的建筑围护结构特征、占用时间表和边界条件（例如天气和价格信号）。TL 方法结合了模型切片、模仿学习和微调来处理源建筑和目标建筑之间的不同状态空间和奖励函数。结果表明，与 RBC 和在线 DRL 控制器相比，所提出的方法可降低 10% 的电力成本，并将日平均温度违规率平均值降低 10% 至 40%。此外，在线 TL 相对于 RBC 可以实现 9% 和 11% 的自给自足和自消费最大化。相反，在传导或归纳设置中，与离线 DRL 相比，在线 TL 的性能较差。然而，离线深度强化学习（DRL）代理应该至少训练 15 个episode，才能达到与在线 TL 相同的性能水平。因此，所提出的在线 TL 方法是有效的、完全无模型的，并且可以直接在真实建筑物中实施，并且具有令人满意的性能。