Abstract

In the rapidly evolving field of reinforcement learning, combination of hierarchical and multiagent learning methods presents unique challenges and opens up new opportunities. This paper discusses a combination of multilevel hierarchical learning with subgoal discovery and multiagent reinforcement learning with hindsight experience replay. Combining these approaches leads to the creation of multiagent subgoal hierarchy algorithm (MASHA) that allows multiple agents to learn efficiently in complex environments, including environments with sparse rewards. We demonstrate the results of the proposed approach in one of these environments inside the StarCraft II strategy game, in addition to making comparisons with other existing approaches. The proposed algorithm is developed in the paradigm of centralized learning with decentralized execution, which makes it possible to achieve a balance between coordination and autonomy of agents.

中文翻译：

---

马尔可夫决策过程中协作多智能体强化学习的分层方法

摘要

在快速发展的强化学习领域，分层学习方法和多智能体学习方法的结合提出了独特的挑战，并开辟了新的机遇。本文讨论了多级分层学习与子目标发现和多智能体强化学习与事后经验回放的结合。结合这些方法可以创建多智能体子目标层次算法（MASHA），该算法允许多个智能体在复杂环境（包括奖励稀疏的环境）中高效学习。除了与其他现有方法进行比较之外，我们还在《星际争霸 II》策略游戏中的其中一种环境中展示了所提出方法的结果。所提出的算法是在集中学习和分散执行的范式中开发的，这使得可以实现代理的协调和自治之间的平衡。