The complexity of chip design has consistently grown, adhering to Moore’s law. In this paper, we examine a crucial step in integrated circuit design called chip macro placement. Traditionally, human experts are consulted to optimize placement for reduced power consumption, but this requires significant effort. Recently, machine learning-based methods have emerged to address this task, showing promising results. Our work aims to tackle key bottlenecks in current reinforcement learning (RL) based methods, which often suffer from sparse rewards, large state–action search spaces, and unstable training, resulting in suboptimal solutions. To address these issues, we propose a novel approach called Hierarchical Reinforcement Learning for Placement (HRLP). This method distinguishes itself in two ways. First, we introduce a hierarchical RL framework that learns human expert design patterns by assigning agent-specific placement sub-tasks (referred to as options in reinforcement learning) instead of primitive actions at each time step within an episode. Second, we generate dense rewards for each episode by calculating the difference in local wirelength and congestion. We conducted experiments on eight public chip design benchmarks, adaptec and bigblue series. Our model has an average improvement of 9.29 % and 14.25 % on Half Perimeter Wire Length (HPWL), the huge improvement shows that our method is capable of converging to a better sub-optimal than DeepPlace and providing a promising future for AI to aid the design of circuit placement.

中文翻译：

---

集成电路中芯片宏布局的分层强化学习

遵循摩尔定律，芯片设计的复杂性不断增加。在本文中，我们研究了集成电路设计中的一个关键步骤，称为芯片宏布局。传统上，会咨询人类专家来优化布局以降低功耗，但这需要付出巨大的努力。最近，基于机器学习的方法已经出现来解决这一任务，并显示出有希望的结果。我们的工作旨在解决当前基于强化学习（RL）的方法中的关键瓶颈，这些瓶颈通常会受到奖励稀疏、状态动作搜索空间大和训练不稳定的影响，从而导致解决方案不理想。为了解决这些问题，我们提出了一种称为分层强化学习安置（HRLP）的新方法。该方法有两个特点。首先，我们引入了一个分层强化学习框架，该框架通过分配特定于代理的放置子任务（在强化学习中称为选项）而不是在情节中的每个时间步执行原始操作来学习人类专家设计模式。其次，我们通过计算本地线长和拥塞的差异，为每个情节生成密集的奖励。我们在 8 个公共芯片设计基准、adaptec 和 bigblue 系列上进行了实验。我们的模型在半周线长度（HPWL）上平均提高了 9.29% 和 14.25%，巨大的改进表明我们的方法能够收敛到比 DeepPlace 更好的次优值，并为 AI 帮助解决问题提供了广阔的前景。电路布局设计。