Improving Timing & Power Trade-Off in Post-Place Optimization Using Multi-Agent Reinforcement Learning

In recent years, post-place optimization has emerged as a critical stage in physical design, aiming to improve power, performance, and area (PPA). Among various optimization techniques, buffer insertion, gate sizing, and $V_{\text {th }}$ assignment have become leading optimization techniques for decades. However, these techniques are traditionally applied sequentially, leading to a critical suboptimality problem. Each technique obtains its own iterations performed step by step, preventing the best-optimal optimization for a target instance. To address this limitation, we propose a novel reinforcement learning (RL) based post-place optimization framework that performs various optimization techniques simultaneously. Moreover, to overcome the persistent headache of chip design, timing and power trade-off, we employ multiple agents that target each specific objective. By leveraging deep RL and graph neural network (GNN), our framework models optimal policies, dynamically selecting the most effective action given a target instance. Consequently, our framework obtained an improved Pareto-frontier set compared to comparison baselines while exhibiting 80 % and 43 % improvements in total negative slack and leakage power, respectively. The results demonstrate that our method outperforms weighted-sum-based co-optimization methods in optimizing timing and power.