Advanced Reinforcement Learning Solution for Clock Skew Engineering: Modified Q-Table Update Technique for Peak Current and IR Drop Minimization

This paper discloses a Reinforcement Learning (RL) solution implemented to decrease the peak current by alteration of the clock skews. Clock skews are elements of the clock network calculated throughout the Clock Tree Synthesis (CTS) phase of physical design. Initially, the physical design tools targeted obtaining a balanced clock tree and decreasing the clock skew as low as possible. The resulting zero-skew clock tree caused a drastic increase in the current demand for the battery. The proposed solutions in this paper comprise a Reinforcement Learning agent that maneuvers throughout the design and updates the clock arrival time of each register by either adding, removing, or not changing it. The agent’s end game is to maximize the clock arrival distribution of the design. The Reinforcement learning solution allows the exploration and optimization of the clock tree synthesis process beyond the heuristic algorithms employed by traditional Electronic Design Automation (EDA) tools. This paper contains two experiments using the Reinforcement Learning algorithm. The first experiment results indicate a 35% reduction in peak current and a significant reduction in IR decrease (from package to transistor) in the chosen benchmarks. The second experiment modified the Q-table renewing technique, which resulted in another additional 10% improvement compared to the first experiment. In both experiments, the agent traverses the environment and explores different options despite creating timing violations and obtaining a substantial negative feedback reward for the actions taken. However, the timing violation fixed later results in the agent obtaining a future reward for modifying the clock arrival time of other registers. The overall process resulted in the broader spread of clock arrival distribution.