Peeking Into the Black Box: A Tutorial on Automated Design Optimization and Parameter Search

Automated optimization algorithms are extensively used to search for optimal design parameters in applications ranging from designing compilers and analog circuits to crafting embedded machine-learning technology. System optimization is particularly laborious as the pertinent design space includes various conflicting objectives and a swarm of free parameters. Automating parameter optimization can enhance system quality while ensuring low design cost and high performance. In this tutorial, we review several recent optimization tools for selecting design parameters and architecture. These tools include heuristic methods, reinforcement learning (RL), and evolutionary strategies. We elaborate on the basics of each algorithm and explain the benefits and tradeoffs when each is applied to system design. To demonstrate how these methods can be applied, we review several realworld examples: analog circuit design, neural network compression, and the design of domain-specific accelerators. We also touch on possible research directions and remaining challenges that need to be addressed.