Logic Synthesis Tools’ Testing via Configuration Diversification With Combinatorial Multiarmed Bandit

As circuit design complexity increases, ensuring the accuracy and reliability of logic synthesis tools becomes crucial. However, inherent faults can cause synthesis failures, affecting netlist accuracy. Traditional testing approaches rely on code generators, and their effectiveness is highly influenced by generator configurations. For an effective test configuration, it must produce test programs that are both bug-revealing (i.e., capable of triggering bugs) and diverse (i.e., ensuring the detection of various bug types). To achieve this goal, we propose LoSyTe: Logic Synthesis Tools’ Testing via Configuration Diversification with CMAB. LoSyTe conducts comprehensive testing of logic synthesis tools through dynamic optimization of test-program generator configurations. It consists of three main components. The configuration selection component applies a multiarmed bandit algorithm to select and optimize configuration combinations based on test feedback. Test-program vectorization component turns test-programs into feature vectors to quantify test diversity. The equivalence checking component checks the consistency of synthesized netlists with Verilog tests to identify synthesis faults and guide strategy adjustments. We assess LoSyTe on both proprietary commercial logic synthesis tools (i.e., Vivado) and open-source logic synthesis tools (i.e., Yosys). The results show that LoSyTe successfully discovered 20 bugs of four types; all the reported bugs have been confirmed or fixed by developers. Compared to baseline methods, LoSyTe improves detection effectiveness from 70.00% to 466.67%, with 41.18% of the bugs uniquely identified. In addition, the Vivado community highly praises the bug reports submitted by LoSyTe, recognizing their significant contribution to tool improvement.