Q-Learning-Based Obfuscation Finite State Machine for Hardware IP Protection

Defense against attack methods such as intellectual property (IP) piracy, overproduction, and reverse engineering in the integrated circuit (IC) supply chain has become a significant research focus. To protect hardware IP cores, traditional obfuscation methods hide the original function of the circuit by adding redundant states. However, this approach may lead to a significant increase in the number of flip-flops, resulting in a large hardware overhead. This paper proposes a Q-Learning Finite State Machine (QL-FSM) obfuscation methodology for IP protection, where the inherently unreachable states of the original circuit are utilized as obfuscated states. The scheme first uses the Q-Learning algorithm (QL) to determine the connection relationships of the obfuscated states and automatically generates the Verilog code of the QL-FSM with varying numbers of states. The constructed QL-FSM is then applied to both a benchmark circuit and a hardware cryptographic algorithm circuit, using the key sequence $K$ to control the obfuscated state transitions and protect the hardware IP core. Next, the benchmark circuit and cryptographic algorithm circuit are used to verify the QL-FSM and the hardware IP core. Finally, the effectiveness and low overhead of the QL-FSM obfuscation method are validated through experiments using both the benchmark circuit and the cryptographic algorithm circuit. The experimental results show that when only unreachable states are used, the average area overhead and average power consumption overhead increase by 5.86% and 3.20%, respectively.