HSBP: A Hybrid Secure Branch Predictor Design Against Transient Execution Attacks

Transient execution attacks exploit speculative execution mechanisms in modern high-performance processors, leaving traces of sensitive data at the microarchitectural level and then stealing secret data through cache side-channel attacks. These attacks break existing security boundaries and are widely present in mainstream processors, posing a serious threat to computer system security. This paper proposes a hybrid secure branch predictor (HSBP) design, for RISC-V processors, against transient execution attacks. Our approach implements a secure branch direction prediction mechanism based on mistraining pattern detection and output randomization in BIM and TAGE, as well as a secure branch target prediction mechanism based on complex indexing and entry encryption in BTB, fundamentally weakening attackers' control over branch prediction results and manipulation of branch prediction behavior. Furthermore, this paper introduces a transient execution attack threat monitor based on hardware performance counters that observes key microarchitectural events, thereby addressing the architectural layer's opacity to microarchitectural attacks and providing effective supplementary defense capabilities. Experimental evaluation demonstrates that the proposed design effectively mitigates typical transient execution attacks such as Spectre-V1 and Spectre-V2, while maintaining an optimal balance among security, performance, and hardware resource utilization, with an average performance overhead of only 2.36% and hardware resource increase of approximately 1.47%.