Anti-Jamming 5G Millimeter-Wave Communication via Joint Analog and Digital Beamforming: A Bayesian Optimization Approach

5G millimeter-wave (mmWave) communications are essential for enabling ultra-high-speed, low-latency wireless connectivity to support data-intensive applications. However, the highly directional nature and sensitivity of mmWave signals make them particularly susceptible to jamming attacks. Therefore, securing 5G mmWave communication systems against jamming attacks is critical for ensuring reliable wireless connectivity in mission-critical applications. In this paper, we propose an online Bayesian Optimization (BayOpt) framework for joint analog and digital beamforming optimization at a mmWave communication device, aimed at maximizing its packet decoding rate under a constant jamming attack. By modeling the optimization objective as a black-box function and leveraging online learning to guide beam search, the BayOpt framework efficiently identifies near-optimal beam configurations in both the analog and digital domains while not requiring any knowledge of the jamming strategy or channel conditions. We have implemented the proposed anti-jamming solution on a 28 GHz mmWave testbed and conducted extensive evaluations across four distinct jamming scenarios. Over-the-air experiments demonstrate the effectiveness of the BayOpt framework in suppressing jamming interference. Notably, in a scenario where the jamming signal is 10 dB stronger than the desired signal, the BayOpt-enabled mmWave receiver achieves 73% of the throughput observed in a jamming-free environment.