Robust AI-Assisted Successive Interference Cancellation for Coexistence of RF Sensors and 5G Signals in Shared Spectrum

The coexistence of active 5G communication signals and passive sensors in shared spectrum environments presents significant challenges due to radio frequency interference (RFI). This paper introduces a novel two-stage successive interference cancellation (SIC) architecture leveraging artificial intelligence (AI) to mitigate interference and preserve the integrity of passive sensing. The first stage employs a deep multilayer perceptron (D-MLP) trained with the Levenberg-Marquardt (LM) algorithm to reconstruct dominant active signals. The second stage, powered by a Bayesian Regularization (BR)-trained D-MLP, addresses residual non-linear and weak interference. Together, these stages achieve superior interference cancellation, ensuring robust separation of active and passive signals. The proposed architecture is evaluated in scenarios with varying interference complexities, including single and multiple active sources. Results demonstrate that the AI-assisted SIC framework significantly outperforms conventional methods, effectively reconstructing and removing 5G signals even under challenging conditions, such as low signal-to-noise ratios. The system also showcases adaptability, maintaining high performance when trained on one gain level and tested on another. This research advances the field by providing a scalable and robust solution for enabling reliable spectrum coexistence, particularly for Earth observation and environmental monitoring.