Intelligent Omni-Surface-Aided Integrated Sensing and Communications Based on Deep Reinforcement Learning With Knowledge Transfer

This paper investigates an intelligent omni-surface (IOS)-assisted integrated sensing and communication (ISAC) system, where a base station provides both target sensing and communication services with an IOS. The sensing signal-to-noise ratio (SNR) is maximized while satisfying the communication requirement by optimizing IOS configurations. Conventional approaches typically need real-time and accurate channel state information (CSI) and have high computational complexity, making them difficult to implement in realistic systems. To circumvent this problem, this paper puts forth a new framework based on deep reinforcement learning (DRL) with knowledge transfer. In particular, an online learning scheme called Deep reinforcement learning IOS-ISAC (DeepOSC), is first proposed to optimize the reflecting and refracting coefficients of the IOS. Thereafter, to enable powerful reasoning and fast decision-making, we incorporate an echo state network (ESN) with separate output into DeepOSC. To further accelerate convergence, two transfer learning approaches, namely staged policy reuse (SPR) and staged policy distillation (SPD), are developed to guide the learning process of a newly deployed agent by leveraging policies of pre-trained agents. Numerical results show that compared to various benchmarks, DeepOSC attains significant sensing and communication performance gains and is more robust against outdated CSI coefficients. In addition, in comparison to conventional neural networks, ESN shortens the run-time of DeepOSC by more than ten times and is more efficient for temporal inference. Besides, we demonstrate the capabilities of SPR and SPD in accelerating the convergence of DeepOSC.