Dual-Functional IRS Design for ISAC via Reinforcement Learning in Unknown Environment

Integrated sensing and communications (ISAC) has been envisioned as a pivotal technology in solving the exacerbated spectrum congestion problem. Meanwhile, intelligent reflective surface (IRS) presents a promising avenue for enhancing both communications and sensing in severe obstruction scenarios, such as dense urban area. In this paper, we propose a reinforcement learning (RL)-based IRS-assisted ISAC design that enables simultaneous multi-user downlink communications and multi-target sensing via optimizing the IRS reflection coefficients in unknown high-obstruction scenarios. In order to facilitate the IRS-assisted ISAC design, we first introduce a two-phase method that can achieve precise channel estimation with a small number of pilot signals. Subsequently, we employ the RL to co-design the dual-functional IRS, striving for an optimal performance balance between communications and sensing in unknown environments. Specifically, to address the non-convex problems arising from the RL training, we propose to design the Gramian matrix first, followed by matrix decomposition to obtain the optimal IRS reflection coefficients. Extensive simulations have demonstrated that the two-phase method can accurately estimate the channel, and the IRS-assisted ISAC designed via RL exhibits an excellent dual-functional performance in the unknown environment.