Deep Reinforcement Learning-based Beamforming Design in ISAC-assisted Vehicular Networks

Integrated sensing and communication (ISAC) will become an important feature in the future wireless communication networks. To achieve the high-rate communication and high-precision sensing, joint design of beamforming and power allocation is essential in the ISAC-assisted vehicular networks. This paper studies the downlink ISAC-assisted beamforming design and power allocation in the vehicle-to-infrastructure (V2I) communication networks to maximize the achievable sum-rate while guaranteeing the targeted sensing accuracy. The Cramer-Rao lower bounds (CRLBs) are introduced to characterize the estimation performance of the angle and distance between the target vehicle and the roadside unit (RSU). Due to the non-convex CRLBs sensing constraints, we propose an intelligent scheme based on deep reinforcement learning (DRL) to design the beamforming and allocate the power. In this scheme, the reward function is related to the communication sum-rate and CRLBs sensing constraints. Simulation results show that the proposed scheme significantly improves both communication and sensing performance compared to the benchmark schemes.