Frame Structure Design and Task Scheduling in 5 G NR for ISAC Systems: A Deep Reinforcement Learning Approach

The frame structure is regarded as one of the implementation concerns for integrated sensing and communications (ISAC). Unlike other solutions, the shared use of new radio (NR) pilots is particularly promising due to their adaptability to existing standards and infrastructure. However, the integration of sensing capabilities introduces new challenges for the pilots and frame structure designs. Generally, the sensing requirements are diverse and dynamically changing, calling for adaptive adjustments to the frame structure. The intricate interplay of various trade-offs between sensing and communication metrics complicates the design of the adaptive frame structure. To address this issue, we introduced a two-timescale frame adjustment and task scheduling (FATS) mechanism leveraging the NR position reference signal (PRS). The prime objective is to maximize the ISAC synergistic rate, a unified metric defined by sensing mutual information (SMI) and the queued transmission rate. Then, a twin agent based FATS (Twin-FATS) algorithm was proposed that synergizes deep Q-network (DQN) and soft actor-critic (SAC) to simultaneously enable the adaptive adjustment of configurable parameters and efficient resource utilization within the frames. Comprehensive simulations were conducted to validate its superiority over other strategies, with ISAC synergistic rate increased by up to 21.95%, and the task completion rate improved by up to 16.3%.