Beamspace and Frequency Domain ISAC Resource Allocation Using Reinforcement Learning

In the emerging 6G systems, joint sensing and communications require efficient spatial and frequency domain waveform design and resource allocation to meet system performance objectives. Traditional structured optimization algorithms often face practical challenges such as non-convexity, high computational demand, lack of adaptability, and significant performance degradation when the assumed model is invalid or dynamic. Reinforcement learning (RL) offers a data-driven alternative that leverages observed data to overcome these deficits. This paper combines RL principles with acquired model awareness of radio environment dynamics, thereby enhancing the learning data efficiency and making the algorithm more interpretable than traditional model-free RL methods. We introduce an RL algorithm based on Thompson sampling for allocating sub carriers and beams in the beamspace domain to achieve desired performance levels in wireless communications and radar sensing tasks. The proposed RL method effectively learns from its experiences and balances sensing and communications functionalities, achieving superior target detection performance and competitive communication rates compared to traditional methods.