Q-learning-based multirate transmission control scheme for RRM in multimedia WCDMA systems

In this paper, a Q-learning-based multirate transmission control (Q-MRTC) scheme for radio resource management in multimedia wide-band code-division multiple access (WCDMA) communication systems is proposed. The multirate transmission control problem is modeled as a Markov decision process where the transmission cost is defined in terms of the quality-of-service (QoS) parameters for enhancing spectrum utilization subject to QoS constraint. We adopt a real-time reinforcement learning algorithm, called Q-learning, to accurately estimate the transmission cost for the MRTC. In the meantime, we successfully employ the feature extraction method and radial basis function network (RBFN) for the Q-function that maps the original state space into a feature vector that represents the resultant interference profile. The state space and memory-storage requirement are then reduced and the convergence property of the Q-learning algorithm is improved. Simulation results show that the Q-MRTC for a multimedia WCDMA system can achieve higher system throughput by an amount of 80% and better users' satisfaction than the interference-based MRTC scheme, while the QoS requirements are guaranteed. Also, compared to the table-lookup method, the storage requirement is reduced by 41%.