Spectral Efficient Two-Stage Beamforming for UAV MIMO under Energy Constraint: A Budgeted Combinatorial MAB-Based Approach

This paper proposes a budgeted combinatorial multi-armed bandit (BC-MAB)-based two-stage beamforming scheme for an energy-constrained unmanned aerial vehicle MIMO system. The proposed scheme learns the pre-beamformer from the historical data which reduces the pilot overhead and uses the bangperbuck ratio to address the energy constraint. In the first stage, the pre-beamformer is designed using a reinforcement learning method based on BC-MAB to maximize spectral efficiency. The total energy and the energy consumption are set as the budget and the cost, respectively. Consider the energy constraint, the bangperbuck ratio is introduced to represent the trade-off between the received energy per slot and the number of communication slots. A pre-beamforming algorithm is proposed to maximize the bangperbuck ratio to achieve a good trade-off and reduce the dimension of instantaneous CSI (ICSI). In the second stage, a multi-user beamforming algorithm is designed to maximize spectral efficiency using the reduced ICSI, where the problem is transformed to convex using Lagrangian transformation and fractional programming. Simulations validate the superiority of the proposed scheme.