Distributed Inter-cell Interference Coordination for Small Cell Wireless Communications: A Multi-Agent Deep Q-Learning Approach

Densely deployed small cell multiple-input multiple-output (MIMO) systems can potentially improve the system capacity. However, their overlapping and neighboring cells lead to an increase in inter-cell interference (ICI) and thus decrease the system capacity. To suppress such ICI, base stations (BSs) can perform the exhaustive search (ES) in order to find the optimal combination of transmit power levels and beamforming vectors from a pre-defined codebook. However, ES requires a large amount of computational complexity that grows exponentially with the number of BSs. To reduce the complexity, a super-vised learning (SL) scheme for neural networks (NNs) that can approximate ES with much less complexity has been proposed. However, the SL scheme for NNs still needs training data found by ES, which makes it difficult to apply the scheme into systems including many BSs. To cope with such a problem, this paper introduces independent deep Q-learning (IDQL) that can be classified into multi-agent reinforcement learning. The proposed IDQL-based scheme can control both the transmit power and beamforming in a distributed manner. Simulation results show that the proposed IDQL-based scheme can achieve the system capacity close to that of the SL scheme for NNs, even though the proposed scheme does not require any training data.