Distributed Learning in Multiuser OFDMA Femtocell Networks

This paper elaborates on self-organized and distributed interference management for femtocells that share the available radio resources with macrocells. A multi-agent learning approach is examined, based on distributed Q-learning, where femtocell base stations control their transmit power, such that the femtocell capacity is maximized, while the aggregated downlink interference generated at macro users' receivers is maintained within acceptable limits. The distributed Q-learning algorithm is carried out at the femto nodes, in the way that the interference is controlled at each resource block. The contribution of this work is to integrate multi-user scheduling in the operation of the macrocell network, so that instantaneous changes, with 1 ms granularity, are encountered in the perception that the femtocell agents get of the environment under observation. We demonstrate that, by relying on 3GPP Long Term Evolution (LTE) compliant signalling from the macro network on the intended macrocell scheduling policies, the proposed learning approach allows each femto node to react on these instantaneous changes in the environment, such that the femto-to-macro interference is appropriately controlled.