BeFEMTO's self-organized and docitive femtocells

In this paper we use the emerging paradigm of docition for self-organized femtocell networks which are central to the ICT-BeFEMTO project. We consider that the femtocells are intelligent devices implementing a learning process through which they make decisions without the guidance of a centralized entity. In distributed settings, however, the learning may be complex and slow due to coupled decision making processes resulting in non-stationarities. The docitive paradigm proposes a timely solution based on knowledge sharing, which allows femtocells to develop new capacities for selecting appropriate actions. We demonstrate that this improves the femtocells' learning ability and accuracy, and gives them strategies for action selection in unvis-ited states. We evaluate the docitive paradigms in the context of a 3GPP compliant OFDMA (Orthogonal Frequency Division Multiple Access) based femtocell network modeled as a multi-agent system, where the agents implement a real-time multi-agent reinforcement learning technique known as decentralized Q-learning. Our goal is to solve the well known coexistence problem between macro and femto systems by controlling the aggregated interference generated by multiple femtocells at the macrocell receiver. We propose different docitive algorithms and we show their superiority to the well know paradigm of independent learning in terms of speed of convergence and precision.