Deep reinforcement learning for joint functional split and network function placement in vRAN

The virtualized radio access network (vRAN) placement problem consists of jointly choosing a functional split and the placement of virtualized network functions on vRAN nodes scattered in the network. The most prominent solutions present optimal approaches to solve the problem, but they are computationally expensive for large instances. Non-exact approaches emerge as alternatives to solve the vRAN placement problem, mainly using machine learning, which is largely fostered by the standardization bodies in next-generation networks. In this context, we present an approach to solve the problem using deep reinforcement learning (DRL), where the objective is to jointly minimize the number of computing resources used and maximize the vRAN centralization level. To build our DRL agent, we started from a traditional optimization formulation that guided the agent development inside a conventional DRL framework. We compare our solution with two exact optimization models from the literature, including one that has a DRL solution. Since our proposed design was based on a most advanced optimal model, it was able to outperform one of the exact optimization models and, as a consequence, its DRL agent.