Deep Reinforcement Learning Based Coexistence Management in LPWAN

This paper addresses the coexistence problem in Low-Power Wide-Area Networks (LPWANs), focusing on LoRa, a leading technology in this domain. Current LPWANs lack effective mechanisms to handle coexistence, especially in urban areas where numerous networks and devices may be operating in the limited spectrum. Existing approaches, including collision resolution techniques and adaptations from other wireless technologies like WiFi, are inadequate due to LPWANs' unique characteristics, such as long-range communication and severe energy constraints. Existing learning based approach addresses this issue through embedded Q-learning at low-power for LoRa nodes, imposing computation and energy overhead for them and limiting the network's learning capability by using simple Q-table learning at individual nodes. We propose a novel system design leveraging the computational capabilities of LoRa Network Servers (LNS) for coexistence management. By offloading learning and computation tasks to LNS, the proposed framework employs deep Q-learning, a powerful reinforcement learning technique, to adapt dynamically to complex coexistence scenarios. By exploiting the LNS's global view of the communication channels, our framework enables more effective learning and decision-making compared to decentralized approaches. Furthermore, our system design seamlessly integrates with traditional LoRaWAN infrastructure, imposing minimal overhead on low-power nodes. We evaluate our approach through physical experiments and large-scale simulations in NS-3, considering various coexistence scenarios for a LoRa network. Our results show that, in comparison with the state-of-the-art decentralized learning method, our scheme achieves up to 70.97%, 62.91%, and 47.01% of improvement in packet reception rate, energy per packet, and average transmission attempts per packet, respectively.