A Scalable Reinforcement Learning Framework for Wireless Sensor Networks: Spatiotemporal Cluster-Based Multi-Agent Reinforcement Learning

Wireless Sensor Networks face significant scalability challenges when implementing Reinforcement Learning techniques for adaptive decision-making. Traditional approaches utilizing either fully distributed node-level agents or completely centralized controllers become increasingly inefficient as network size grows. This paper introduces Spatiotemporal Cluster-Based Multi-Agent Reinforcement Learning (SCMARL), a novel framework that addresses these limitations by partitioning the network into clusters based on both geographic proximity and temporal data correlation patterns. SCMARL employs a hierarchical architecture where dedicated Cluster Agents optimize localized policies for member nodes, decomposing the global state-action space into manageable subspaces. Our experimental validation, combining a real-world deployment with large-scale simulations up to 1000 nodes, demonstrates SCMARL’s superior performance over centralized approaches. Results show up to 52.5% reduction in computation time, 112.5% better battery conservation, and 19% improved prediction accuracy at large scales. The framework achieves 37-50% better state space compression ratios across all network sizes, with a critical performance crossover point at approximately 100 nodes. SCMARL enables efficient resource management in large-scale WSNs while maintaining system-wide coherence through its cluster-based coordination mechanism.