A Software-Defined MARL-Based Architecture for AUV Cluster Network to Enable Cooperative and Smart Underwater Target Tracking

With the rapid development of underwater network and communication technology (e.g., the underwater acoustic, optical communication technologies), Software-Defined Networking (SDN) has been widely promoted for underwater monitoring network-enabled applications, especially the emerging AUV cluster network-based underwater target tracking. However, to achieve high-performance and smart underwater target tracking, a significant challenge to design an intelligent architecture integrating efficient management and smart determination exists. In this article, based on the above motivation and our previous work at the SDN-enabled underwater monitoring networks, we propose a Software-Defined Multi-Agent Reinforcement Learning (SD-MARL) architecture for the AUV cluster network to combine the similarity between the SDN technology (centralized management, distributed operation) and the CTDE-based MARL architecture (centralized training, decentralized execution). Particularly, for the application with the SD-MARL, we propose an SD-MARL-based underwater target tracking scheme, integrating a cooperative control model, an SD-MARL-based beacon framework, a cooperative target tracking framework, and an interrupted obstacle avoidance framework, respectively. Evaluation results show that our proposed SD-MARL-based underwater target tracking scheme can perform efficient underwater target tracking and better than some traditional MARL architecture in tracking accuracy and underwater communication efficiency.