Learning-Based Transmission Scheduling Over USV-Oriented Maritime Communication Networks for Tracking Maintenance

In maritime Internet of Things (IoT), unmanned surface vehicles (USVs) can replace humans to perform some dangerous and challenging tasks. However, due to complex environment in a remote maritime scenario (e.g., fierce waves and electromagnetic interference) and limited communication resources, determining an optimal transmission schedule for USVs to accomplish tracking tasks efficiently presents a significant challenge. Therefore, in this article, we investigate a strategic approach to solving the transmission scheduling problem for USV tracking in a remote maritime scenario. A novel maritime IoT framework integrating sensing, transmission, and control is proposed. First, a learning-based adaptive Kalman filtering algorithm (LAKF) capable of operating in time-varying noise environments is developed. Second, a novel dueling double deep Q-network tracking (D3QNT) algorithm incorporating the discrete-time kinematic model characteristics of USVs is proposed. The reward mechanism of this algorithm, which is designed based on the estimation of the future states, provides effective solutions for transmission scheduling problems in USV tracking tasks. Building upon this foundation, the scenario has further been extended to that with an infinite temporal dimension. The simulation results show that, compared with other learning-based algorithms in existing literature, our proposed LAKF-D3QNT algorithm has better control performance and significantly reduces the tracking error in infinite-time scenarios.