Deep Reinforcement Learning Based Joint Resource Allocation and Service Migration for Smart-Buoy-Enabled Maritime Multi-Access Edge Computing Networks

Multi-access edge computing (MEC) is a promising technology that can enhance the computational capabilities of maritime Internet of Things (MIoT) systems. By deploying smart buoys equipped with MEC servers (MSs) and base stations (BSs) along fairway boundaries, the distributed computing cluster can significantly improve the timeliness and efficiency in processing latency-sensitive and resource-intensive MIoT applications. However, due to the limited computational resources and service coverage of these smart buoys, mobile MIoT users (MUs), e.g., vessels, may experience degraded access latency and service discontinuity. Resource allocation (RA) and service migration serve as effective techniques for integrating and coordinating the communication, computing, and energy resources of MIoTs. This paper investigates a joint RA and service migration (JRASM) optimization problem for smart-buoy-enabled maritime MEC networks to minimize the long-term task processing latency of MUs while ensuring service continuity. The JRASM optimization addresses the coupled challenges of resource competition, random task generation, and vessel mobility constraints, while integrating collision avoidance mechanisms. Subsequently, the mixed-integer nonlinear programming problem is reformulated as a Markov Decision Process (MDP) and solved by employing deep reinforcement learning (DRL) methods. A novel branching double deep Q network (Branching DDQN)-based JRASM algorithm is designed to decompose the hybrid discrete-continuous action space into independent subspaces, achieving enhanced learning efficiency while reducing parameters by 40%. Simulation results demonstrate that the proposed Branching DDQN-based JRASM algorithm achieves significantly superior performance in terms of latency reduction and task completion rate compared to other DRL-based JRASM baselines, particularly under high-workload scenarios.