Digital Twin-Driven Formation Control of ROVs: An Integral Reinforcement Learning-Based Solution

Formation control of remotely operated vehicles (ROVs) has been regarded as the basis of many sophisticated marine missions. However, the high communication energy consumption and weak environment perception ability on ROVs make it challenging to achieve this task. To overcome the above challenge, this article develops a digital twin (DT)-driven formation control approach for ROVs. We first establish a virtual twin model for each ROV by extracting the motion parameters and environment information. With the collected states from ROVs, an integral reinforcement learning (IRL) based formation controller is designed to drive the motion outputs of DT model. After that, the optimal control policy from the DT model is employed to accomplish formation task for each ROV. To reduce the matching error and ensure the formation stability, an IRL-based optimization algorithm is conducted by using the data interaction between DT model and ROVs. Note that the DT-driven formation solution not only can reduce the communication energy consumption by periodically feeding back the real-data of ROVs to the DT model, but also can improve the perception ability of ROVs by reconstructing a virtual twin environment. Finally, experimental results are provided to verify the effectiveness of our solution.