Point Cloud-Based End-to-End Formation Control Using a Two Stage SAC Algorithm

This study develops a novel end-to-end formation strategy for leader-follower formation control of mobile robots that uses onboard LiDAR sensors in non-communication environments. The main contributions of this letter are twofold: Firstly, we propose a point cloud-based LiDAR servoing control method (PCLS) aimed at ensuring mobile robots achieve the predefined formation performance without direct communication. Secondly, an innovative two-stage Soft Actor-Critic (TSSAC) algorithm is presented, specifically designed for end-to-end training of PCLS. This algorithm skillfully combines the strengths of a distance-based agent (serving as a “teacher”) and a point cloud-based agent (serving as a “student”), effectively addressing the issues of slow convergence and insufficient generalization in deep reinforcement learning methods that use high-dimensional features (such as point clouds, images) as inputs. Furthermore, as part of our method, we designed a novel reward function and normalized the point cloud inputs to provide consistent incentives for the agent across diverse formation tasks, thereby facilitating better learning and adaptation to formation tasks in different environments. Finally, through extensive experiments conducted in the Gazebo simulator and real-world environments, we confirmed the effectiveness of the proposed method. Compared to other formation control strategies, our approach relies solely on onboard LiDAR sensors, without the need for additional communication devices, while ensuring excellent transient and steady-state performance.