Confidence-Aware Robust Dynamical Distance Constrained Reinforcement Learning for Social Robot Navigation

Navigating in a crowded social environment without collisions or freezing is a crucial and challenging task. Recent studies have demonstrated considerable success using Deep Reinforcement Learning for social robot navigation. However, the lack of safety guarantees remains a prevalent problem among these methods. To address this limitation, we present a comprehensive safety framework that builds upon a novel multi-agent environment single-agent decision-making formulation and the strengths of Safe Reinforcement Learning (Safe RL) methods. First, considering the uncertainty of pedestrians and the inaccuracy of trajectory predictions, we explicitly embed confidence-weighted uncertainty and risk measurement in the interaction graph network to induce adaptive policies. Next, we replace the state-wise robust-form unshaped cost with a cumulative robust dynamical distance constraint, which establishes a stronger and essential safety objective while ensuring the invariance of the optimal policy. Then we use reachability-based Safe RL to ensure state-wise safety assurance of the feasible part and the risk minimization of the infeasible part. Simulation and virtual-reality evaluation results demonstrate that our proposed method outperforms the state-of-the-art methods in various metrics, such as success rate and collision energy, achieving the best trade-off between safety and efficiency. Our proposed method yields a policy that exhibits desired collision avoidance behavior in both simulation and real-world experiments. Note to Practitioners—The motivation of this paper is to enhance the safety of robots navigating in human-filled environments without being overly conservative. Typical application scenarios include mobile robots navigating through crowds and autonomous vehicles moving through unprotected intersections. On one hand, existing deep reinforcement learning-based navigation methods have shown remarkable capabilities in complex dynamic environments but often lack guaranteed safety constraints. On the other hand, due to the uncertainty introduced by other uncontrollable agents in the environment, unreasonable safety constraints can lead to suboptimal optimization problems, resulting in overly conservative policies. This paper starts from the high-level intrinsic safety objective and proposes a comprehensive safety framework that balances safety and efficiency. For clarity, the proposed approach is structured into several key modules, including confidence-weighted uncertainty, robust dynamical distance constraints, and safe reinforcement learning, among others. It is important to recognize that these modules form a tightly coupled system. We validate the feasibility and effectiveness of our proposed approach through theoretical analysis, simulations, virtual-reality tests, and real-world experiments. In future work, this approach will be extended to incorporate technologies such as sim-to-real transfer and static obstacle handling, enabling more effective deployment in real-world social environments.