Hydrodynamics Regularization in Reinforcement Learning for Navigating Crowded Scenarios

The navigation task in dense crowds is a key research problem in real-world scenarios. It requires an agent to avoid collisions in dynamic environments and reach the agent's destination, ensuring high accuracy and efficiency in its decisions. Existing methods typically treat pedestrians as rigid bodies, detect object bounding boxes, and use rigid body dynamics to guide agent behavior. However, in densely crowded scenarios, this approach may lead to suboptimal path planning solutions, thereby imposing more stringent constraints on the agent's action space. In some real-world navigation scenarios, pedestrians can avoid collisions with minor posture adjustments without having to change their direction. In this letter, we propose Hydrodynamics Regularization to address the challenges posed by the modeling of pedestrians in dense crowd environments. This method treats pedestrians as deformable bodies and leverages fluid dynamics equations to compute socially compliant velocities and postures for the agent in dense crowds. By introducing soft constraints on the action space, it effectively reduces collisions between the agent and surrounding obstacles. We validate the effectiveness of Hydrodynamics Regularization in reinforcement learning navigation problems with partially observable Markov processes. Extensive experiments demonstrate that Hydrodynamics Regularization effectively mitigates collisions caused by the modeling approach, especially in dense crowd environments, allowing the navigation agent to achieve higher success rates, fewer collisions, and shorter completion times. The Hydrodynamics Regularization module is a plug-and-play component that can be seamlessly integrated into any reinforcement learning algorithm, demonstrating excellent generalizability.