Crowd Prediction and Autonomous Navigation with Partial Observations

Operating autonomous vehicles safely and efficiently in crowded pedestrian environments is a challenging task, especially with partial observations. This is because autonomous vehicles cannot always determine the current state of the pedestrians and take optimal actions due to limitations of onboard sensors such as visual occlusion. Furthermore, scenarios involving partial observations are normally not included in training data for navigation models as it requires extra hybrid training for various partial observations scenarios. To address this challenge, we propose a novel method that incorporates a transformer-based human multi-modal trajectory prediction model into a deep reinforcement learning framework to determine the optimal driving strategy. Furthermore, we introduce a novel metric and use a subset of observations from an existing pedestrian dataset for the evaluation of crowd prediction and navigation methods under various partial observations scenarios. We also validate the robustness and efficiency of our proposed method through several experiments and compare to the state-of-the-art methods.