Leveraging 4D Voxel Representation to Enhance Sim2Real Transfer in Autonomous Navigation

Traditional visual representations such as depth images and point clouds are highly susceptible to noise in real-world environments, hindering their effectiveness in autonomous navigation tasks. This paper presents a novel visual information representation module designed to enhance the robustness and performance of reinforcement learning for obstacle avoidance and local path planning in dynamic, unstructured environments. We construct a four-dimensional (4D) voxel representation by filtering point clouds, aligning multiple frames into a unified spatiotemporal structure. A 4D sparse convolutional neural network (4D SparseCNN) then encodes this representation for policy learning. Utilizing this 4D voxel representation, we train an agentagnostic obstacle avoidance local path planner via reinforcement learning with curriculum, ensuring robust performance even with the action space constraints inherent in Ackermann-steering models. We leverage large-scale parallel training to accelerate policy optimization and improve training efficiency, enabling effective learning within complex simulated environments. Experimental results in both simulated and real-world scenarios demonstrate that our method significantly improves navigation performance in dynamic, unstructured settings.