PillarFlowNet: A Real-time Deep Multitask Network for LiDAR-based 3D Object Detection and Scene Flow Estimation

Mobile robotic platforms require a precise understanding about other agents in their surroundings as well as their respective motion in order to operate safely. Scene flow in combination with object detection can be used to achieve this understanding. Together, they provide valuable cues for behavior prediction of other agents and thus ultimately are a good basis for the ego-vehicle's behavior planning algorithms. Traditionally, scene flow estimation and object detection are handled by separate deep networks requiring immense computational resources. In this work, we propose PillarFlowNet, a novel method for simultaneous LiDAR scene flow estimation and object detection with low latency and high precision based on a single network. In our experiments on the KITTI dataset, PillarFlowNet achieves a 16.3 percentage points higher average precision score as well as a 21.4 % reduction in average endpoint error for scene flow compared to the state-of-the-art in multitask LiDAR object detection and scene flow estimation. Furthermore, our method is significantly faster than previous methods, making it the first to be applicable for real-time systems.