Lidar With Velocity: Correcting Moving Objects Point Cloud Distortion From Oscillating Scanning Lidars by Fusion With Camera

Lidar point cloud distortion from moving object is an important problem in autonomous driving, and recently becomes more demanding with the emerging of oscillating type lidars, which feature back-and-forth scanning patterns and complex distortions. Accurately correcting the point cloud distortion would not only describe the 3D moving objects more accurately, but also enable accurate estimation of moving objects’ velocities with enhanced prediction and tracking capabilities. A lidar and camera fusion approach is proposed to correct the oscillating lidar distortions with full velocity estimation. Lidar measures the time-of-flight distance accurately in the radial direction but only with sparse angular information while camera as a complementary sensor could provide a dense angular resolution. In addition, the proposed framework utilizes a probabilistic Kalman-filter approach to combine the estimated velocities and track the moving objects with their real-time velocities and correct point clouds. The proposed framework is evaluated on real road data and consistently outperforms other methods. The complete framework is open-sourced to accelerate the adoption of the emerging lidars.