Sparse-PointNet: See Further in Autonomous Vehicles

Since the density of LiDAR points reduces significantly with increasing distance, popular 3D detectors tend to learn spatial features from dense points and ignore very sparse points in the far range. As a result, their performance degrades dramatically beyond 50 meters. Motivated by the above problem, we introduce a novel approach to jointly detect objects from multimodal sensor data, with two main contributions. First, we leverage PointPainting [15] to develop a new key point sampling algorithm, which encodes the complex scene into a few representative points with approximately similar point density. Further, we fuse a dynamic continuous occupancy heatmap to refine the final proposal. In addition, we feed radar points into the network, which allows it to take into account additional cues. We evaluate our method on the widely used nuScenes dataset. Our method outperforms all state-of-the-art methods in the far range by a large margin and also achieves comparable performance in the near range.