Instance Embedding as Queries for DETR-based LiDAR Panoptic Segmentation

LiDAR panoptic segmentation (LPS) offers a comprehensive perception of the driving environment for autonomous vehicles, providing valuable and relevant information for downstream tasks such as scene understanding or 4D panoptic tracking. Current state-of-the-art (SOTA) methods are predominantly based on the detection-transformer (DETR) paradigm. These approaches formulate panoptic segmentation as a problem of predicting the association scores of points within each group, which can be either a thing class instance or a stuff class. To this end, groups are represented by a fixed number of randomly initialized vectors called queries, whose values remain constant during inference. In this approach, queries are required to represent a wide range of objects that present in any particular scene. Recent studies on DETR application in the image domain have attempted to resolve this issue by introducing a strategy for adaptive query generation on a per-scene basis, which has achieved promising results, while in the LiDAR domain, existing DETR-based approaches have yet to explore adaptive mechanisms for queries generation. Motivated by that insight, our work proposes a technique for query representation called clustered feature aggregation (CFA) - a method for computing the features associated with an instance and using them as a query for the segmentation of that instance. To cluster points belonging to the same group, this study introduces shifted point clustering (SPC), an enhancement of traditional clustering techniques, as it secures high-confidence instance accumulation. Our proposed methods, in combination with a strong DETRbased baseline, establish new SOTA results for the panoptic segmentation challenge, outperforming previous top-performers by 1.20% and 1.94%, in PQ metric, on SemanticKITTI and Panoptic Nuscenes datasets, respectively