Panoptic Segmentation of CT Images of Non-Small Cell Lung Cancer Data

This paper proposes panoptic segmentation to identify regions of interest (RoIs) in medical images where intensity profiles of RoIs have significant overlap and inconsistent across different instances of same type of RoI. Inspired by the proposal of unified architecture and unified segmentation model for multi-task segmentation, namely semantic, instance and panoptic segmentation, we propose a custom panoptic segmentation scheme for the non-small cell lung cancer (NSCLC) dataset. A typical challenge of such dataset is to differentiate image segments representing primary and secondary focus of tumours having non-homogeneous and overlapping intensity profile. In the proposed architecture, outputs of image feature extractor for each 2D CT slice images and the corresponding text based description of the ground truth RoIs (e.g. photo of left lung) are mapped to a transformer-based decoder to generate segmented output. During inference, image along with a task query (e.g. panoptic or instance segmentation) generates test segmentation map. Based on a conservative performance metric introduced in this paper, we have shown that segmentation by the proposed model is better than its close competitors.