Locate-Then-Delineate: A Free-Text Report Guided Approach for Pneumothorax Segmentation in Chest Radiographs

We present a novel solution for accurate segmentation of pneumothorax from chest radiographs utilizing free-text radiology reports. Our solution employs text-guided attention to leverage the findings in the report to initially produce a low-dimensional region-localization map. These prior region maps are integrated at multiple scales in an encoder-decoder segmentation framework via dynamic affine feature map transform (DAFT). Extensive experiments on a public dataset CANDID-PTX, show that the integration of free-text reports significantly reduces the false positive predictions, while the DAFT-based fusion of localization maps improves the positive cases. Our method achieves a DSC of 0.60 for positive and 0.052 FPR for positive and negative cases, respectively, and 0.70 to 0.85 DSC for medium and large pneumothoraces.