Learning Task-Level Pseudo-Text Prompt for Improved Medical Image Segmentation

Precise segmentation of important regions in medical images is an extremely challenging task through uni-modal approaches that are solely based on the image modality. This can be improved by introducing additional textual semantic information as witnessed by the numerous text-assisted medical image segmentation methods. In reality, however, the majority of medical data appear as uni-modal data without corresponding text annotations which entail substantial costs. To address this, in this paper, we propose a task-level pseudo-text prompt learning approach for medical image segmentation, which can learn task-level textual semantic information. By integrating task-level keywords with learnable parameters, we can optimize the pseudo-text prompt with the segmentation task. Furthermore, we design an image-text symmetric encoding architecture to extract multilevel image-text feature pairs so that the learned text features can optimize the corresponding visual features at each level. In the testing phase, our model directly uses trained text features to assist visual segmentation thus reducing the model size, and adaptively adjusts the text features through the weight module. Experimental results on three medical image datasets show that our proposed model outperforms other uni-modal segmentation models or text-assisted visual segmentation models with a small model size. Our code can be available at: https://github.com/Rango-bit/PseTNet_code.git.