Multi-Task Breast Ultrasound Image Classification and Segmentation Using Swin Transformer and VMamba Models

Breast cancer represents a significant women's health issue worldwide. Ultrasound imaging is a critical technique for early detection. Additionally, AI-based methods are proving to be crucial. This research compares deep learning methods such as VGG-16, ResNet-50, Swin Transformer, and VMamba in classifying breast ultrasound images as benign or malignant and for segmentation tasks. Notably, this research is the pioneer in utilizing the VMamba model for both the classification and segmentation of breast ultrasound images. Additionally, we have developed a multi-task learning framework that simultaneously produces classification and segmentation results compatible with all underlying networks. This work is a notable advancement in the application of AI within medical imaging, highlighting its potential in early cancer diagnosis and advocating for a combined approach to improve outcomes. We benchmark four existing deep learning techniques in breast ultrasound image analysis and multi-task learning. Our code is available at https://github.com/kuanhuang0624/buscseg.