Multi-Task Breast Ultrasound Image Segmentation and Classification Using Convolutional Neural Network and Transformer

Breast ultrasound (BUS) imaging offers a non-invasive and radiation-free method to examine breast tissues. Automated BUS image segmentation and classification can help doctors identify lesions and possible abnormalities early, enabling healthcare professionals to detect breast cancer or other conditions in time for early intervention. In this research, we first conduct a comprehensive performance comparison between transformer networks and convolutional networks; secondly, we propose a novel approach by merging segmentation and classification networks, creating a multitask network tailored explicitly for BUS image segmentation and classification; thirdly, we thoroughly investigate network performance and refine training parameters to prevent overfitting. Finally, we create a user-friendly GUI demo showing our classification and segmentation results. The results demonstrate that the ResNet-50 Multi-Task model exhibits the best overall performance for both segmentation and classification tasks.