MedPipe: End-to-End Joint Search of Data Augmentation and Neural Architecture for 3D Medical Image Classification

Data augmentation plays a crucial role in deep learning-based medical imaging analysis, but manually designing tailored data augmentation strategies for each dataset is impractical. Although automatic data augmentation (ADA) techniques have been explored, they often focus solely on data augmentation without considering the importance of neural architecture. Similarly, neural architecture search (NAS) methods mainly concentrate on optimizing the neural architecture while overlooking the impact of data augmentation. However, both data augmentation and neural architecture are interrelated and should be considered together. The joint optimization of data augmentation and neural architecture can lead to improved model performance by harnessing the complementary effects of customized data augmentation strategies and compatible neural architectures. Despite this, the seamless integration of data augmentation and neural architecture search remains under-explored. To address this research gap, we propose MedPipe, an approach that enables end-to-end joint search of data augmentation and neural architecture. We introduce a compact data augmentation search space and unify data augmentation and neural architecture into a cohesive network. This allows simultaneous exploration, optimizing their synergy for enhanced performance. Experimental evaluation on nine medical datasets highlights the necessity of the joint search for data augmentation and neural architecture, demonstrating the superior performance of our approach. Our work opens up possibilities for future applications in diverse medical domains.