Domain-Specific Knowledge Distillation for Melanoma Depth in Dermatological Imaging

Melanoma, a highly aggressive form of skin cancer, is primarily assessed by its depth of invasion (Breslow thickness), a key prognostic factor influencing clinical management. Deep learning has shown promising results in dermatological image analysis, yet its effectiveness is often hindered by overfitting due to limited and imbalanced datasets. Knowledge Distillation (KD) has emerged as a potential solution, enabling the transfer of knowledge from a large, complex teacher model to a smaller, more efficient student model. However, the impact of using domain-specific datasets for distillation remains underexplored in medical imaging.In this study, we investigate the role of dataset specificity in knowledge distillation for melanoma depth classification. We compare two KD approaches: one using a general-purpose dataset (ImageNet) and another using a dermatology-specific dataset. Performance is evaluated through classification metrics, feature visualization techniques (UMAP and t-SNE), Grad-CAM interpretability maps, and robustness tests under image perturbations. Our findings reveal that while ImageNet-distilled models achieve slightly better classification performance, the dermatology-specific distillation provides more structured feature representations, suggesting improved task-specific knowledge retention. Additionally, models trained with domain-specific KD demonstrate enhanced resilience to real-world variations, such as image noise and brightness changes.These results highlight the potential of tailored knowledge distillation strategies in medical imaging, suggesting that leveraging domain-specific datasets can improve model interpretability and robustness while mitigating overfitting. Future work will explore hybrid distillation techniques and domain adaptation strategies to further enhance deep learning applications in dermatological oncology.