Distilling Knowledge for Efficient Multiclass Skin Cancer Classification using Lightweight CNN Model

Skin cancer is a critical public health threat due to its high incidence and the significant number of deaths it causes annually, especially when it remains undiagnosed in its early stages. In recent years, deep learning has shown promising results in early skin cancer detection. However, many existing skin cancer classification models are computationally heavy and unsuitable for deployment in low-resource environments. To address this limitation, we propose a knowledge distillation-based approach to develop a lightweight yet high-performing multiclass skin cancer detection model. In this study, we utilize several teacher models, including CNN, VGG16, VGG19, ResNet50, Inception V3, and MobileNetV2, to transfer knowledge to a lightweight CNN student model. We evaluate the performance using the HAM10000 dataset, containing 10,000 dermoscopic images across seven skin cancer types. The temperature parameter, which softens the logits during distillation, is also varied to assess its impact. Our metrics include accuracy, precision, and F1 score. The results show that using CNN as the teacher model, the lightweight CNN student model achieved an accuracy of 98.44%. This was accomplished with a significant reduction in model size, as the student model contains only 260,167 parameters, compared to the teacher’s 1.28 million parameters. This highlights the feasibility of deploying skin cancer classification models in computationally constrained environments without sacrificing performance.