Enhancing Skin Lesion Classification with ResNet and ViT: A Study on Feature Combination and Selection

Early detection of skin lesions, including melanoma, is crucial for effective medical intervention. In this study, we compare two cutting-edge deep learning models—ResNet50, a proven Convolutional Neural Network (CNN), and Vision Transformers (ViTs), a newer, transformer-based approach—for multi-class skin lesion classification using the HAM10000 dataset. By extracting features from both models and combining them with techniques like dimensionality reduction Principal Component Analysis (PCA) and Recursive Feature Elimination (RFE), we aim to boost classification accuracy. Our experiments, evaluated through key metrics such as accuracy, precision, recall, and F1-score, reveal valuable insights into the strengths and limitations of CNNs versus ViTs. This work not only advances the field of automated dermatological diagnosis but also paves the way for more refined and accurate skin lesion classifiers in the future.