Skin Lesion Classification Through Test Time Augmentation and Explainable Artificial Intelligence
Despite significant advancements in the automatic classification of skin lesions using artificial intelligence (AI) algorithms, skepticism among physicians persists. This reluctance is primarily due to the lack of transparency and explainability inherent in these models, which hinders their widespre...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2025-01-01
|
| Series: | Journal of Imaging |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2313-433X/11/1/15 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832588308391133184 |
|---|---|
| author | Loris Cino Cosimo Distante Alessandro Martella Pier Luigi Mazzeo |
| author_facet | Loris Cino Cosimo Distante Alessandro Martella Pier Luigi Mazzeo |
| author_sort | Loris Cino |
| collection | DOAJ |
| description | Despite significant advancements in the automatic classification of skin lesions using artificial intelligence (AI) algorithms, skepticism among physicians persists. This reluctance is primarily due to the lack of transparency and explainability inherent in these models, which hinders their widespread acceptance in clinical settings. The primary objective of this study is to develop a highly accurate AI-based algorithm for skin lesion classification that also provides visual explanations to foster trust and confidence in these novel diagnostic tools. By improving transparency, the study seeks to contribute to earlier and more reliable diagnoses. Additionally, the research investigates the impact of Test Time Augmentation (TTA) on the performance of six Convolutional Neural Network (CNN) architectures, which include models from the EfficientNet, ResNet (Residual Network), and ResNeXt (an enhanced variant of ResNet) families. To improve the interpretability of the models’ decision-making processes, techniques such as t-distributed Stochastic Neighbor Embedding (t-SNE) and Gradient-weighted Class Activation Mapping (Grad-CAM) are employed. t-SNE is utilized to visualize the high-dimensional latent features of the CNNs in a two-dimensional space, providing insights into how the models group different skin lesion classes. Grad-CAM is used to generate heatmaps that highlight the regions of input images that influence the model’s predictions. Our findings reveal that Test Time Augmentation enhances the balanced multi-class accuracy of CNN models by up to 0.3%, achieving a balanced accuracy rate of 97.58% on the International Skin Imaging Collaboration (ISIC 2019) dataset. This performance is comparable to, or marginally better than, more complex approaches such as Vision Transformers (ViTs), demonstrating the efficacy of our methodology. |
| format | Article |
| id | doaj-art-cc08f26f0c8d46f09fa968d20e3f9ee5 |
| institution | Kabale University |
| issn | 2313-433X |
| language | English |
| publishDate | 2025-01-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Journal of Imaging |
| spelling | doaj-art-cc08f26f0c8d46f09fa968d20e3f9ee52025-01-24T13:36:16ZengMDPI AGJournal of Imaging2313-433X2025-01-011111510.3390/jimaging11010015Skin Lesion Classification Through Test Time Augmentation and Explainable Artificial IntelligenceLoris Cino0Cosimo Distante1Alessandro Martella2Pier Luigi Mazzeo3Dipartimento di Ingegneria Informatica, Automatica, e Gestionale “Antonio Ruberti”, Sapienza Università di Roma, Via Ariosto, 25, 00185 Roma, ItalyIstituto di Scienze Applicate e Sistemi Intelligenti (ISASI), Consiglio Nazionale delle Ricerche (CNR), DHITECH, Campus Università del Salento, Via Monteroni s.n., 73100 Lecce, ItalyDermatologia Myskin, Poliambulatorio Specialistico Medico-Chirurgico, 73030 Tiggiano, ItalyIstituto di Scienze Applicate e Sistemi Intelligenti (ISASI), Consiglio Nazionale delle Ricerche (CNR), DHITECH, Campus Università del Salento, Via Monteroni s.n., 73100 Lecce, ItalyDespite significant advancements in the automatic classification of skin lesions using artificial intelligence (AI) algorithms, skepticism among physicians persists. This reluctance is primarily due to the lack of transparency and explainability inherent in these models, which hinders their widespread acceptance in clinical settings. The primary objective of this study is to develop a highly accurate AI-based algorithm for skin lesion classification that also provides visual explanations to foster trust and confidence in these novel diagnostic tools. By improving transparency, the study seeks to contribute to earlier and more reliable diagnoses. Additionally, the research investigates the impact of Test Time Augmentation (TTA) on the performance of six Convolutional Neural Network (CNN) architectures, which include models from the EfficientNet, ResNet (Residual Network), and ResNeXt (an enhanced variant of ResNet) families. To improve the interpretability of the models’ decision-making processes, techniques such as t-distributed Stochastic Neighbor Embedding (t-SNE) and Gradient-weighted Class Activation Mapping (Grad-CAM) are employed. t-SNE is utilized to visualize the high-dimensional latent features of the CNNs in a two-dimensional space, providing insights into how the models group different skin lesion classes. Grad-CAM is used to generate heatmaps that highlight the regions of input images that influence the model’s predictions. Our findings reveal that Test Time Augmentation enhances the balanced multi-class accuracy of CNN models by up to 0.3%, achieving a balanced accuracy rate of 97.58% on the International Skin Imaging Collaboration (ISIC 2019) dataset. This performance is comparable to, or marginally better than, more complex approaches such as Vision Transformers (ViTs), demonstrating the efficacy of our methodology.https://www.mdpi.com/2313-433X/11/1/15skin disease classificationskin datasettest time augmentationexplainable artificial intelligenceexplanatory taskconvolution neural network |
| spellingShingle | Loris Cino Cosimo Distante Alessandro Martella Pier Luigi Mazzeo Skin Lesion Classification Through Test Time Augmentation and Explainable Artificial Intelligence Journal of Imaging skin disease classification skin dataset test time augmentation explainable artificial intelligence explanatory task convolution neural network |
| title | Skin Lesion Classification Through Test Time Augmentation and Explainable Artificial Intelligence |
| title_full | Skin Lesion Classification Through Test Time Augmentation and Explainable Artificial Intelligence |
| title_fullStr | Skin Lesion Classification Through Test Time Augmentation and Explainable Artificial Intelligence |
| title_full_unstemmed | Skin Lesion Classification Through Test Time Augmentation and Explainable Artificial Intelligence |
| title_short | Skin Lesion Classification Through Test Time Augmentation and Explainable Artificial Intelligence |
| title_sort | skin lesion classification through test time augmentation and explainable artificial intelligence |
| topic | skin disease classification skin dataset test time augmentation explainable artificial intelligence explanatory task convolution neural network |
| url | https://www.mdpi.com/2313-433X/11/1/15 |
| work_keys_str_mv | AT loriscino skinlesionclassificationthroughtesttimeaugmentationandexplainableartificialintelligence AT cosimodistante skinlesionclassificationthroughtesttimeaugmentationandexplainableartificialintelligence AT alessandromartella skinlesionclassificationthroughtesttimeaugmentationandexplainableartificialintelligence AT pierluigimazzeo skinlesionclassificationthroughtesttimeaugmentationandexplainableartificialintelligence |