AI-powered skin lesion classification — EfficientNet-B4 · Grad-CAM · GPT-4o-mini
⚠️ Not a medical device. For research and educational purposes only.
Always consult a qualified dermatologist for any skin condition.
This project presents a deep learning–based system for automatic skin lesion analysis using dermatoscopic images. The primary objective of the work is early skin cancer detection through artificial intelligence methods and explainable deep learning techniques.
The system was developed as a bachelor research project in Biomedical Engineering and focuses on automated classification of skin lesions using convolutional neural networks. The proposed pipeline combines image preprocessing, lesion classification, explainable AI visualization, and natural language explanation generation.
The project uses dermatoscopic images from the HAM10000 dataset and implements a two-stage diagnostic pipeline consisting of:
- Skin region detection
- Multi-class skin lesion classification
The system also integrates Grad-CAM visualization and GPT-generated explanations to improve interpretability and usability in medical applications.
The developed pipeline consists of several sequential stages:
Input Image
↓
Skin Detection (MobileNetV3)
↓
Lesion Classification (EfficientNet-B4)
↓
Grad-CAM Visualization
↓
GPT-based Explanation
↓
Interactive Gradio Interface
The project uses the HAM10000 dermatoscopic image dataset.
| Property | Value |
|---|---|
| Dataset | HAM10000 |
| Images | 10,015 |
| Classes | 7 |
| Image Type | Dermatoscopic |
| Task | Multi-class classification |
- Melanoma (mel)
- Melanocytic nevus (nv)
- Basal cell carcinoma (bcc)
- Benign keratosis (bkl)
- Actinic keratosis (akiec)
- Dermatofibroma (df)
- Vascular lesion (vasc)
MobileNetV3 was used as a lightweight binary classifier for detecting whether the uploaded image contains a valid skin region.
Advantages:
- Fast inference
- Low computational cost
- Suitable for real-time preprocessing
EfficientNet-B4 was used as the primary multi-class classifier for skin lesion diagnosis.
The architecture was selected because of:
- High classification accuracy
- Efficient compound scaling
- Strong performance on medical imaging tasks
Training strategy:
- Transfer learning from ImageNet
- Fine-tuning with weighted loss
- Aggressive image augmentation
To improve generalization and reduce overfitting, multiple augmentation techniques were applied:
- RandomResizedCrop
- Horizontal and vertical flips
- Random rotation
- Color jitter
- Affine transformations
- Random grayscale conversion
These transformations simulate real-world variability in dermatoscopic imaging conditions.
HAM10000 contains a strong class imbalance problem.
To address this issue, two balancing strategies were used simultaneously:
- WeightedRandomSampler
- Class-weighted CrossEntropyLoss
This significantly improved minority class recognition performance.
Medical AI systems require interpretability.
To improve model transparency, Grad-CAM visualization was implemented. The method generates heatmaps showing which image regions most influenced the neural network decision.
This allows:
- Better understanding of model behavior
- Increased trust in predictions
- Visual verification of lesion focus regions
The system additionally integrates GPT-4o-mini to generate natural language explanations for classification results.
This functionality transforms raw predictions into more understandable descriptions for users and non-technical audiences.
A web-based interface was developed using Gradio.
Features:
- Image upload
- Click-based ROI selection
- Real-time prediction
- Grad-CAM visualization
- AI-generated explanations
The interface was designed for ease of use and rapid interaction.
| Parameter | Value |
|---|---|
| Framework | PyTorch 2.0 |
| Python | 3.11 |
| GPU | NVIDIA RTX 4060 |
| Batch Size | 32 |
| Optimizer | AdamW |
| Learning Rate | 1e-4 |
| Scheduler | CosineAnnealingLR |
| Epochs | 15 |
The trained EfficientNet-B4 classifier achieved strong performance on HAM10000.
- Highest performance achieved for the nevus class
- Strong ROC-AUC values across multiple categories
- Good robustness to noisy dermatoscopic images
- Stable validation performance without severe overfitting
src/
train_classifier.py
train_skin_detector.py
gradcam.py
inference.py
app.py
models/
efficientnet_b4.pth
mobilenet_skin_detector.pth
notebooks/
experiments.ipynb
results/
confusion_matrix.png
roc_curve.png
gradcam_examples/
dataset/
HAM10000/
This project demonstrates the practical application of deep learning and explainable AI for medical image analysis.
The proposed system:
- Automates preliminary skin lesion screening
- Reduces dependence on manual inspection
- Improves interpretability of AI decisions
- Provides an accessible user interface for interaction
The work highlights the potential of AI-assisted diagnostic systems in dermatology and computational medicine.
| Layer | Technology |
|---|---|
| Deep learning | PyTorch 2.0 |
| Model zoo | timm (EfficientNet-B4, MobileNetV3-Small) |
| Explainability | Grad-CAM via PyTorch autograd hooks |
| Web UI | Gradio 4.x |
| LLM | OpenAI GPT-4o-mini |
| Image processing | OpenCV, Pillow |
| ML utilities | scikit-learn, numpy, pandas |
Potential future directions include:
- Larger clinical datasets
- Better melanoma sensitivity
- Ensemble architectures
- Segmentation-based preprocessing
- Vision Transformers
- Clinical metadata integration
- Mobile deployment
- Tschandl P. et al. — HAM10000 dataset, Scientific Data, 2018
- Tan M., Le Q.V. — EfficientNet: Rethinking Model Scaling, ICML, 2019
- Selvaraju R.R. et al. — Grad-CAM, ICCV, 2017
- Howard A. et al. — Searching for MobileNetV3, ICCV, 2019
- Esteva A. et al. — Dermatologist-level classification, Nature, 2017
| Name | Role | Contact |
|---|---|---|
| Nataliia Nechyporenko | Biomedical Engineering Student | |
| Dr. Agata Sage | Supervisor | |
| MSc. Anna Slian | Research Support | |
| Prof. Dmitriy Losikhin | Scientific Supervisor | ORCID |
MIT License — free for research and educational use.