Medical Image Classification Using Deep Learning with Explainable AI

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056

Volume: 13 Issue: 04 | Apr 2026 www.irjet.net p-ISSN: 2395-0072

Medical Image Classification Using Deep Learning with Explainable AI

Mrs. Manoranjani1 , B. Abarna2, D. Siva Nagarjuna3, G. Madhulika4

123 UG Student, Department of Computer Science and Engineering, Dhanalakshmi Srinivasan University, Tiruchirappalli, Tamilnadu, India

4Assistant Professor, Department of Computer Science and Engineering, Dhanalakshmi Srinivasan University, Tiruchirappalli, Tamilnadu,India

Abstract - Medicalimageclassificationisacriticaltaskin modern healthcare systems, enabling early detection and diagnosis of various diseases [6]. Accurate interpretation of medicalimagessuchasX-ray,CT,andMRIscansplaysavital role in improving patient outcomes. However, traditional manual analysis is time-consuming and prone to human error. In recent years, deep learning techniques, particularly Convolutional Neural Networks (CNN), have demonstrated remarkable performance in image classification tasks [1][3][4]. These models automatically learn hierarchical features from data, eliminating the need for manual feature extraction. Despite their success, deep learning models often lack interpretability, making them difficult to trust in sensitive applications such as healthcare. To address this limitation, this paper proposes a CNN-based medical image classification system integrated with Explainable Artificial Intelligence (XAI). Specifically, Gradient-weighted Class Activation Mapping (Grad-CAM) is used to generate heatmaps that highlight important regions influencing model predictions [5]. This enhances transparency and allows medical professionals to understand the reasoning behind the model's decisions. The proposed model is evaluated using standard performance metrics including accuracy, precision, recall, and F1-score. Experimental results show that the system achieves high classification accuracy while providing meaningful visual explanations. The combination of performance and interpretability makes the proposed approach suitable for real-world clinical applications.

Key Words: Deep Learning, Medical Imaging, CNN, GradCAM,ExplainableAI,Healthcare

1. INTRODUCTION

Medical imaging technologies such as X-ray, Computed Tomography(CT),andMagneticResonanceImaging(MRI) arewidelyusedinmodernhealthcarefordiseasediagnosis and monitoring [6]. These imaging techniques provide detailed information about internal body structures, enabling early detection of diseases such as cancer, pneumonia,andneurologicaldisorders.

Traditionally, medical image analysis is performed manually by radiologists. Although expert knowledge ensures accurate diagnosis, manual interpretation is timeconsuming and may lead to inconsistencies due to human

error and fatigue. With the increasing volume of medical data, there is a growing need for automated systems that canassistinefficientandaccuratediagnosis.

Deep learning has emerged as a powerful tool for medical imageanalysis.ConvolutionalNeuralNetworks(CNN)have shown exceptional performance in image classification tasks due to their ability to automatically extract features from raw data [1][3][4]. These models learn complex patterns and representations, making them suitable for handlinghighdimensionalmedicalimages.

Despite their effectiveness, CNN-based models often act as black-box systems, providing predictions without explanations. This lack of transparency is a major limitationinhealthcareapplications,whereunderstanding the reasoning behind decisions is crucial. Medical professionals require interpretable models to ensure reliabilityandtrust.

Toovercomethischallenge,Explainable Artificial Intelligence (XAI) techniques are used to provide insights into model predictions. In this work, Grad-CAM is employed to generate visual explanations by highlighting important regions in the input images [5]. The proposed system combines CNN-based classification with Grad-CAM visualization to achieve both high accuracy and interpretability.

The main objective of this work is to develop an efficient and interpretable medical image classification system that can assist healthcare professionals in making informed decisions.

II. LITERATURE REVIEW

Several research studies have explored the application of deep learning techniques in medical image classification. Litjens et al. presented a comprehensive survey on deep learning in medical imaging, highlighting its effectiveness intaskssuchassegmentation,detection,andclassification [6]. Their work demonstrated that deep learning models outperform traditional machine learning methods in variousmedicalimagingapplications.

Rajpurkaretal.developedCheXNet,adeeplearningmodel for detecting pneumonia from chest X-ray images [2]. The

Volume: 13 Issue: 04 | Apr 2026 www.irjet.net

model achieved performance comparable to experienced radiologists, indicating the potential of AI in healthcare. This study emphasized the importance of large datasets and deep architectures for improving classification accuracy.

Deep architectures such as VGGNet and ResNet have also been widely used for image classification tasks [3][4]. VGGNet utilizes deep convolutional layers to extract hierarchical features, while ResNet introduces residual connections to overcome the problem of vanishing gradients. These models have significantly improved the performanceofimageclassificationsystems.

However,amajorlimitationofthesemodelsistheirlackof interpretability. They function as black-box systems, making it difficult to understand how predictions are made. This is particularly problematic in healthcare applications,wheretransparencyisessential.

To address this issue, Explainable AI techniques such as Grad-CAM have been introduced. Grad-CAM provides visual explanations by highlighting important regions in the image that influence the model's prediction [5]. This improvestrustandusabilityincriticalapplications.

The proposed system builds upon these existing works by integrating CNN-based classification with Grad-CAM visualization to achieve both high performance and interpretability.

Fig.-1:. Samplemedicalimagesfromthedatasetusedfor

III.PROPOSED SYSTEM

Theproposedsystemaimstoclassifymedicalimagesusing a deep learning approach while providing interpretability throughExplainableAItechniques.Thesystemisdesigned to process medical images and generate both predictions andvisualexplanations.

The workflow beginswithdata acquisition, where medical images are collected from publicly available datasets. Theseimagesarethenpre-processedtoensureconsistency in size and quality. Preprocessing steps include resizing, normalization,andnoiseremoval.

The pre-processed images are fed into a Convolutional NeuralNetwork(CNN)model.TheCNNconsistsofmultiple convolutional layers, pooling layers, and fully connected layers. These layers work together to extract features and classifyimagesintodifferentcategories[1][3].

To enhance interpretability, Grad-CAM is applied to the trained model. Grad-CAM generates heatmaps that highlightimportantregionsintheimagethatcontributeto themodel'sprediction[5].Thisallowsuserstounderstand thereasoningbehindtheclassification.

The proposed system provides both accurate predictions and visual explanations, making it suitable for real-world healthcareapplications.

IV.METHODOLOGY

The methodology of the proposed system consists of several stages, including data collection, preprocessing, featureextraction,classification,andevaluation.

In the data collection stage, medical images are obtained from publicly available datasets. These datasets contain labelled images representing different disease categories, ensuringreliabletrainingandevaluation[6].

During preprocessing, images are resized to a fixed resolution and normalized to improve consistency. Noise removal techniques are also applied to enhance image quality. Data augmentation methods such as rotation and flippingmaybeusedtoincreasedatasetdiversity.

Featureextractionandclassificationareperformedusinga Convolutional Neural Network (CNN). The CNN automatically learns hierarchical features from the input images, enabling accurate classification [3][4]. The model is trained using labelled data to learn discriminative patterns.

Grad-CAM is used for explainability. It analyses gradients flowing into the final convolutional layer to generate heatmaps that highlight important regions influencing predictions [5]. This provides visual explanations and improvesinterpretability.

A. Data collection

Theperformanceofadeeplearningmodellargelydepends onthequalityanddiversityofthedatasetusedfortraining. Inthiswork,medicalimagedatasetsarecollected from publiclyavailablesources,which include labelled images corresponding to different disease categories such as normalandabnormalconditions[6].

The dataset consists of various types of medical images suchasX-rayandMRIscans.Eachimageisassociatedwith a ground truth label that represents the presence or

Volume: 13 Issue: 04 | Apr 2026 www.irjet.net

absence of a particular disease. Proper data collection ensures that the model learns meaningful patterns and generalizeswelltounseendata.

To improve robustness, the dataset may include images captured under different conditions such as varying brightness, contrast, and resolution. This diversity helps the model to perform reliably in real world scenarios. The collected dataset is divided into training, validation, and testing sets to evaluate the performance of the model effectively.

B. Image Pre-processing

Image preprocessing is an essential step in medical image analysis as it improves the quality and consistency of the input data. In this work, several preprocessing techniques are applied to prepare the images for training the CNN model.

Initially, all images are resized to a fixed dimension to ensure uniformity in input size. This is necessary because CNN models require consistent input dimensions. Next, normalization is performed to scale pixel values to a standard range, which helps in faster convergence during training.

Noise reduction techniques are applied to remove unwantedartifactspresentinmedicalimages.Additionally, data augmentation methods such as rotation, flipping, and scaling are used to artificially increase the size of the dataset. These techniques help in reducing overfitting and improvingthegeneralizationcapabilityofthemodel.

Overall, preprocessing enhances the quality of the input data and contributes to improved classification performance.Featureextractionandclassification

A CNN model is used for feature extraction and classification. The model consists of convolutional layers, pooling layers, and fully connected layers that learn hierarchicalfeaturesfromtheinputimages[3][4].

C. Explainability Using Grad-CAM

Featureextractionandclassificationareperformedusinga ConvolutionalNeuralNetwork(CNN),whichiswidelyused for image analysis tasks due to its ability to learn hierarchicalfeaturesautomatically[1][3][4].

TheCNNarchitectureconsistsof multiplelayers,including convolutional layers, pooling layers, and fully connected layers. Convolutional layers are responsible for extracting featuressuchasedges,textures,andshapesfromtheinput images.Poolinglayersreducethespatialdimensionsofthe feature maps, which helps in reducing computational complexityandpreventingoverfitting.

The extracted features are then passed to fully connected layers, where classification is performed. The final output layer uses a soft max function to assign probabilities to different classes. The class with the highest probability is selectedasthepredictedoutput.

The model is trained using labelled data, and optimization techniques such as backpropagation and gradient descent are used to minimize the loss function. This enables the model to learn discriminative features and achieve high classificationaccuracy.

D. Explainability Using Grad-CAM

One of the major challenges of deep learning models is their lack of interpretability. To address this issue, Explainable Artificial Intelligence (XAI) techniques are incorporated into the proposed system. In this work, Gradient-weightedClassActivationMapping(Grad-CAM)is used to provide visual explanations for model predictions [5].

Grad-CAM works by computing the gradients of the target class with respect to the feature maps of the final convolutionallayer.Thesegradientsareusedtogeneratea heatmap that highlights important regions in the input image. The heatmap is then superimposed on the original image to visualize the areas that influenced the model’s decision.

Thisapproachhelpsinunderstandingwhetherthemodelis focusing on relevant regions of the image. In medical applications, this is particularly important as it allows healthcare professionals to verify the correctness of the model’spredictions.

By providing visual explanations, Grad-CAM improves transparency and builds trust in the system, making it moresuitableforreal-worldhealthcareapplications.

E. Performance Evaluation

Theperformanceoftheproposedmodelisevaluatedusing standard evaluation metrics, which provide a comprehensive assessment of classification performance. These metrics include accuracy, precision, recall, and F1score.

Accuracymeasurestheoverallcorrectnessofthemodel by calculating the ratio of correctly classified samples to the totalnumberofsamples.Precisionindicatestheproportion of true positive predictions among all positive predictions made by the model. Recall measures the ability of the modeltocorrectlyidentifypositivesamples.

TheF1-scoreistheharmonic meanofprecisionandrecall, providing a balance between the two metrics. These metrics are particularly important in medical image classification,wherebothfalsepositivesandfalsenegatives

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056

Volume: 13 Issue: 04 | Apr 2026 www.irjet.net p-ISSN: 2395-0072

canhavesignificantconsequences.

The evaluation results demonstrate that the proposed system achieves high performance across all metrics, indicating its effectiveness in medical image classification tasks.

V.SYSTEM ARCHITECTURE

Theoverallarchitectureoftheproposedsystemconsistsof multiple stages, including data input, preprocessing, feature extraction, classification, and explainability. The workflow begins with the input of medical images, which are first subjected to preprocessing techniques such as resizingandnormalizationtoensureconsistency.

The pre-processed images are then passed through a Convolutional Neural Network (CNN) model. The CNN consists of multiple convolutional layers that extract important features from the images, followed by pooling layers that reduce dimensionality and improve computational efficiency [3][4]. The extracted features are then passed to fully connected layers, where classification isperformed.

To enhance interpretability, Grad-CAM is applied after the classification stage. Grad-CAM generates heatmaps that highlightimportantregionsintheimagethatcontributeto the prediction [5]. These heatmaps are overlaid on the originalimagestoprovidevisualexplanations.

The architecture ensures efficient processing of medical images and provides both accurate predictions and interpretable outputs. This makes the system suitable for real-worldhealthcareapplications.

Fig -2:SystemArchitectureofProposedMedicalImage ClassificationSystem

VI.IMPLEMENTATION DETAILS

[1] TheproposedsystemisimplementedusingPythonand deeplearning librariessuch asTensorFlowand Keras. The modelistrainedonasystemwithsufficientcomputational

resourcestohandlelargemedicalimagedatasets.

The dataset is divided into training, validation, and testing sets to ensure proper evaluation of the model. The CNN model is trained using labelled data, and hyperparameters suchaslearningrate,batchsize,andnumberofepochsare carefullyselectedtoachieveoptimalperformance.

The training process involves forward propagation, loss computation, and backpropagation to update model weights. The model is trained for multiple epochs until convergence is achieved. Techniques such as dropout and data augmentation are used to prevent overfitting and improvegeneralization.

Aftertraining,themodelisevaluatedonthetestdatasetto measure its performance. Grad-CAM is integrated into the systemtogeneratevisualexplanationsforpredictions.The implementation ensures that the system is efficient, accurate,andinterpretable.

[2] The use of deep learning frameworks simplifies the development process and allows for easy scalability of the system.

VII.RESULTS AND DISCUSSION

Theperformanceoftheproposedsystemisevaluated using standard medical image datasets. The CNN model demonstrates strong classification performance,achievinghighaccuracyacrossdifferent categories[1][2].

The evaluationmetricsusedinthisstudyincludeaccuracy, precision, recall, and F1-score. These metrics provide a comprehensiveunderstandingofthemodel'sperformance. Theresultsindicatethatthemodeliscapableofaccurately classifyingmedicalimageswithminimalerror.

Precision and recall values show that the model effectively identifies positive cases while minimizing false positives and false negatives. The F1-score provides a balance betweenprecisionandrecall,confirmingtherobustnessof themodel.

Grad-CAM visualizations play a crucial role in understanding the model's decision-making process. The generated heatmaps highlight important regions in the images that influence the predictions [5]. These visual explanations help in verifying whether the model is focusingonrelevantfeatures.

The results demonstrate that the integration of explainability techniques improves transparency and trust in the system. Compared to traditional approaches, the proposed system provides both accurate predictions and meaningfulexplanations.

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056

Volume: 13 Issue: 04 | Apr 2026 www.irjet.net p-ISSN: 2395-0072

Overall,theexperimentalresultsconfirmthattheproposed approach is effective for medical image classification and canbeappliedinreal-worldhealthcarescenarios.

VIII.PERFORMANCE ANALYSIS

The proposed system demonstrates significant improvements in both classification accuracy and interpretability. The use of CNN enables automatic feature extraction,reducingtheneedformanualintervention.

The integration of Grad-CAM enhances the usability of the system by providing visual explanations. This is particularly important in medical applications, where

understanding the reasoning behind predictions is essential.

However, the system may face limitations when dealing withhighlycomplexdatasetsorlow-qualityimages.Future improvements can address these challenges by incorporatingmoreadvancedmodelsandlargerdatasets. The results indicate that the proposed system is a promisingsolutionformedicalimageclassificationtasks.

IX.CONCLUSION

This paper presents a deep learning-based approach for medical image classification using Convolutional Neural Networks integrated with Grad-CAM for interpretability. The proposed system achieves high classification accuracy whileprovidingvisualexplanationsforpredictions.

The results demonstrate that the model performs effectively across various evaluation metrics. The integration of Explainable AI techniques enhances transparency, making the system more reliable for healthcareapplications[1][5].

Overall, the proposed approach improves diagnostic efficiency and supports better decision-making in clinical environments.

X. FUTURE WORK

Futureworkcanfocusonimprovingtheperformanceofthe model by using larger and more diverse datasets. The system can be extended to support multiple imaging modalitiessuchasCTandMRI.

Advanced deep learning techniques such as transfer learning and hybrid models can be explored to further enhance accuracy. Additionally, deploying the system in real-time healthcare environments can improve its practicalusability.

XI. ACKNOWLEDGEMENT

Theauthorswouldliketoexpresstheirsinceregratitudeto the faculty members and guide for their continuous support, valuable suggestions, and guidance throughout the development of this project. We also thank our institution for providing the necessary resources and infrastructure to successfully complete this work. Special thanks to the developers and contributors of open-source datasetsandtoolsthatmadethisresearchpossible.

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056

Volume: 13 Issue: 04 | Apr 2026 www.irjet.net p-ISSN: 2395-0072

REFERENCES

[1] A. Krizhevsky, I. Sutskever, and G. E. Hinton, "ImageNet Classification with Deep Convolutional Neural Networks," Advances in Neural Information ProcessingSystems,2012.

[2] P. Rajpurkar et al. "CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learning,"arXivpreprintarXiv:1711.05225,2017.

[3] K. Simonyan and A. Zisserman, "Very Deep Convolutional Networks for Large-Scale Image Recognition,"arXivpreprintarXiv:1409.1556,2014.

[4] K. He, X. Zhang, S. Ren, and J. Sun, "Deep Residual Learning for Image Recognition," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR),2016.

[5] R. R. Selvaraju et al., "Grad-CAM: Visual Explanations fromDeepNetworksviaGradient-BasedLocalization," Proceedings of the IEEE International Conference on ComputerVision(ICCV),2017.

[6] G.Litjensetal.,"ASurveyonDeepLearninginMedical Image Analysis," Medical Image Analysis, vol. 42, pp. 60–88,2017.

[7] D. Shen, G. Wu, and H.-I. Suk, "Deep Learning in MedicalImageAnalysis,"AnnualReviewofBiomedical Engineering,vol.19,pp.221–248,2017.

[8] O. Ronneberger, P. Fischer, and T. Brox, "U-Net: Convolutional Networks for Biomedical Image Segmentation,"MICCAI,2015.

[9] M. T. Ribeiro, S. Singh, and C. Guestrin, "Why Should I Trust You? Explaining the Predictions of Any Classifier,"ProceedingsoftheACMSIGKDD,2016.

[10] S. Lundberg and S.-I. Lee, "A Unified Approach to Interpreting Model Predictions," Advances in Neural InformationProcessingSystems,2017.

[11] H. Greenspan, B. van Ginneken, and R. M. Summers, "Deep Learning in Medical Imaging: Overview and Future Promise," IEEE Transactions on MedicalImaging,vol.35,no.5,2016.

[12] A.Estevaetal.,"Dermatologist-LevelClassification of Skin Cancer with Deep Neural Networks," Nature, vol.542,pp.115–118,2017.