Artificial intelligence is increasingly capable of analyzing radiology images and reports with high accuracy. However, a major barrier to clinical adoption is that AI systems typically cannot explain why they made a particular decision -- they function as black boxes that produce results without justification.
This lack of transparency is a serious problem in medicine. Clinicians need to understand and trust AI recommendations before acting on them. A misclassification that cannot be explained is potentially more dangerous than a human error, because there is no way to identify or correct the underlying reasoning flaw.
Explainable AI (XAI) is an emerging field focused on making AI decisions interpretable. Rather than simply providing a classification (e.g., 'abnormal'), XAI systems show which features -- specific words in a report, or specific regions in an image -- drove the AI's decision.
Radiology is a particularly promising domain for XAI because the AI operates on structured inputs (reports and images) that clinicians already understand. Highlighting the relevant words or image regions bridges the gap between AI outputs and clinical reasoning, supporting verification rather than blind trust.
The study used the Indiana University chest X-ray dataset, a publicly available collection of 3,169 textual radiology reports and 6,471 X-ray images. Each report was labeled as either 'normal' (no pathological findings) or 'abnormal'. The goal was to classify these labels automatically and then explain the classifications.
For text classification, a large number of approaches were evaluated: traditional machine learning (logistic regression, SVM, random forests), ensemble methods (hard and soft voting), sequential deep learning (RNNs, LSTMs, BiLSTMs), transformer models (BERT, RoBERTa, ALBERT, DistilBERT, XLNet), and large language models (GPT-2, T5, LLaMA-2, and LLaMA-3.1). Each model was fine-tuned with domain-appropriate hyperparameters.
For image classification, multiple CNNs were compared: ResNet50, ResNet152, EfficientNet B7, VGG16, DenseNet121, and DenseNet169. Images were resized to 255x255 pixels and normalized. Standard data augmentation techniques (random crops, flips, color adjustments) were applied to improve generalization.
Explainability was applied post-classification using two complementary XAI methods: SHAP (Shapley Adaptive Explanations), which assigns importance scores to features using game theory, and LIME (Local Interpretable Model-Agnostic Explanations), which builds simple local approximations around individual predictions to identify decisive features.
For text classification, LLaMA-3.1 achieved the highest accuracy at 98%, with precision, recall, and F1-score all at 98-99%. This large language model substantially outperformed traditional models (logistic regression: 94%, SVM: 94%) and even advanced transformers like BERT and RoBERTa (both 96%). LLaMA-2 and T5 were close behind at 98% and 97% accuracy.
Statistical validation confirmed LLaMA-3.1's reliability: a Cohen kappa of 0.981 (near-perfect agreement beyond chance), chi-square and Fisher exact tests both showed p less than 0.001 (highly significant alignment between predicted and actual labels), and the McNemar test (p=0.25) showed balanced error rates between false positives and false negatives.
For image classification, DenseNet169 and DenseNet121 both achieved the highest accuracy at 84%, substantially outperforming simpler models (logistic regression: 66%, SVM: 65%, ResNet50: 64%, EfficientNet B7: 72%). The gap between text and image performance reflects the inherent difficulty of extracting clinical meaning from raw pixel data compared to structured language.
Traditional machine learning models performed poorly on images (random forest: 59%) but reasonably well on text (logistic regression: 94%), confirming that deep learning architectures are essential for medical image analysis while both approaches can be competitive for text classification.
LIME applied to text reports identified specific words as the most influential features. For an abnormal report, words like 'spine,' 'thoracic,' 'chest,' and 'degenerative' (in blue) pushed predictions toward abnormal, while 'normal' and 'no' (in orange) indicated absence of findings and pushed toward normal. These alignments match genuine clinical reasoning, confirming the model focuses on medically relevant language.
For a normal report, LIME assigned high probability (0.96) to the normal class, driven by words like 'normal,' 'no,' and 'acute' -- standard phrases radiologists use to confirm the absence of findings. This level of interpretability allows clinicians to audit whether the model is using appropriate signals or relying on spurious correlations.
SHAP applied to text provided more nuanced explanations. For an abnormal report, terms like 'pneumothorax' and 'effusion' received high positive SHAP values (pushing toward abnormal), while 'lungs' and 'heart' received negative values (pushing toward normal). The final prediction reflected the net balance of these contributions, allowing clinicians to understand why borderline cases were classified as they were.
LIME applied to X-ray images generated visual heatmaps highlighting the image regions most influential for the model's decision. For abnormal images, the highlighted regions corresponded to areas with potential pathological findings (shown in red/yellow), while normal images showed more distributed, lower-intensity highlighting. These visual explanations allow radiologists to verify whether the AI is looking at the right anatomical areas.
SHAP provides both global and local feature importance -- it can show which terms are most important across thousands of reports (global), and which specific words drove the decision for a single patient's report (local). This is particularly useful in ambiguous cases where the AI must distinguish between overlapping conditions like early pneumonia and mild pulmonary edema.
LIME complements SHAP by providing highly intuitive local explanations. By showing the exact words or image pixels that tipped the decision, LIME makes it easy for radiologists to spot false positives (AI calling a normal report abnormal) and false negatives (AI missing abnormal findings). This transparency allows clinicians to contest or confirm AI outputs rather than accepting them blindly.
The combination of SHAP and LIME creates a complementary explanation system: SHAP summarizes overall model behavior and identifies the most globally influential features, while LIME explains individual cases in a format accessible to clinicians without technical backgrounds. Together, they provide both strategic understanding and case-level accountability.
A critical caveat is that this study confirmed only technical alignment between AI explanations and expected clinical terminology. Whether these explanations actually improve radiologist decision-making in practice -- saving time, reducing errors, or increasing confidence -- was not tested and remains a key question for future clinical validation studies.
The Indiana University dataset has important limitations: it is relatively small, uses older imaging protocols, and consists largely of templated reports that may not reflect the diversity of real clinical environments. High accuracy on this dataset may therefore overestimate how well these models would perform on more varied, modern data from multiple institutions.
Text and images were analyzed separately rather than together. A multimodal framework that jointly analyzes both modalities would better reflect how radiologists actually work -- integrating visual findings with written narratives to reach a diagnosis. This remains an important gap for future work.
The SHAP and LIME explanations were not formally validated with radiologists. The study assumes that highlighting medically relevant terms constitutes meaningful explainability, but this assumption requires structured user studies -- reader tests, workflow simulations, integration with reporting systems -- to confirm whether these explanations actually improve clinical confidence and decision quality.
Future priorities include: testing on larger and more diverse datasets (MIMIC-CXR, CheXpert); adding complementary explanation methods (Grad-CAM for images, counterfactual explanations for text); and conducting formal radiologist evaluation studies to measure the real-world impact of XAI on diagnostic accuracy and workflow efficiency.
This study demonstrates that state-of-the-art AI models can classify radiology reports with high accuracy -- 98% for text (LLaMA-3.1) and 84% for images (DenseNet169) -- and that these decisions can be made interpretable using SHAP and LIME explanations.
The key insight is that explainability is not optional -- it is an essential component of any AI system intended for clinical use. Without transparency, even highly accurate AI tools will face resistance from clinicians who reasonably demand to understand and verify the recommendations they act on.
The finding that AI explanations consistently highlight clinically meaningful terms (like 'pneumothorax' and 'effusion' for abnormal findings, and 'no' and 'normal' for normal findings) provides an important initial validation that these models are reasoning correctly rather than exploiting spurious statistical patterns.
The broader implication for medical AI is that performance benchmarking alone is insufficient. Future AI tools for radiology and other diagnostic specialties must be evaluated on both accuracy and interpretability, with formal clinical validation studies to confirm that AI explanations translate into better patient care.