Explainability in Clinical Artificial Intelligence Decision Support: a Systematic Review of Empirical Evidence on Clinician Trust, Reliance, and Decision Quality

Counts differ because each database indexes a different set of publications. We treat OpenAlex as the canonical count; Google Scholar is not shown (no API, and crawling it violates its ToS).

Background: Explainable artificial intelligence (XAI) is increasingly proposed as a way to make clinical decision-support systems more transparent and acceptable to healthcare professionals. However, explanation does not necessarily improve clinical decisions: it may help users identify relevant evidence, have no measurable effect, or encourage excessive reliance on incorrect AI advice. This review synthesises empirical evidence on how clinician-facing explanations influence trust, reliance, and decision quality. Methods: A structured evidence search was conducted on 20 October 2026 using web-indexed scholarly search, PubMed/PMC records, and publisher-hosted full-text pages. Search concepts combined explainable AI, clinical decision support, clinicians/physicians, trust/reliance, diagnostic accuracy, and human-AI decision-making. A core set of ten empirical studies was identified and characterised from the study table and reference list of Rosenbacke et al. (2024), with key primary reports checked against publisher or PubMed records. More recent evidence was used to contextualise the findings. Because a complete exportable database search and independent duplicate screening were not available in this workflow, this is a transparent structured systematic evidence synthesis rather than a fully reproducible database-level review; no new PRISMA record counts are claimed. Results: The evidence indicates that explanations can improve perceived understandability and sometimes support decision performance, but their effects on trust are inconsistent and context-dependent. In a chemotherapy decision-support experiment involving 41 medical practitioners, example-based and counterfactual explanations were easier to understand than some alternatives, yet explanations also increased willingness to accept AI recommendations, including incorrect ones. In an X-ray experiment involving 223 physicians, annotated AI advice improved diagnostic accuracy particularly for non-radiology experts, while confidence in clinicians' own diagnoses did not meaningfully change. Other studies reported increased trust or intention to use AI, no clear effect, or concerns about complex, irrelevant, or time-consuming explanations. Conclusion: XAI should not be evaluated by whether it increases trust alone. Evaluation should test whether clinicians can distinguish correct from incorrect recommendations, whether reliance is appropriately calibrated, and whether explanations fit clinical workflows. Larger, preregistered, multisite studies using realistic clinical tasks and common outcome measures are needed.