Explainable AI in Critical Systems: Techniques, Trust and Challenges

Counts differ because each database indexes a different set of publications. We treat OpenAlex as the canonical count; Google Scholar is not shown (no API, and crawling it violates its ToS).

In highly sensitive areas like healthcare, cybersecurity, transport and finance, where an unexplainable or incorrect decision could have dire consequences, AI also aids in making decisions. Explainable AI (XAI) has become a widely suggested way to create transparency, accountability and trust, yet the literature is scattered both by technique, domain and evaluation methods. A focused critical review of 14 studies published from 2020 to 2026 is presented, consisting of 11 surveys and systematic reviews, 2 studies that were empirical with an intrusion detection focus and 1 clinician-based evaluation of the heatmap methods. We categorize the various XAI techniques according to scope, timing, model dependence and explanation form, distinguish between the technical trustworthiness of an explanation and the human calibration of trust and combine the evaluation of explanations in practice. The evidence reveals that post-hoc, local methods, in particular SHAP, LIME and Grad-CAM, outstood in terms of applications, explanations have progressed with greater speed than explanation validation, and most of the studies did not focus on quantitative evaluation or user studies, while expert judgements of the plausibility of explanations were not reliably associated with the quality of the models. We argue that explanations in critical systems must be seen as artefacts for safety, which must be validated, and that a research agenda should be pursued in the areas of standardized evaluation, human centered studies, privacy preserving explanation and regulatory alignment.

• Explainable Artificial Intelligence (XAI): What we know and what is left to attain Trustworthy Artificial Intelligence

  Information Fusion 2023 · 1646 citations