Fooling LIME and SHAP

Counts differ because each database indexes a different set of publications. We treat OpenAlex as the canonical count; Google Scholar is not shown (no API, and crawling it violates its ToS).

As machine learning black boxes are increasingly being deployed in domains such as healthcare and criminal justice, there is growing emphasis on building tools and techniques for explaining these black boxes in an interpretable manner. Such explanations are being leveraged by domain experts to diagnose systematic errors and underlying biases of black boxes. In this paper, we demonstrate that post hoc explanations techniques that rely on input perturbations, such as LIME and SHAP, are not reliable. Specifically, we propose a novel scaffolding technique that effectively hides the biases of any given classifier by allowing an adversarial entity to craft an arbitrary desired explanation. Our approach can be used to scaffold any biased classifier in such a way that its predictions on the input data distribution still remain biased, but the post hoc explanations of the scaffolded classifier look innocuous. Using extensive evaluation with multiple real world datasets (including COMPAS), we demonstrate how extremely biased (racist) classifiers crafted by our framework can easily fool popular explanation techniques such as LIME and SHAP into generating innocuous explanations which do not reflect the underlying biases.

Last synced 2026-08-22 14:19 UTC

• Explainable AI in Corporate Finance: A SHAP-Based Approach to Enhancing Transparency

  Businesses 2026 · 0 citations

• Estimating adversarial transferability from feature-importance similarity in explainable intrusion detection systems

  International Journal of Information Security 2026 · 0 citations

• Explainable Artificial Intelligence in Healthcare: a Multi-criteria Benchmark of Explanation Faithfulness, Stability and Consensus in Clinical Risk Prediction

  Научный журнал "Вестник Актюбинского регионального университета имени К Жубанова" 2026 · 0 citations

• Monitoring Explanation Drift in Longitudinal NLP Pipelines: A Cross-Domain Study under Temporal Distribution Shift

  Unspecified venue · 0 citations

• Explainability for Forecasting Task with Close-Up Option

  AI 2026 · 0 citations

• A Modality-Adaptive Software Architecture for Explainable Clinical Decision Support

  GlobeIS International Journal of Global Information Systems 2026 · 0 citations

• Improving AI Explainability Through Design of Experiments

  Journal of Chemometrics 2026 · 0 citations