Equally accurate, differently explained: Stability of permutation feature importance across machine learning classifiers on clinical tabular data

Counts differ because each database indexes a different set of publications. We treat OpenAlex as the canonical count; Google Scholar is not shown (no API, and crawling it violates its ToS).

Feature-importance explanations are increasingly used to justify machine learning (ML) models in clinical decision support, yet they are usually reported from a single fitted model without evidence that they would recur if the model were refitted. This study quantifies the stability of permutation feature importance across five widely used classifiers, namely logistic regression (LR), radial-basis support vector machine (SVM), random forest (RF), histogram gradient boosting (GB) and a multilayer perceptron (MLP), on three public clinical tabular datasets: Wisconsin diagnostic breast cancer (WDBC), Pima diabetes and Cleveland heart disease. Using 10 × 5-fold stratified cross-validation (50 fits per model and dataset), we measured discrimination and calibration and computed AUC-based permutation importance on every held-out fold. Stability was summarised with pairwise Spearman correlation, top-5 Jaccard overlap and top-feature recurrence. On WDBC, LR, SVM, RF and GB were statistically indistinguishable in AUC (0.990–0.995; corrected resampled t-test p ≥ 0.089), yet their explanation stability ranged from 0.32 (RF) to 0.61 (SVM), these differences were significant (Holm-adjusted Wilcoxon p < 0.001), and the models disagreed on the most important feature. Stability was highest where predictors were few and weakly correlated (Pima, 0.70–0.87, with glucose ranked first in every fit of four models) and lowest where many near-redundant features were present. Reducing the training data to 25% lowered AUC by at most 0.03 for the four well-behaved models but reduced stability markedly (e.g., 0.90 to 0.68 for LR on Pima), whereas degenerate MLP fits produced spuriously stable importance. Predictive accuracy therefore does not certify explanation reliability, and importance-based explanations should be reported together with resampling-based stability estimates.

• Predictive accuracy does not establish feature-importance robustness: a leave-top feature-out analysis in late-life depression

  Biomedical Signal Processing and Control 2026 · 0 citations

• Stability and robustness of explainable artificial intelligence methods in clinical prediction models: Protocol for a systematic review

  Journal of Artificial Intelligence Machine Learning and Neural Network 2026 · 0 citations