Comparing Four Explanation Methods for Melanoma Classification: Fidelity, Utility and Trust

Counts differ because each database indexes a different set of publications. We treat OpenAlex as the canonical count; Google Scholar is not shown (no API, and crawling it violates its ToS).

Post-hoc explanation methods are used to make skin-lesion classifiers interpretable but are usually judged on one criterion, most often whether the heatmap looks plausible. We asked whether technical fidelity, a clinical utility proxy and computational trust calibration rank explanation methods the same way. An EfficientNet-B0 melanoma classifier was trained on the 10,015 HAM10000 dermoscopic images with a lesion-level split, and four methods (Grad-CAM, Integrated Gradients, LIME and occlusion sensitivity) were applied to 800 stratified test images. Explanations were compared with lesion masks using intersection-over-union under four binarisation rules, pointing-game accuracy, deletion and insertion curves, and a cascading parameter-randomisation sanity check. Utility was measured by lesion hit rate, centroid distance and artefact attribution, and trust by the fidelity gap between correct and incorrect predictions. The classifier reached a test area under the receiver operating characteristic curve of 0.9295. LIME and occlusion led on localisation and could not be separated statistically (Holm-corrected p = 0.465). Integrated Gradients ranked second on pointing game and centroid distance but failed the sanity check, keeping a rank correlation of 0.33 with its original map after seven of nine randomisation stages. All 12 trust comparisons were negative; stratifying by predicted class showed that fidelity tracked model confidence, and only Grad-CAM kept a significant positive difference, which may itself partly reflect confidence. Agreement between methods peaked at a rank correlation of 0.22. The axes disagree enough to change which method would be chosen, so multi-axis evaluation with a faithfulness gate should become standard.