Disagreement reliability routing for multimodal large language model evidence pipelines with imperfect counterfactual priors

Counts differ because each database indexes a different set of publications. We treat OpenAlex as the canonical count; Google Scholar is not shown (no API, and crawling it violates its ToS).

Multimodal large-language-model (LLM) evidence pipelines often attach rule-derived outcomes to candidate remediation actions even though only the executed action yields a factual production outcome. Treating those counterfactual rules as ground truth can miscalibrate routing, while discarding them wastes structured operational knowledge. We introduce EviRoute-DR, a disagreement-reliability routing method that learns when rule priors fail and combines corrected-rule experts with direct factual outcome experts through a reliability-conditioned meta-learner. The study uses 2,100 production cases, seven candidate actions per case (14,700 case-action records), and exactly one observed production outcome per case. Base models are trained on 600 factual cases, combined on 200, and evaluated on 1,300 held-out factual cases spanning main, temporal-labelled, hard, and workflow-shift cohorts. On factual outcomes, EviRoute-DR achieves AUC 0.926 and Brier score 0.0910, compared with 0.921/0.0971 for direct logistic regression and 0.924/0.0922 for a random-forest baseline. The paired bootstrap Brier improvement over logistic regression is 0.0060 (95\% CI 0.0021--0.0092). A separate production-derived counterfactual replay shows a stronger cost-resolution frontier at moderate operating costs, reaching approximately 0.815 resolution near 12 cost units. Because the logged action is always the argmax of the production score vector, we do not interpret those scores as stochastic propensities and make no inverse-propensity, doubly robust causal, or production-effect claim. The results support disagreement-aware reliability fusion as a practical methodology for evidence routing under partial factual feedback.

• Evidence-Inference Reconstruction: When The Evidence Is Recalled But The Reasoning Goes Wrong

  arXiv (Cornell University) 2026 · 0 citations

• Is This Evidence Decision-Critical? Learning to Verify Rule-Governed Decisions

  arXiv (Cornell University) 2026 · 0 citations

• Adaptive Switching Between Self-Evidence and Mutual-Evidence Mechanisms in Multiple Large Language Models: A Unified Framework for Dynamic Evidence Coordination

  Zenodo (CERN European Organization for Nuclear Research) 2026 · 0 citations