The Evidence You Do Not Give the AI: Holdout Asymmetry and Epistemic Self-Correction Under Model Mismatch

Counts differ because each database indexes a different set of publications. We treat OpenAlex as the canonical count; Google Scholar is not shown (no API, and crawling it violates its ToS).

This paper introduces holdout asymmetry, the epistemic advantage created when a human deliberately retains evidence outside an AI system’s conversational context and reveals it only after the system has committed to an interpretation or prediction. The distinction matters because large language models are highly capable of retrospective explanation: once evidence enters context, a sufficiently flexible model can often construct an auxiliary account that preserves an existing theory without taking on new predictive risk. Building on Lakatos’s distinction between progressive and degenerating research programmes, the paper identifies degenerating accommodation as a failure mode in which conversational models preserve an evolving explanatory frame through successive post hoc auxiliaries. Unlike conventional multi-turn sycophancy measures centred on stance change, concession or resistance, the proposed approach evaluates the evolution of theory structure across turns. The paper develops a four-stage protocol — Reserve → Commitment → Reveal → Revision — in which evidence is deliberately withheld until the model has made risk-bearing commitments. It also introduces prediction vacancy as the diagnostic condition in which hidden evidence arrives but no prior prediction exists for it to test. A proposed experiment comparing pretrained base models, instruction-tuned models and preference-optimised conversational models distinguishes autoregressive context-conditioning from preference-based sycophancy as possible mechanisms. The paper further argues that persistent memory may gradually consume the user’s epistemic reserve, creating a tension between continuity, personalisation and independent auditability. It concludes that preserving human epistemic agency may sometimes require deliberately maintaining information outside the AI system.