Testing Non-Substitution in Language Models: An evaluation protocol and public reference suite

Counts differ because each database indexes a different set of publications. We treat OpenAlex as the canonical count; Google Scholar is not shown (no API, and crawling it violates its ToS).

Non-substitution distinguishes an estimate from the event, authorization, or participation it represents. This paper specifies a prospective test of whether an explicit non-substitution framing improves language-model decisions beyond a strong authorization-and-evidence framing. It separates an exploratory comparison of existing instruction packages from a controlled comparison in which three shared operational checklists differ only in their framing sentence. Forty public, author-labeled development cases illustrate five families of decisions. Matched boundary and supported variants penalize blanket withholding. A separate proposed study uses fresh independently labeled templates, crossed scenario forms and checklist paraphrases, and a fixed model roster. The supplied analysis validates a complete request manifest, retains missing responses in denominators, and computes paired, family-stratified template bootstrap summaries. Synthetic fixtures check its accounting and resampling; they are not model evidence or a calibration study. Benefit thresholds, scope of inference, and collection prerequisites are explicit. The protocol is ready for criticism as a methods proposal; independent labeling, model selection, tokenizer checks, calibration and sensitivity analysis, and a preregistration freeze remain required before confirmatory collection. A future positive result would concern specified instruction effects on fictional tasks, not general alignment or the truth of the broader philosophical framework. Research status: Prospective methods paper; public development examples, controlled prompts, and analysis software with synthetic checks; no model experiment. Not peer reviewed. Contents: Paper in PDF, HTML, and Markdown, README, supplementary materials, license notices, and file manifest. Version 1.1 revises the unpublished version 1.0 draft. Licenses: The paper, research protocols, prompts, cases, manifests, results, and synthetic fixture/check outputs use CC BY 4.0. All .py files, supplement/README.md, supplement/STUDY_SCHEMA.md, and supplement/development/README.md use MIT. See LICENSE.md for scope. Related work: The conceptual antecedent is mFLO 111, Invariant Relation and Non-Substitution, version 1.1, https://doi.org/10.5281/zenodo.22805469.