Annotating Pre-Reliance Handling for AI-Generated Claims: A Preliminary Protocol and Feasibility Pilot

Counts differ because each database indexes a different set of publications. We treat OpenAlex as the canonical count; Google Scholar is not shown (no API, and crawling it violates its ToS).

Large language model outputs can mix stable factual information, time-sensitive claims, interpretations, recommendations, and high-stakes advice within a single response. This working paper introduces pre-reliance handling as a claim-level annotation target: what form of checking or epistemic treatment may be warranted before a user relies on an atomic AI-generated claim. The preliminary protocol distinguishes a low-risk baseline (VA1), source checking (VA2), currentness checking (VA3), expert or responsible-authority checking (VA4), and provisional or interpretive handling (VA5). These categories are presented as an operational synthesis of distinctions motivated by prior research, not as newly discovered verification practices. A feasibility pilot used four prompts and eight independently generated ChatGPT answers covering public information, organizational explanation, currentness-sensitive information, and domain-sensitive advice. The answers were decomposed into 163 atomic claims. The initial protocol over-assigned VA5 to ordinary empirically testable generalizations; after revision from Section 3 v0.4 / Guideline v0.1 to Section 3 v0.5 / Guideline v0.2, VA5 assignments decreased from 46 to 20, 27 handling annotations in the organizational-behaviour prompt set changed, and no new core category was required. The study does not claim an exhaustive or validated taxonomy, independent annotation reliability, or demonstrated effects on user behaviour. Its narrower contribution is to show that differentiated pre-reliance handling can be operationalized at the atomic-claim level in the tested material while documenting the boundary problems exposed by pilot application. The companion public package includes the four prompts, eight raw AI-generated responses, 163 claim-level annotations, a codebook, boundary log, Annotation Guideline v0.2, and revision log.