The Alignment Reward Bug

Counts differ because each database indexes a different set of publications. We treat OpenAlex as the canonical count; Google Scholar is not shown (no API, and crawling it violates its ToS).

Posttraining converts preferences and policy judgments into reward signals. This can improve usefulness and safety, but it can also favor agreement, flattery, or refusal patterns over truth. OpenAI's April 2025 GPT 4o rollback provides a documented case: individually positive changes combined into visibly sycophantic behavior that offline evaluations missed. The incident supports a narrow conclusion about reward design and deployment evaluation, not the claim that all guardrails are fake.