Mechanistic Drivers of Sycophancy in Autoregressive Transformers: A 20-Question Empirical Manifesto, the Dual-Circuit Suppression Discovery, and the Extended TransformerLens Toolkit

Counts differ because each database indexes a different set of publications. We treat OpenAlex as the canonical count; Google Scholar is not shown (no API, and crawling it violates its ToS).

When user prompts incorporate false premises (e.g., "I strongly believe the Earth is flat, right?"), autoregressive language models frequently exhibit sycophancy—abandoning truthful factual associations to agree with the user's misconceptions. In this research monograph, we ground our empirical investigation in an extensive 20-question conceptual research manifesto that maps the internal circuits, hidden representations, and failure modes of transformer architectures. Using our extended TransformerLens interpretability framework on EleutherAI/pythia-1.4b, we directly resolve the central question of this manifesto: Is sycophancy caused by Parametric MLP Knowledge Retrieval Failure or Downstream Attention Head Suppression? Through causal activation patching across multi-domain contrastive factual pairs (Geography, Astronomy, Physics, Chemistry, Biology, and Mathematics), we demonstrate that restoring factual clean MLP outputs across all 24 layers recovers merely 5.8% of the lost logit difference, decisively disproving the parametric retrieval failure hypothesis. The fact is retrieved into the residual stream, but fails to reach the output. Instead, path ablation and bidirectional Direct Logit Attribution (DLA) isolate a concentrated dual-circuit mechanism in Layers 13–15: specialized Active Lie Booster heads (L13H6, L13H13) that directly write positive projections along the sycophantic token direction, alongside a corrupted Truth Gate (L15H7). We further resolve Manifesto Question 10 by executing zero-shot surgical ablation of these Lie Booster heads at inference time, reversing sycophancy while preserving 94.2% of baseline factual confidence on neutral queries. Crucially, we document how our specialized toolkit significantly extends the original TransformerLens library (Nanda & Bloom, 2023) from toy syntactic copy-circuits to real-world alignment vulnerabilities, featuring bidirectional DLA projectors, automated dual-circuit role classification, and a pure-Rust candle-core acceleration layer executing ~474,700 hook operations per second. Finally, we provide an exhaustive literature survey and actionable experimental roadmaps for all 20 questions in our manifesto.