Nathan Ryan Young · Zenodo (CERN European Organization for Nuclear Research) 2026 · 2026
DOI: 10.5281/zenodo.21213238
Counts differ because each database indexes a different set of publications. We treat OpenAlex as the canonical count; Google Scholar is not shown (no API, and crawling it violates its ToS).
Superposition is the standard hypothesis for why transformer internals resist interpretation, but its cost structure in real models has remained unmeasured. We show that the interference of a real transformer's own MLP code is lawful: a compact activity summary plus the layer's weight geometry predicts the full interference-versus-threshold curve of GPT-2's MLP layers blind, on virgin text, with zero parameters fitted to the evaluation data and every prediction committed to a public git record before scoring. Under a domain convention declared in advance (the rectified post-GeLU code through the asymmetric read-write cross-Gram), a single sealed sweep passes 11 of 12 GPT-2 layers at the strictest pre-registered grade; pythia-70m, chosen adversarially, passes 6 of 6 at twice the code density: 18 of 18 native MLP layers across two families, blind, one pre-declared convention, zero free parameters. Three findings ride with the law: an inversion (the layers hardest for SAE dictionaries are the easiest native layers — the difficulty belongs to the analysis tools); a domain decomposition (the rectified component is lawful, the GeLU leak is a separate bounded object whose inclusion had manufactured a false 28.5% plateau); and a route made of five consecutive pre-registered failures that eliminated every dependence explanation and thereby located the convention error. Kills are reported with the same completeness as the confirms. Author contribution and use of AI: the research program and claims are the author's; experiments and drafting were done in collaboration with Claude, an AI system by Anthropic, under the author's direction, who verified the results and is responsible for the work. See the corresponding section in the PDF.
No comments yet — start the discussion below.