
Yusuke Maeda · Zenodo (CERN European Organization for Nuclear Research) 2026 · 2026
DOI: 10.5281/zenodo.22167627
Counts differ because each database indexes a different set of publications. We treat OpenAlex as the canonical count; Google Scholar is not shown (no API, and crawling it violates its ToS).
A language model usually reads the past, updates its hidden state, and moves on. What changes if it forms a new present—and then looks back once more? This technical note presents an architecture for compact language models built around that question. The model maintains one evolving present state while drawing on several forms of evidence: explicit token retrieval through local and global attention, sequence history compressed into recurrent memory, representation changes accumulated across depth, and the variance hidden behind each attention-weighted average. These signals are brought together by a learned relational controller. It decides how the evidence should reshape the feed-forward transformation and how far the model should move toward its proposed next state. The final layer adds a distinctive second reading. Keys, transformed values, recurrent memory, and selected depth evidence remain fixed while the updated present generates a new query. The model therefore revisits the same past from a newly formed point of view. The architecture also includes multi-range future latent prediction, encouraging the present representation to carry information about near, middle, and more distant continuations. This design-stage report documents the motivation, computation graph, initialization, implementation details, and compute contract before large-scale pretraining. It establishes a concrete and testable architecture whose performance will be evaluated in future training and benchmark experiments. Included files: English and Japanese editions.
No comments yet — start the discussion below.