Arnold's Bifurcation Principle Applied to AdamW Training: Real-Time Hardware Early Warning of Second-Moment Collapse Preceding Loss Spikes

Counts differ because each database indexes a different set of publications. We treat OpenAlex as the canonical count; Google Scholar is not shown (no API, and crawling it violates its ToS).

Large-scale AI training runs using the AdamW optimizer – the optimizer used by essentially every major AI lab and company training large models today – are frequently disrupted by loss spikes: sudden, sharp jumps in training loss that can derail days or weeks of expensive compute. Recent published work (Bai et al., 2025) identified a specific, AdamW-only mechanism behind many of these spikes: a parameter’s Adam second-moment estimate can quietly decay toward an abnormally small value over an extended stretch of training, silently inflating that parameter’s effective step size, until an ordinary-sized gradient arrives and triggers a disproportionately large, destabilizing update. This mechanism is not our discovery. What we show here is that this decay has a detectable statistical signature, observable directly from ˆ v(t) – a quantity every AdamW optimizer already computes internally, at every step, for every parameter, at zero additional cost – and that a simple, fixed, training-free precursor rule can flag a parameter heading toward this failure mode well before the destabilizing gradient arrives. In a controlled synthetic reconstruction of this mechanism, our precursor rule correctly flagged 10 out of 10 danger cases, with zero false alarms across 10 independent healthy control runs, and a mean lead time of approximately 1,700 training steps ahead of the destabilizing event. All results here are from a synthetic reconstruction of the published mechanism, not real AdamW training logs; the natural next step, requiring no new hardware to test, is checking this same rule against real ˆ v(t) logs from an actual large-scale training run.