
Vo Anh Khoa Nguyen · Zenodo (CERN European Organization for Nuclear Research) 2026 · 2026
DOI: 10.5281/zenodo.24243286
Counts differ because each database indexes a different set of publications. We treat OpenAlex as the canonical count; Google Scholar is not shown (no API, and crawling it violates its ToS).
Autoregressive generation in Large Language Models (LLMs) deployed within extended multi-turn dialogs and autonomous agentic workflows suffers from an acute memory wall. Because conventional Key-Value (KV) caching retains representations for all preceding tokens, memory consumption scales linearly O(N) with sequence length and batch size, precipitating catastrophic Out-Of-Memory (OOM) failures or massive resource underutilization. In this work, we present VANTAGE-V-AI (v8.2.0), a high-performance inference runtime designed to bound KV-cache memory consumption to a strictly fixed budget O(W) while maintaining numerical parity and stability. VANTAGE integrates three complementary mechanisms: (1) Streaming Attention Sinks, which persistently retain initial anchor tokens to anchor attention distribution scores alongside a recency sliding window, eliminating softmax entropy collapse; (2) 2-Bit Packed Ternary Quantization ({-1, 0, +1}) packing four ternary values per byte, achieving an 8x reduction in KV-cache storage; and (3) a drop-in runtime architecture that wraps standard Hugging Face Transformer decoders in two lines of code without retraining. We empirically benchmark VANTAGE on dual NVIDIA Tesla T4 GPUs across a 1,024-token prompt using Qwen/Qwen2.5-3B-Instruct. Following MLSys asynchronous execution protocols, we report end-to-end wall-clock timing including host-device PCIe transfers and multi-GPU synchronization. The system establishes 100.0% pairwise token parity (64/64 tokens) against the uncompressed baseline, achieves 0.988 average coherence in long-context Needle-In-A-Haystack retrieval, and prevents explosive softmax perplexity divergence.
No comments yet — start the discussion below.