Ruibang Li, Guan Luo, Yiwei Zhang, Dongye Zhao, Bing Li, Jin Gang Gao, Fudong Ge, Bing Li, Weiming Hu · ACM International Conference on Multimedia (ACM MM) 2026 · 2026
DOI: 10.1145/3767308.3835141
Counts differ because each database indexes a different set of publications. We treat OpenAlex as the canonical count; Google Scholar is not shown (no API, and crawling it violates its ToS).
Standard softmax self-attention excels in global modeling but suffers from O(N2) complexity, limiting scalable deployment in vision applications. Linear recurrent attention models offer favorable efficiency, yet their compressed state transition can cause certain information retrieval deficiencies, and their causal nature is not aligned with 2D visual data. In this work, we revisit the hybridization of linear and softmax attention from a fresh layer-stacking perspective. First, through a systematic study on hybridization schedules, we demonstrate that sparsely inserting softmax attention into the middle to late layers effectively mitigates the long range retrieval bottlenecks of linear architectures. Second, guided by the structural demands of layer-wise hybrid backbones, we introduce VDelta, a high speed unidirectional visual linear recurrent block built upon Gated DeltaNet, with improved adaptation of causal recurrence to 2D visual modeling. By combining this hybridization schedule with the VDelta linear layer, we propose SoLA-Vision (Softmax-Linear Attention Vision), a hierarchical vision backbone featuring a fine-grained layer-wise design. By seamlessly integrating our macro and micro level designs, this hierarchical backbone achieves an exceptional trade-off among modeling capacity, computational complexity, and real world inference speed. For classification on the ImageNet-1K dataset, SoLA-Vision outperforms purely linear and other hybrid attention models, while achieving up to 4× higher throughput than conventional multidirectional visual recurrent models. In dense prediction tasks, SoLA-Vision consistently surpasses strong baselines by a considerable margin.
Last synced
This work has 1 recorded citation, but citing papers have not been linked locally yet.
No comments yet — start the discussion below.