Jiaying Zhu, Yurui Zhu, Xin Biao Lu, Wenrui Yan, Dong Li, Kunlin Liu, Xueyang Fu, Zheng-Jun Zha · ACM International Conference on Multimedia (ACM MM) 2026 · 2026
DOI: 10.1145/3767308.3835408
Counts differ because each database indexes a different set of publications. We treat OpenAlex as the canonical count; Google Scholar is not shown (no API, and crawling it violates its ToS).
Multimodal Large Language Models (MLLMs) encounter significant computational and memory bottlenecks from the massive number of visual tokens generated by high-resolution images or multi-image inputs. Previous token compression techniques are often constrained by heuristic rules that risk discarding critical information. They may suffer from biases, such as attention sinks, that lead to sharp performance drops under aggressive compression ratios. To address these limitations, we reformulate token compression as a lightweight plug-and-play framework and turn it into an end-to-end learnable decision process. Specifically, we propose VisionSelector, a scorer module decoupled from the MLLM backbone that incorporates a differentiable Top-K mechanism and a curriculum annealing strategy to bridge the training–inference gap, enabling efficient and adaptive token selection across various compression rates. Remarkably lightweight with only 12.85M trainable parameters, VisionSelector demonstrates generalization across various compression rates and adaptively identifies critical tokens. This leads to superior performance across the evaluated compression budgets, evidenced by preserving 100% performance on MME with a 30% retention budget, outperforming representative heuristic baselines by 12.14 percentage points at a 10% retention budget, and doubling prefill speed. Our code is available at https://github.com/JulietChoo/VisionSelector.
arXiv (Cornell University) 2026 · 0 citations
Last synced
This work has 14 recorded citations, but citing papers have not been linked locally yet.
No comments yet — start the discussion below.