Yiru Wang, Zichong Gu, Yu Gao, An-Qing Jiang, Shuo Wang, Yuwen Heng, Hao Sun · ACM International Conference on Multimedia (ACM MM) 2026 · 2026
DOI: 10.1145/3767308.3836083
Counts differ because each database indexes a different set of publications. We treat OpenAlex as the canonical count; Google Scholar is not shown (no API, and crawling it violates its ToS).
Contemporary embodied multimedia systems demand agents capable of seamlessly bridging high-dimensional sensory streams, complex natural language instructions, and precise physical interactions. While Vision-Language-Action (VLA) models show promise in this direction, their deployment in safety-critical, real-time scenarios like autonomous driving is constrained by inefficient processing of redundant visual tokens, weak 3D spatial grounding, and an opaque semantic-to-control gap. We propose HiST-VLA, a hierarchical and spatio-temporal VLA framework specifically designed to handle the rigorous multimodal requirements of embodied systems. Our core contributions are: 1) A dynamic token sparsification mechanism that dynamically prunes redundant visual tokens based on self-similarity attention, serving as an efficient cross-modal alignment layer for high-throughput multi-view video streams. 2) A multi-stage, semantics-aligned hierarchical planner that bridges granular language commands with precise motion primitives, incorporating confidence-aware regularization for transparent uncertainty quantification. Extensive evaluation on the NAVSIM v2 benchmark demonstrates state-of-the-art performance, achieving an EPDMS of 88.6 on the open-loop Navtest set—surpassing prior SOTA methods by 1.5 points. Crucially, we provide in-depth analyses of cross-modal token efficiency, interpretability, and real-world deployment feasibility, establishing a robust paradigm for trustworthy embodied multimedia intelligence.
Last synced
This work has 6 recorded citations, but citing papers have not been linked locally yet.
No comments yet — start the discussion below.