Chenyang Li, Kaige Li, Zeyu Jiang, Changhao Chen · ACM International Conference on Multimedia (ACM MM) 2026 · 2026
DOI: 10.1145/3767308.3835574
Counts differ because each database indexes a different set of publications. We treat OpenAlex as the canonical count; Google Scholar is not shown (no API, and crawling it violates its ToS).
Despite progress in Embodied AI, Vision-and-Language Navigation (VLN) systems remain vulnerable to adversarial visual disturbances. Most existing methods rely on white-box access to target model gradients, which is often unrealistic for real-world deployed systems and computationally exhaustive due to recursive backpropagation for optimization, limiting their applicability. While previous black-box methods predominantly target single-step, instantaneous decision tasks, they struggle to handle the task complexities and temporal dependencies of VLN. This highlights the need for a gradient-free attack method that can effectively disrupt the multi-step sequential perception–action loop using only observable inputs and outputs. Therefore, we propose AdvNav, a behavior-guided black-box adversarial attack framework that disturbs an agent's first-person views during navigation. To construct an informative surrogate objective for effective optimization guidance in gradient-free search under the black-box setting, we design a dual-granularity behavior-based feedback, aggregating a trajectory-level performance score representing overall navigation degradation, an action-level reward score considering the potential decision risk, and a deviation indicator, all of which are extracted from the agent's self-output behaviors. This feedback guides a hybrid optimization strategy that (i) heuristically tunes perturbation strength via adaptive updates and (ii) evolves noise spatial structure genetically, to iteratively discover the most disruptive noise configuration. Evaluated against Transformer-based model HAMT and LLM-based model MapGPT with two types of backbones on R2R dataset, AdvNav achieves 49.70%, 65.96%, and 87.30% Attack Success Rate, respectively. The result demonstrates the effectiveness and generality of AdvNav, reveals critical perception vulnerabilities and offers insights for the design of future resilient VLN models.
arXiv (Cornell University) 2026 · 0 citations
arXiv (Cornell University) 2026 · 0 citations
arXiv (Cornell University) 2026 · 0 citations
No comments yet — start the discussion below.