Haotian Jin, Yang Li, Quancai Liu, Sa Zhu, Haihui Fan, Feifei Dai, Bo Li · ACM International Conference on Multimedia (ACM MM) 2026 · 2026
DOI: 10.1145/3767308.3835580
Counts differ because each database indexes a different set of publications. We treat OpenAlex as the canonical count; Google Scholar is not shown (no API, and crawling it violates its ToS).
Large vision–language models (VLMs) have become increasingly powerful, but their security risks have risen just as sharply—especially backdoor attacks, in which a model behaves normally on clean inputs yet produces malicious outputs when a hidden trigger is present. To our knowledge, we are the first to expose how vulnerable vision–language models (VLMs) are to soft-prompt backdoors and introduce the Dual-stream Soft Prompt (DSP) Attack. DSP inserts tiny soft prompt channels into both the visual and textual branches, enabling output manipulation while keeping the entire backbone frozen. We further design two dual-stream prompt variants: Orthogonal Prompts (OP), where visual and textual prompts are independent, and Mutual-Guided Prompts (MGP), where the two prompts are directionally aligned. Experiments on several mainstream VLM architectures and public datasets show that DSP requires only a small poisoning budget and minimal additional parameters, yet markedly increases both single-trigger and dual-trigger attack success rates. These findings expose an overlooked threat surface and establish a stronger baseline for future research in defending against backdoor attacks on VLMs.
No comments yet — start the discussion below.