Yuan Liu, Saihui Hou, Shibei Meng, Jiabao Du, Yongzhen Huang · ACM International Conference on Multimedia (ACM MM) 2026 · 2026
DOI: 10.1145/3767308.3835328
Counts differ because each database indexes a different set of publications. We treat OpenAlex as the canonical count; Google Scholar is not shown (no API, and crawling it violates its ToS).
Image Difference Captioning (IDC) focuses on generating natural language descriptions of the differences between similar image pairs. While methods based on Multimodal Large Language Models (MLLMs) have significantly advanced the development of IDC, their generalizability to unseen environments and novel change types remains limited. Motivated by the generalizable knowledge acquisition of rule-based Reinforcement Learning (RL), we explore this paradigm to learn robust difference perception policies. However, applying rule-based RL to IDC presents two primary challenges: (1) Ambiguous rewards for open-ended outputs. The diversity of difference captions makes it difficult to provide verifiable rewards for policy optimization. (2) Inefficient difference perception policy learning. The limitations of MLLMs in difference-aware cross-modal alignment result in a suboptimal initial policy space for subsequent RL training. To overcome the aforementioned challenges, we propose IDC-RFT, a Reinforcement Fine-Tuning framework that utilizes rule-based Reinforcement Learning (RL) for IDC. First, to provide verifiable feedback for open-ended captions, we introduce a dual-dimensional caption reward mechanism, comprising the Multi-Change Structural Reward (MCSR) and Semantic Reward (SR). Second, aiming to improve policy learning efficiency, we develop a dual-modality contrastive optimization strategy by incorporating Text- and Vision-oriented Contrastive Difference Identification (T-CDI and V-CDI) as auxiliary tasks. Experiments across seven benchmarks demonstrate that IDC-RFT significantly outperforms state-of-the-art methods with identical backbones.
No comments yet — start the discussion below.