Hengyang Lu, Jipeng He, Mingcai Chen, Wei Hua Fang · ACM International Conference on Multimedia (ACM MM) 2026 · 2026
DOI: 10.1145/3767308.3835832
Counts differ because each database indexes a different set of publications. We treat OpenAlex as the canonical count; Google Scholar is not shown (no API, and crawling it violates its ToS).
Real-world multimodal datasets suffer from noisy correspondence, which degrades model performance. The existing methods rely on global similarity to measure mismatch between samples. However, they overlook a critical phenomenon, locally semantically inconsistent false positives, where high CLIP similarity scores are incorrectly assigned to semantically mismatched samples. Consequently, such false positive pairs are used in model training without being filtered out, which undermines the model's performance. To address this issue, we propose a novel Localized Consistency Mining (LoCM) method to enhance the reliability of vision-language models during fine-tuning on noisy correspondence data. The LoCM framework comprises two modules: (1) A Local Noise Discrimination module that identifies fine-grained inconsistency by complementing CLIP's global matching with local correspondence analysis, enabling further noise filtering. (2) A Local Mismatch Refinement module realigns filtered samples with the integration of local consistency discrimination, mitigating local inconsistencies in the refined samples of existing methods. Experiments on the Flickr30K, MS-COCO and real-world noisy CC152K datasets demonstrate that LoCM significantly outperforms state-of-the-art approaches. Specifically, at a 60% noise rate, it improves Rsum by 14.9 on Flickr30K and 7.5 on MS-COCO. LoCM also obtains a 3.1 improvement on CC152K.
ACM International Conference on Multimedia (ACM MM) 2026 · 0 citations
No comments yet — start the discussion below.