Yishuo Yan, Zhe Xue, Sunmuduo Li, Xiangyang Kong, Can Xiao, Junfeng Zhang · · 2026
DOI: 10.1145/3841452.3841488
Counts differ because each database indexes a different set of publications. We treat OpenAlex as the canonical count; Google Scholar is not shown (no API, and crawling it violates its ToS).
Multi-modal Knowledge Graphs (MMKGs) provide a natural substrate for knowledge-intensive question answering because they connect structured relational facts with entity-centered visual evidence. Recent LLM-based reasoning methods have improved multi-hop exploration over knowledge graphs, but extending them to MMKGs remains challenging: agents often observe graph structure only through opaque intermediate states, couple textual and visual evidence too early, and behave unstably in long-horizon search. To address these issues, we propose GraMMA (Graph-Conditioned Multi-Modal Agent), an inspectable framework for MMKG reasoning with three aligned designs: (1) structured subgraph context as explicit observation space, (2) relation-aware cross-modal verification for selective visual grounding, and (3) evolutionary policy refinement for robust exploration. GraMMA routes primarily with textual and structural cues, and invokes visual evidence only when needed through a dedicated CrossModal action. We construct a 300-question evaluation set from ImgFact through human-agent collaboration and evaluate GraMMA on four MMKG benchmarks. Experiments demonstrate that GraMMA substantially outperforms existing methods, particularly in multi-hop reasoning and visual-dependent question answering scenarios.
Engineering Applications of Artificial Intelligence 2026 · 0 citations
arXiv (Cornell University) 2026 · 0 citations
No comments yet — start the discussion below.