Lin Chunhui, Qiuhao Xie, Muquan Li, Deyin Li, Dongyang Zhang, Ke Qin · ACM International Conference on Multimedia (ACM MM) 2026 · 2026
DOI: 10.1145/3767308.3835326
Counts differ because each database indexes a different set of publications. We treat OpenAlex as the canonical count; Google Scholar is not shown (no API, and crawling it violates its ToS).
Data-free model merging offers an attractive way to consolidate multiple Multimodal Large Language Models (MLLMs) into a single system without retraining, reducing deployment cost while potentially combining complementary multimodal capabilities. However, existing heterogeneous MLLM merging methods primarily focus on mergeability under architectural heterogeneity while overlooking a more fundamental issue: preserving multimodal alignment. Here, multimodal alignment refers to the coordination between visual encoding and language reasoning encoded in language-side parameters. As a result, source information may be transferred along incompatible directions, silently disrupting vision--language coordination even when local parameter matching appears reasonable. To address this challenge, we propose APT-Merging, a strictly data-free alignment-preserving knowledge transfer framework for heterogeneous MLLM merging. APT-Merging comprises three key components: (1) a subspace extraction strategy based on truncated SVD that exposes the alignment-critical directional structure of the original model; (2) a dynamic mapping mechanism that aligns source components with compatible directions in the original model while filtering directionally incompatible information; and (3) a protective injection scheme that transfers only reliability-gated positive surplus along singular directions supported by the original model, thereby preventing off-subspace perturbations and reducing negative transfer. Extensive experiments on two heterogeneous MLLM settings and eight vision-language benchmarks demonstrate that APT-Merging achieves the strongest overall performance among compared data-free baselines. Further ablations show that alignment preservation is the most influential factor in achieving robust heterogeneous MLLM merging.
arXiv (Cornell University) 2026 · 0 citations
No comments yet — start the discussion below.