Chuxiao Zuo, Yao Zhu, Minqiang Xu, Manhong Wang, Yunke Zhang, F T Huang · ACM International Conference on Multimedia (ACM MM) 2026 · 2026
DOI: 10.1145/3767308.3836615
Counts differ because each database indexes a different set of publications. We treat OpenAlex as the canonical count; Google Scholar is not shown (no API, and crawling it violates its ToS).
Multimodal speaker identification systems face two key challenges in real-world deployment: missing modalities and language mismatch between training and testing conditions. In practical scenarios, background multi-speaker conversations, ambient noise, and overlapping speech further degrade identification accuracy. To address these challenges, we propose a multimodal polyglot speaker identification system for the POLY-SIM 2026 Grand Challenge. The system is fundamentally built upon Adaptive Modality Routing (AMR), a modality fusion module that dynamically assesses per-sample input quality and integrates modality information. Specifically, AMR employs two modality adapters to process the embeddings extracted from a linguistically robust audio encoder (W2V-BERT 2.0) and a large-scale pretrained face encoder (IResNet-18), producing modality-adapted embeddings. Based on these adapted embeddings, a trainable router estimates dynamic modality weights, which are subsequently applied to aggregate the modality-specific logits for the final prediction. To optimize this routing mechanism, we adopt a modality-aware training strategy that constructs four types of sample pairs to simulate diverse input conditions, with KL divergence serving as explicit supervision for weight assignment. Experimental results on the POLY-SIM 2026 evaluation set show that the proposed system achieves identification accuracy of 99.93% (English multimodal, P3), 100.00% (Urdu multimodal, P5), 97.50% (English audio-only, P4), and 98.83% (Urdu audio-only, P6). The average accuracy across all four protocols is 99.07%,surpassing the Fusion and Orthogonal Projection (FOP) baseline by 32.73 percentage points.
ACM International Conference on Multimedia (ACM MM) 2026 · 1 citations
ACM International Conference on Multimedia (ACM MM) 2026 · 1 citations
ACM International Conference on Multimedia (ACM MM) 2026 · 1 citations
Last synced
This work has 1 recorded citation, but citing papers have not been linked locally yet.
No comments yet — start the discussion below.