Ke Xiao, Qiyuan Wang, Christos Anagnostopoulos, Zhuoran Tan, Wenhao Li · ENLIGHTEN (Jurnal Bimbingan dan Konseling Islam) 2026 · 2026
Counts differ because each database indexes a different set of publications. We treat OpenAlex as the canonical count; Google Scholar is not shown (no API, and crawling it violates its ToS).
Driven by the imperative to leverage privacy-sensitive data scattered across decentralized devices, federated fine-tuning has emerged as a vital paradigm for adapting large language models without compromising data privacy. Yet, its practical efficacy is bottlenecked by severe client resource heterogeneity. Existing truncation-based methods typically couple the transmitted rank with the trainable rank, which (i) under-utilizes bandwidth on communication-rich but compute-limited clients and (ii) exacerbates truncation-induced gradient drift. To address this, we propose FedHera, a resource-decoupled framework that explicitly differentiates information reception from gradient optimization. FedHera employs a spectrum-preserving allocation strategy to maximize the transfer of global knowledge (via high-rank singular values) within bandwidth limits, irrespective of training constraints. Furthermore, we introduce a prefix-gating mechanism that utilizes the downloaded high-capacity basis as a frozen reference to guide local updates, thereby minimizing the optimization gap caused by aggressive truncation. Extensive experiments under different heterogeneous settings show that FedHera improves stability and accuracy over state-of-the-art baselines.
No comments yet — start the discussion below.