Current Research and Open Gaps in Federated Large Language Models: A Systematic Review

Counts differ because each database indexes a different set of publications. We treat OpenAlex as the canonical count; Google Scholar is not shown (no API, and crawling it violates its ToS).

The integration of Large Language Models (LLMs) with Federated Learning (FL), known as FedLLMs, enables decentralized model training while preserving data privacy. However, this integration introduces novel challenges while also inheriting existing issues from FL and LLMs. This systematic review addresses the research question: How are the key challenges in Federated Large Language Models being addressed in current research, and what gaps remain to be explored? We analyzed 168 studies across four major databases, applying eligibility criteria that resulted in 24 studies for detailed synthesis. Findings were organized across twelve key challenges: multimodal foundation model integration; data erasure, legal, ethical, and property rights; environmental sustainability and energy efficiency; federated learning over black-box LLMs; personalization and domain adaptation; continual learning and continuous data streams; communication overhead and efficiency constraints; security and robustness; heterogeneous data, devices, and resources; computational efficiency and resource constraints; and data privacy preservation. While some challenges have received attention, solutions are often limited to narrow domains or experimental datasets, and several remain unaddressed. These insights highlight critical gaps and provide a foundation for future research toward generalizable, responsible, and sustainable LLM technologies.