Revealing and Mitigating the Impact of LLM-Generated Content on Retrieval-Augmented Generation Systems

Counts differ because each database indexes a different set of publications. We treat OpenAlex as the canonical count; Google Scholar is not shown (no API, and crawling it violates its ToS).

In the era of large language models (LLMs), the Internet is flooded with LLM-generated content, including both aligned LLM-generated content that contains equivalent facts to human-written content and distorted LLM-generated content with factual discrepancies. However, there is a significant gap in understanding and addressing the impact of such content on Retrieval-Augmented Generation (RAG) systems, which rely on accurate retrieved knowledge to generate response. This paper comprehensively evaluate the impact of aligned and distorted LLM-generated content on the retrieval and generation stages of RAG systems. Notably, we reveal that distorted LLM-generated content is not only more likely to be retrieved but also significantly degrades generation quality, whereas aligned content can, in some cases, enhance it. To mitigate this, we propose a factual consistency-aware adaptive filtering approach to selectively filter out distorted LLM-generated content from retrieved documents. Experimental results demonstrate that our method significantly improves generative performance in scenarios that mix with LLM-generated content and is broadly applicable across various RAG systems.