Gianluca De Stefano, Lea Schönherr, Giancarlo Pellegrino · ACM Workshop on Artificial Intelligence and Security (AISec) 2026 · 2026
DOI: 10.1145/3847352.3848095
Counts differ because each database indexes a different set of publications. We treat OpenAlex as the canonical count; Google Scholar is not shown (no API, and crawling it violates its ToS).
Retrieval Augmented Generation (RAG) is a technique commonly used to equip LLMs with out-of-distribution knowledge by collecting, indexing, retrieving, and providing to the model specific information useful to answer user questions. The knowledge is often collected from public document repositories, providing a gateway for indirect prompt injections: attackers may craft documents specifically optimized to control the generated answer. In this paper, we investigate the practical performance of optimization techniques in hijacking the outputs of RAG pipelines. First, we review the frameworks commonly used to build RAG applications, derive a prototypical architecture, and identify critical parameters. We then examine prior work, seeking techniques for performing indirect prompt manipulations. We adopted five techniques across two macro-families and evaluated the criteria used to assess their effectiveness. With this knowledge, we implement Rag-n-Roll , a framework to evaluate the end-to-end results of LLM-based pipelines, introducing new metrics to better understand the benign or malicious nature of the generated answers. Our results show that established evaluation practices are insufficient to precisely assess attacks against RAG-based pipelines, leading to overestimates of their success. Furthermore, while attacks on the default configuration can successfully control up to 70% of the generated answers, we show that these performances depend heavily on previously ignored pipeline parameters. Finally, we distill our findings into actionable tips for developers looking to harden their RAG pipelines.
Last synced
This work has 25 recorded citations, but citing papers have not been linked locally yet.
No comments yet — start the discussion below.