Chunk Size and Retrieval Depth Optimization in a Minimal Retrieval-Augmented Generation Pipeline

Counts differ because each database indexes a different set of publications. We treat OpenAlex as the canonical count; Google Scholar is not shown (no API, and crawling it violates its ToS).

Retrieval‑augmented generation (RAG) links a large language model to a set of texts. But RAG works well only when two settings are right: how big each piece of text (a "chunk") is, and how many pieces the model reads. People often choose these by habit, not by data. In this study we tested how both settings change the results of a simple RAG pipeline. We used a closed-domain set of 120 academic-regulation documents (246,800 tokens in total). We cut the documents into chunks of four sizes (128, 256, 512, and 1024 tokens) with 25% overlap, and we also tried one version with no overlap. Each chunk was turned into a vector with all-MiniLM-L6-v2 and stored in FAISS with exact inner-product search. Llama-3.1-8B-Instruct then answered questions using the retrieved chunks, at retrieval depths k = 1, 3, 5, and 10. We used 200 question-answer pairs: 120 answerable items as a development set (used to choose all settings) and a held-out test set of 60 answerable items plus 20 unanswerable items (used only once). On the development set, retrieval recall was best at 256-token chunks (recall = 0.825 at k = 5), while context precision fell steadily from 0.428 to 0.172 as chunks grew. Answer F1 was highest at k = 5 (71.3) and then dropped to 69.2 at k = 10, even though recall kept rising. Our final configuration was L = 256, O = 64 (25% overlap), k = 5. On the held-out test set this configuration reached an F1 of 70.1. That is 37.5 points higher than a closed-book baseline (p < 0.001) and 8.9 points higher than a BM25 baseline (p = 0.004). Correct abstention on unanswerable questions improved from 15.0% to 70.0%. Manual error analysis showed that 17.9% of the failures were caused by chunk-boundary fragmentation.