Emma Bakker, Simon Lupart, Balazs Borsos, Dorottya Papp · ACM International Conference on Information and Knowledge Management (CIKM) 2026 · 2026
DOI: 10.1145/3799682.3840207
Counts differ because each database indexes a different set of publications. We treat OpenAlex as the canonical count; Google Scholar is not shown (no API, and crawling it violates its ToS).
Technical-medical documentation poses a challenging setting for grounded retrieval-augmented generation (RAG): valid answers often depend on exact thresholds, workflow constraints and combinations of documented conditions. However, existing RAG benchmarks rarely provide the fine-grained evidence, completeness, citation, abstention and safety annotations needed to evaluate such behaviour. We introduce QFR-RAG, a multi-level evaluation resource for grounded RAG over Quantitative Flow Ratio (QFR) documentation for cardiology software used in angiography-based assessment of coronary disease. QFR-RAG comprises a 400-question benchmark: 50 technical extraction questions with reference answers and gold evidence, 50 multi-evidence questions annotated with 103 required evidence slots and 278 atomic answer nuggets and 300 paired adversarial questions targeting nonsensical requests, false premises and unsafe relaxation of documented constraints. Its layered evaluation protocol separates retrieval sufficiency, exact correctness, nugget-level completeness, citation grounding, abstention and safety-preserving refusal or correction, revealing empirically distinct failure modes across retrieval and generation. As a baseline validation, we benchmark retrieval pipelines and general-purpose and medically specialised language models of different sizes, showing that strong performance on exact technical extraction does not imply complete multi-evidence answering. QFR-RAG provides a deeply annotated framework for evaluating grounded, complete and safety-aware RAG in controlled technical-medical software documentation and illustrates an annotation design that can be adapted to similar parameter-dense domains.
ACM International Conference on Information and Knowledge Management (CIKM) 2026 · 0 citations
No comments yet — start the discussion below.