Larissa Pusch, Alexandre Courtiol, Tim Conrad · Journal of Web Semantics 2026 · 2026
DOI: 10.1016/j.websem.2026.100890
Counts differ because each database indexes a different set of publications. We treat OpenAlex as the canonical count; Google Scholar is not shown (no API, and crawling it violates its ToS).
Large Language Models (LLMs) have shown strong performance across language tasks, but remain limited in knowledge-intensive domains due to hallucinations, outdated information, and limited explainability. Text-based retrieval-augmented generation (RAG) helps ground model outputs in external sources but often struggles with multi-hop reasoning. Knowledge Graphs (KGs), in contrast, support precise, explainable querying, yet require specialized query languages that limit accessibility. Using an interactive framework in which users refine generated queries, we evaluate how well current LLMs can bridge this gap for non-expert users by generating and explaining Cypher graph queries and detecting faults across varying levels of structural and semantic complexity. Our multi-model evaluation provides insight into where models succeed and fail, how performance varies across domains, and which query features most affect accuracy. Our core quantitative evaluation is a 90-query benchmark on a synthetic movie KG that measures query explanation quality and fault detection across multiple LLMs, complemented by two smaller real-life query-generation experiments on a Hyena KG and the MaRDI (Mathematical Research Data Initiative) KG. Our findings demonstrate that transparent, auditable KG querying is feasible but not yet reliable without human oversight. The code is available at https://git.zib.de/lpusch/talk2cypher .
No comments yet — start the discussion below.