LLM Hallucination of Citations in Economics Persists With Web‐Enabled Models

Counts differ because each database indexes a different set of publications. We treat OpenAlex as the canonical count; Google Scholar is not shown (no API, and crawling it violates its ToS).

Despite significant advances since 2023 in large language model (LLM) capabilities and the introduction of web‐enabled search functionality, citation hallucination remains a persistent problem in AI‐generated academic content. We test GPT‐4o, DeepSeek V3, web‐enabled GPT‐4o, and web‐enabled GPT‐5.1 across prompts derived from Journal of Economic Literature categories. Using a hierarchical citation validation framework, we find that even an advanced web‐enabled model produces non‐existent citations. While web‐enabled models show improvement over their predecessors, the error rates remain high for scholarly applications. Our findings suggest that the fundamental challenge of citation hallucination persists despite technological advances, with implications for academic integrity and the reliable use of AI in research contexts. We provide an online tool for researchers to run LLMs through the benchmark test using our prompts.