Cross-Domain Generalization of Lightweight Hallucination Detectors in Large Language Models

Counts differ because each database indexes a different set of publications. We treat OpenAlex as the canonical count; Google Scholar is not shown (no API, and crawling it violates its ToS).

Hallucination in Large Language Models (LLMs)—the generation of plausible but factually incorrect content—poses a critical risk across multiple real-world domains. In software development, AI-generated errors can introduce system failures, security vulnerabilities, and data loss. In healthcare, incorrect medical advice or misidentified drug interactions produced by an LLM can directly endanger patient safety. More broadly, the unchecked propagation of hallucinated information can create societal misinformation and public harm. Current hallucination detectors exhibit a significant weakness when transferred across domains. The linguistic characteristics of each task—including writing style, conversational tone, response length, and structural presentation—differ substantially. Dialogue responses are inherently conversational, while Summarization outputs are concise condensations of longer documents, both differing fundamentally from structured Question-Answering. In this work, we investigate whether four lightweight, compute-efficient features—semantic similarity, response length, lexical overlap, and unique word ratio—are sufficient to detect hallucinations robustly across all three domains. Unlike heavy black-box methods requiring multiple expensive LLM calls, these features compute in milliseconds, making them highly scalable for real-world deployment. Our empirical evaluation demonstrates that the combined feature set achieves an in-domain AUROC of 0.856—approaching the theoretical maximum of 1.0—while maintaining a strong out-of-distribution AUROC of 0.837 across unseen domains. The negligible generalization gap of only +0.019 indicates that these lightweight features capture domain-agnostic hallucination signals that transfer seamlessly across QA, Dialogue, and Summarization tasks, offering a robust and compute-efficient alternative to expensive LLM-in-the-loop evaluators.