Context Token Frequency-Based Logit Correction for Improving Context Faithfulness in Generative Language Models

Counts differ because each database indexes a different set of publications. We treat OpenAlex as the canonical count; Google Scholar is not shown (no API, and crawling it violates its ToS).

In industrial and engineering question answering over manuals, specifications, contracts, and policy documents, generated answers must remain faithful to source text. Although Retrieval-Augmented Generation (RAG) provides relevant context, Large Language Models (LLMs) may still paraphrase, omit, or distort key expressions during decoding, limiting reliability in source-critical settings. Existing decoding-time methods help but often add inference overhead. This study proposes Context-Frequency Decoding (CFD), a training-free, non-invasive, lightweight method that adds a normalized context-token-frequency bias only to the final logits of an instruction-tuned LLM without modifying parameters or internal representations. As a plug-in soft bias, CFD increases the selection probability of frequent context tokens for source-grounded extraction. Experiments on three extractive question answering datasets across two LLMs show that CFD improves context-grounded answer extraction on specialized-document datasets by 9.3-29.6% over standard decoding and outperforms baselines. CFD remains stable across bias strengths and adds less than 1% additional per-token latency, whereas baselines incur substantially higher decoding cost. These results suggest that CFD is practical for extractive question answering systems requiring source-grounded answer extraction.