Application of Large Language Models and Natural Language Generation in Content Creation: A Retrieval-Augmented and Preference-Aligned Framework

Counts differ because each database indexes a different set of publications. We treat OpenAlex as the canonical count; Google Scholar is not shown (no API, and crawling it violates its ToS).

Large language models (LLMs) have substantially advanced natural language generation (NLG), but high-quality content creation remains constrained by hallucination, limited controllability, retrieval noise, domain shift, and computational cost. This study proposes Retrieval-Augmented and Preference-Aligned Generation (RAP-Gen), a unified framework that integrates retrieval-augmented generation, factual preference alignment, in-context learning, and denoising pre-training. The framework combines a dense semantic retriever with an encoder–decoder generator, introduces pairwise factual-preference supervision for controllable generation, aligns retrieval and generation distributions, and uses structured text corruption to improve robustness to incomplete or noisy inputs. Experiments are reported on summarization, long-form creative writing, and knowledge-intensive question answering using CNN/DailyMail, WritingPrompts, Natural Questions, and general language-modeling data. The reported results show that RAP-Gen improves ROUGE, BERTScore, perplexity, FactScore, and human-rated factuality relative to the baselines listed in the manuscript. Ablation results indicate that removing retrieval, preference alignment, or denoising degrades performance, supporting the complementary roles of external knowledge, preference signals, and robust pre-training. Additional analyses examine the number of retrieved documents, attention over retrieved evidence, robustness under input perturbation, model scaling, and in-context learning. Because the supplied material does not include raw predictions, code, repeated-run variance, annotator agreement statistics, or complete benchmark configurations, the numerical results are interpreted as reported experimental outcomes rather than independently reproducible evidence. The study contributes an integrated design for knowledge-grounded, preference-aware content generation and identifies evaluation and reproducibility requirements for future work.