GeoToken-DPO: Token-Level Preference Alignment of Geoscience Language Models under Observational Evidence Sparsity

Counts differ because each database indexes a different set of publications. We treat OpenAlex as the canonical count; Google Scholar is not shown (no API, and crawling it violates its ToS).

Domain-specialized language models for the Earth, environmental and space sciences are typically obtained through continued pre-training and supervised instruction tuning, and their outputs are then aligned with sequence-level preference objectives such as Direct Preference Optimization. We show that this choice is mismatched to geoscience text, where factual correctness is determined by a sparse set of evidence-bearing tokens—numerical values, physical units, temporal ranges, confidence intervals and variable identifiers—while the remaining tokens are connective fillers. Because the preference signal is shared uniformly across the sequence, these evidence tokens contribute a vanishing fraction of the gradient, and the deficiency is amplified for answers that span several thousand tokens. We propose GeoToken-DPO, a token-level alignment framework with three components. GeoEvidence Aware Weighting replaces the fixed positional prior of prior token-level methods with a learned product of gradient attribution and a lightweight geoscience evidence classifier, augmented by a differentiable dimensional-consistency penalty that discourages unit and magnitude violations. Evidence-Graded Preference Margin replaces binary chosen/rejected supervision with an evidence strength scalar aggregated from retrieved scientific passages, yielding a margin that adapts to how well each candidate is grounded. Position-Aware Gradient Recalibration rescales gradients across relative-position buckets so that terminal evidence is not diluted in long answers, while block-wise checkpointing keeps memory tractable. Starting from the same ClimateGPT checkpoint and corpus, GeoToken-DPO raises the weighted average over climate-specific benchmarks from 77.1 for supervised fine-tuning and 79.6 for the strongest token-level baseline to 82.6, improves misinformation detection by 8.9 points, transfers to ocean science questions with a 5.4-point gain, and reduces training memory by 34.6% relative to a comparable token-level method. Ablations attribute the improvement to the evidence-aware weighting and the graded margin, with the positional recalibration dominating on answers beyond two thousand tokens.