Improving LLMs via Validator-to-Generator Alignment
Quick Answer
The study introduces FCPA, a training objective that enhances generator-validator consistency in large language models (LLMs) by correcting utterance frequency.
Quick Take
Experimental results show improvements of up to 27 percentage points in Pearson correlation on benchmarks like IFEval and HumanEval, while maintaining validator quality across tasks.
Key Points
- Introduces FCPA for frequency-corrected generator-validator consistency in .
- Achieves up to 27 percentage points improvement in Pearson correlation on benchmarks.
- Maintains validator quality across all evaluated tasks.
- Addresses the generator-validator gap in LLM outputs.
- Demonstrates effectiveness over prior methods in real-world applications.
Paper Resources
📖 Reader Mode
~2 min readAbstract:Large language models are inconsistent: varying prompts or including unrelated information can lead to unexpected changes in model outputs. The generator-validator (G-V) gap is one manifestation of this phenomenon, where LLMs generate responses that they then deem as invalid if re-queried to validate them. In this work, we introduce a new formulation of G-V consistency that involves a principled correction for utterance frequency. Specifically, generators often assign low likelihood to valid strings simply because those strings are a priori unlikely, which makes naive notions of G-V consistency unworkable. We show that under a natural model of rational agents answering questions with multiple answers, consistency of the validator with a frequency-corrected generator score emerges naturally. Our method, \emph{\FCPAname} (\FCPA), is a training objective implementing frequency-corrected G-V consistency for real-world LLMs. Our experimental results show that training with \FCPA{} substantially improves both G-V consistency and generator performance over prior methods, with gains of up to $+27$pp in Pearson correlation on IFEval and HumanEval, while preserving validator quality across all evaluated tasks.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2607.02668 [cs.CL] |
| (or arXiv:2607.02668v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2607.02668 arXiv-issued DOI via DataCite |
Submission history
From: Juan Diego Rodriguez [view email]
[v1]
Thu, 2 Jul 2026 18:00:39 UTC (3,550 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →TriAgent: Divergence-Aware Committees for Cost-Efficient Financial Sentiment Analysis
TriAgent introduces a cost-efficient multi-agent system for financial sentiment analysis, combining VADER, FinBERT, and Qwen2.5. It achieves an F1 score of ~0.87 with significant savings of $9.3M/year at a 10M-user scale compared to GPT-4o-mini, while also detecting hallucinations with an AUC of 0.90.