CoRA: Confidence-Rationale Alignment for Reliable Chain-of-Thought Reasoning
Quick Answer
The CoRA framework enhances chain-of-thought reasoning in LLMs by aligning confidence with rationale support, reducing alignment errors by up to 26.51% across MedQA, MathQA, and OpenBookQA benchmarks.
Quick Take
This method utilizes a -based reinforcement learning approach, ensuring that confident answers are backed by substantial rationales, thus improving model reliability.
Key Points
- CoRA reduces confidence-rationale alignment errors by up to 26.51%.
- Utilizes a GRPO-based reinforcement learning framework for improved reasoning.
- Maintains competitive accuracy while enhancing calibration of .
- Evaluated on MedQA, MathQA, and OpenBookQA benchmarks.
- Emphasizes the importance of substantial rationales for confident answers.
Paper Resources
Source Excerpt
arXiv:2606. 14961v1 Announce Type: new Abstract: Chain-of-thought (CoT) reasoning can improve performance, but high answer confidence may be misleading when the accompanying CoT rationale is plausible yet incomplete or poorly supported. We study confidence--rationale alignment: whether a model's confidence in its committed answer is justified by its generated rationale.
We introduce a -based reinforcement learning framework that jointly rewards answer correctness, committed-answer probability, and rubric-based rationale support, where the rubric assesses grounding, coherence, task match, and connection to the selected answer without revealing the gold answer to the judge. …
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →TriAgent: Divergence-Aware Committees for Cost-Efficient Financial Sentiment Analysis
TriAgent introduces a cost-efficient multi-agent system for financial sentiment analysis, combining VADER, FinBERT, and Qwen2.5. It achieves an F1 score of ~0.87 with significant savings of $9.3M/year at a 10M-user scale compared to GPT-4o-mini, while also detecting hallucinations with an AUC of 0.90.