VeryTrace: Verifying Reasoning Traces through Compilable Formalism and Structured Verification
Quick Answer
VeryTrace is a zero-shot verification framework that formalizes reasoning traces into a structured representation, enhancing accuracy in multi-step reasoning tasks across domains like mathematics and robotics.
Quick Take
It utilizes a Domain-Specific Language to clarify dependencies and improve error localization, achieving better results than zero-shot baselines on state-of-the-art without requiring specific training.
Key Points
- Introduces a Domain-Specific Language for explicit step dependencies and executable expressions.
- Combines deterministic checks with LLM audits for effective error localization and repair.
- Demonstrates improved accuracy in competition mathematics, robotics planning, and kinship reasoning.
- Achieves better performance than zero-shot baselines on state-of-the-art LLMs.
- No domain-specific training or in-context examples required for enhanced generalization.
Paper Resources
📖 Reader Mode
~2 min readAbstract:Multi-step reasoning with Chain-of-Thought (CoT) prompting remains fragile: logical errors or hallucinations in early steps silently propagate, producing confident but incorrect conclusions. This paper presents VeryTrace, a zero-shot verification-and-repair framework that formalizes natural-language reasoning traces into a structured, compilable representation. VeryTrace introduces a Domain-Specific Language (DSL) that (i) makes step dependencies explicit, (ii) mechanizes quantitative content as executable expressions, and (iii) structures semantic inferences via deduction schemas. Our hybrid verifier combines deterministic checks for computational correctness, dependency resolution, and constraint satisfaction with targeted LLM audits for non-mechanizable semantic judgments, enabling step-level error localization and repair.
Across three diverse domains-competition mathematics (AIME 2025), robotics planning (LLM-BabyBench), and kinship reasoning (CLUTRR), VeryTrace improves accuracy over zero-shot baselines on state-of-the-art LLMs without requiring domain-specific training or in-context examples, demonstrating that formalized trace verification achieves both precision and generalization.
| Comments: | Accepted at LM4Plan Workshop @ ICML 2026 |
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2606.24124 [cs.AI] |
| (or arXiv:2606.24124v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2606.24124 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Ninghan Zhong [view email]
[v1]
Tue, 23 Jun 2026 04:15:48 UTC (584 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.AI
See more →HOBA: Hierarchical On-Policy Bidding Agents for Adaptive Online Advertising
HOBA (Hierarchical On-policy Bidding Agents) is a novel hierarchical reinforcement learning framework that enhances online advertising bidding systems by improving adaptability and reducing hyperparameter tuning costs. It utilizes a for hyperparameter inference, a SARSA agent for expert model selection, and a dynamic expert pool for bid execution, achieving a +3.6% increase in target cost during large-scale deployment and outperforming state-of-the-art baselines on AuctionNet.