VeryTrace: Verifying Reasoning Traces through Compilable Formalism and Structured Verification
Quick Answer
VeryTrace is a zero-shot verification framework that formalizes reasoning traces into a structured representation, enhancing accuracy in multi-step reasoning tasks across domains like mathematics and robotics.
Quick Take
VeryTrace is a zero-shot verification framework that formalizes reasoning traces into a structured representation, enhancing accuracy in multi-step reasoning tasks across domains like mathematics and robotics. It utilizes a Domain-Specific Language to clarify dependencies and improve error localization, achieving better results than zero-shot baselines on state-of-the-art LLMs without requiring specific training.
Key Points
- Introduces a Domain-Specific Language for explicit step dependencies and executable expressions.
- Combines deterministic checks with LLM audits for effective error localization and repair.
- Demonstrates improved accuracy in competition mathematics, robotics planning, and kinship reasoning.
- Achieves better performance than zero-shot baselines on state-of-the-art LLMs.
- No domain-specific training or in-context examples required for enhanced generalization.
Paper Resources
Article Content
From source RSS / original summaryarXiv:2606. 24124v1 Announce Type: new Abstract: Multi-step reasoning with Chain-of-Thought (CoT) prompting remains fragile: logical errors or hallucinations in early steps silently propagate, producing confident but incorrect conclusions. This paper presents VeryTrace, a zero-shot verification-and-repair framework that formalizes natural-language reasoning traces into a structured, compilable representation.
VeryTrace introduces a Domain-Specific Language (DSL) that (i) makes step dependencies explicit, (ii) mechanizes quantitative content as executable expressions, and (iii) structures semantic inferences via deduction schemas. Our hybrid verifier combines deterministic checks for computational correctness, dependency resolution, and constraint satisfaction with targeted LLM audits for non-mechanizable semantic judgments, enabling step-level error localization and repair.
Across three diverse domains-competition mathematics (AIME 2025), robotics planning (LLM-BabyBench), and kinship reasoning (CLUTRR), VeryTrace improves accuracy over zero-shot baselines on state-of-the-art LLMs without requiring domain-specific training or in-context examples, demonstrating that formalized trace verification achieves both precision and generalization.
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.AI
See more →Arbor: Tree Search as a Cognition Layer for Autonomous Agents
Arbor introduces a framework utilizing structured tree search for optimizing LLM inference, achieving up to 193% throughput-latency improvement compared to vendor-optimized systems. It employs an Orchestrator and Critic agent for stability and coordination, demonstrating hardware-agnostic performance with minimal variance.