The Art of Interrogation: Consistency Amplifies Factuality in Spatial Reasoning
Quick Answer
This paper introduces a self-supervised reinforcement learning framework to enhance spatial reasoning in Large Reasoning Models (LRMs) without ground-truth annotations.
Quick Take
This paper introduces a self-supervised reinforcement learning framework to enhance spatial reasoning in Large Reasoning Models (LRMs) without ground-truth annotations. By implementing consistency verifiers and an optimal transport-based RL strategy, OT-GRPO, the approach achieves accuracy comparable to supervised models while improving generalization across various tasks.
Key Points
- Current LRMs struggle with spatial reasoning tasks, often relying on supervised fine-tuning.
- The proposed framework uses self-supervised reinforcement learning to enhance internal reasoning.
- Consistency verifiers check geometric and semantic consistency, improving model performance.
- OT-GRPO is a new RL strategy tailored for pairwise verifiers, optimizing spatial reasoning.
- This approach achieves accuracy similar to ground-truth supervised models across diverse tasks.
Paper Resources
Article Content
From source RSS / original summaryarXiv:2606. 11918v1 Announce Type: new Abstract: Current Large Reasoning Models (LRMs) exhibit remarkable general capabilities but significantly underperform in spatial reasoning tasks. Existing approaches treat this gap as a knowledge deficit, relying on supervised fine-tuning (SFT) to ingest labeled spatial data from external vision sources or synthetic engines.
In contrast, we argue that for many tasks, spatial reasoning capabilities are already present in pre-trained LRMs but require alignment through logical coherence under geometric 2D and 3D constraints. In this work, we propose a self-supervised reinforcement learning (RL) framework that targets the internal reasoning process without requiring ground-truth annotations.
By formalizing the notion of consistency verifiers -- reward functions that check for geometric and semantic consistency under transformations -- we demonstrate that models can improve their spatial reasoning abilities. We use both image transformations, like flipping, and textual transformations, like swapping the order of objects in the question, and propose a new optimal transport-based RL strategy, OT-GRPO, which is a minimal-matching variant of group relative policy optimization tailored to pairwise verifiers.
We show that this label-free consistency training approaches the accuracy of models trained with ground-truth supervision and achieves similar generalization across diverse tasks and data domains.
Reader Mode unavailable (could not extract clean content).
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.AI
See more →Arbor: Tree Search as a Cognition Layer for Autonomous Agents
Arbor introduces a multi-agent framework utilizing structured tree search for optimizing LLM inference, achieving up to 193% throughput-latency improvement compared to vendor-optimized systems. It employs an Orchestrator and Critic agent for stability and coordination, demonstrating hardware-agnostic performance with minimal variance.