The Art of Interrogation: Consistency Amplifies Factuality in Spatial Reasoning
Quick Answer
This paper introduces a self-supervised reinforcement learning framework to enhance spatial reasoning in Large Reasoning Models (LRMs) without ground-truth annotations.
Quick Take
By implementing consistency verifiers and an optimal transport-based RL strategy, OT-, the approach achieves accuracy comparable to supervised models while improving generalization across various tasks.
Key Points
- Current LRMs struggle with spatial reasoning tasks, often relying on supervised fine-tuning.
- The proposed framework uses self-supervised reinforcement learning to enhance internal reasoning.
- Consistency verifiers check geometric and semantic consistency, improving model performance.
- OT-GRPO is a new RL strategy tailored for pairwise verifiers, optimizing spatial reasoning.
- This approach achieves accuracy similar to ground-truth supervised models across diverse tasks.
Paper Resources
Article Content
From source RSS / original summaryarXiv:2606. 11918v1 Announce Type: new Abstract: Current Large Reasoning Models (LRMs) exhibit remarkable general capabilities but significantly underperform in spatial reasoning tasks. Existing approaches treat this gap as a knowledge deficit, relying on supervised fine-tuning (SFT) to ingest labeled spatial data from external vision sources or synthetic engines.
In contrast, we argue that for many tasks, spatial reasoning capabilities are already present in pre-trained LRMs but require alignment through logical coherence under geometric 2D and 3D constraints. In this work, we propose a self-supervised reinforcement learning (RL) framework that targets the internal reasoning process without requiring ground-truth annotations.
By formalizing the notion of consistency verifiers -- reward functions that check for geometric and semantic consistency under transformations -- we demonstrate that models can improve their spatial reasoning abilities. We use both image transformations, like flipping, and textual transformations, like swapping the order of objects in the question, and propose a new optimal transport-based RL strategy, OT-, which is a minimal-matching variant of group relative policy optimization tailored to pairwise verifiers.
We show that this label-free consistency training approaches the accuracy of models trained with ground-truth supervision and achieves similar generalization across diverse tasks and data domains.
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.AI
See more →HOBA: Hierarchical On-Policy Bidding Agents for Adaptive Online Advertising
HOBA (Hierarchical On-policy Bidding Agents) is a novel hierarchical reinforcement learning framework that enhances online advertising bidding systems by improving adaptability and reducing hyperparameter tuning costs. It utilizes a for hyperparameter inference, a SARSA agent for expert model selection, and a dynamic expert pool for bid execution, achieving a +3.6% increase in target cost during large-scale deployment and outperforming state-of-the-art baselines on AuctionNet.