SVoT: State-aware Visualization-of-Thought for Spatial Reasoning via Reinforcement Learning
Quick Answer
This paper shows that The State-aware Visualization-of-Thought (SVoT) framework enhances spatial reasoning in Multimodal Large Language Models (MLLMs) by generating verifiable intermediate states and visualizations.
Quick Take
The State-aware Visualization-of-Thought (SVoT) framework enhances spatial reasoning in Multimodal Large Language Models (MLLMs) by generating verifiable intermediate states and visualizations. Trained via Group Relative Policy Optimization (GRPO), SVoT achieves state-of-the-art performance with up to 65% accuracy gain on out-of-distribution test sets across five newly established domains, including Pacman and Gather.
Key Points
- SVoT integrates transition reasoning chains for improved multi-hop spatial reasoning.
- The framework generates interleaved, verifiable intermediate states and visualizations.
- Five domains established for systematic evaluation include novel environments like Pacman.
- SVoT achieves state-of-the-art performance with a 65% accuracy gain on specific test sets.
- Group Relative Policy Optimization (GRPO) is used for effective training and reward design.
Paper Resources
Article Content
From source RSS / original summaryarXiv:2606. 11770v1 Announce Type: new Abstract: Spatial reasoning remains a challenge for Multimodal Large Language Models (MLLMs), as it requires reliable multi-hop inference over both intermediate states and state transitions. Current studies often leave intermediate states unverified and treat state transitions as implicit processes, which limits reliability in multi-hop spatial reasoning.
To address this, we propose State-aware Visualization-of-Thought (SVoT), a reinforcement learning framework that generates interleaved, verifiable intermediate states and visualizations. SVoT integrates transition reasoning chains into the generation processes, enabling the model to verify action preconditions and effects through interleaved textual and visual reasoning.
We train SVoT via Group Relative Policy Optimization (GRPO), instantiating verification through reward design and evaluating the efficacy of different fine-grained rewards. As existing benchmarks reduce state transitions to single-variable updates, substantially simplifying the problems, we establish five domains by extending classical environments and introducing two novel domains, Pacman and Gather, that require multi-object interactions and numerical reasoning.
These domains support systematic evaluation of multi-hop spatial reasoning with quantitative verification of generated intermediate states and transition reasoning. SVoT with transition-aware supervision achieves state-of-the-art performance across the introduced domains, yielding up to a 65% absolute accuracy gain on out-of-distribution test sets.
Reader Mode unavailable (could not extract clean content).
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.AI
See more →Arbor: Tree Search as a Cognition Layer for Autonomous Agents
Arbor introduces a multi-agent framework utilizing structured tree search for optimizing LLM inference, achieving up to 193% throughput-latency improvement compared to vendor-optimized systems. It employs an Orchestrator and Critic agent for stability and coordination, demonstrating hardware-agnostic performance with minimal variance.