SVoT: State-aware Visualization-of-Thought for Spatial Reasoning via Reinforcement Learning
Quick Answer
This paper shows that The State-aware Visualization-of-Thought (SVoT) framework enhances spatial reasoning in Multimodal Large Language Models (MLLMs) by generating verifiable intermediate states and visualizations.
Quick Take
Trained via (GRPO), SVoT achieves state-of-the-art performance with up to 65% accuracy gain on out-of-distribution test sets across five newly established domains, including Pacman and Gather.
Key Points
- SVoT integrates transition reasoning chains for improved multi-hop spatial reasoning.
- The framework generates interleaved, verifiable intermediate states and visualizations.
- Five domains established for systematic evaluation include novel environments like Pacman.
- SVoT achieves state-of-the-art performance with a 65% accuracy gain on specific test sets.
- Group Relative Policy Optimization (GRPO) is used for effective training and reward design.
Paper Resources
Article Content
From source RSS / original summaryarXiv:2606. 11770v1 Announce Type: new Abstract: Spatial reasoning remains a challenge for Multimodal (MLLMs), as it requires reliable multi-hop inference over both intermediate states and state transitions. Current studies often leave intermediate states unverified and treat state transitions as implicit processes, which limits reliability in multi-hop spatial reasoning.
To address this, we propose State-aware Visualization-of-Thought (SVoT), a reinforcement learning framework that generates interleaved, verifiable intermediate states and visualizations. SVoT integrates transition reasoning chains into the generation processes, enabling the model to verify action preconditions and effects through interleaved textual and visual reasoning.
We train SVoT via (GRPO), instantiating verification through reward design and evaluating the efficacy of different fine-grained rewards. As existing benchmarks reduce state transitions to single-variable updates, substantially simplifying the problems, we establish five domains by extending classical environments and introducing two novel domains, Pacman and Gather, that require multi-object interactions and numerical reasoning.
These domains support systematic evaluation of multi-hop spatial reasoning with quantitative verification of generated intermediate states and transition reasoning. SVoT with transition-aware supervision achieves state-of-the-art performance across the introduced domains, yielding up to a 65% absolute accuracy gain on out-of-distribution test sets.
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.AI
See more →HOBA: Hierarchical On-Policy Bidding Agents for Adaptive Online Advertising
HOBA (Hierarchical On-policy Bidding Agents) is a novel hierarchical reinforcement learning framework that enhances online advertising bidding systems by improving adaptability and reducing hyperparameter tuning costs. It utilizes a for hyperparameter inference, a SARSA agent for expert model selection, and a dynamic expert pool for bid execution, achieving a +3.6% increase in target cost during large-scale deployment and outperforming state-of-the-art baselines on AuctionNet.