Scene Graph Thinking: Reinforcing Structured Visual Reasoning for Multimodal Large Language Models
Quick Answer
This paper shows that The Scene Graph Thinking (SaGe) paradigm enhances Multimodal Large Language Models (MLLMs) by integrating structured visual reasoning through scene graphs, achieving significant improvements across eight multimodal benchmarks.
Quick Take
The Scene Graph Thinking (SaGe) paradigm enhances Multimodal Large Language Models (MLLMs) by integrating structured visual reasoning through scene graphs, achieving significant improvements across eight multimodal benchmarks. The approach includes an automated data engine for creating structured scene graphs and a two-stage graph-aligned post-training method, resulting in enhanced fine-grained perception and reasoning capabilities.
Key Points
- Introduces automated data engine for converting image-text corpora into structured scene graphs.
- Constructs 120K high-quality training data by sampling reasoning traces from scene graphs.
- Implements two-stage graph-aligned post-training for structured reasoning internalization.
- Achieves significant performance improvements on eight multimodal benchmarks.
- Demonstrates strong effectiveness in fine-grained perception and reasoning tasks.
Paper Resources
📖 Reader Mode
~2 min readAbstract:Multimodal Large Language Models (MLLMs) have demonstrated strong perception and reasoning capabilities. However, most existing models focus on isolated objects and neglect structured relationships for efficient target navigation, limiting their performance on visually intensive tasks. To address this challenge, we introduce Scene Graph Thinking (SaGe), a novel paradigm that enables fine-grained and structured visual reasoning through explicit scene-graph representations. Specifically, we first introduce an automated data engine that converts flat image-text corpora into structured scene graphs, where hierarchical entities constitute the nodes and diverse visual relations define the edges. Building upon this, we construct 120K high-quality training data by sampling reasoning traces from scene graphs. Then, two-stage graph-aligned post-training paradigms are introduced, where supervised fine-tuning internalizes MLLMs with structured reasoning, and subsequent reinforcement fine-tuning proposes node-as-proxy graph rewards to consolidate efficient graph exploration. With curated data and graph-aligned training, our approach achieves significant improvements across eight multimodal benchmarks, demonstrating strong effectiveness on fine-grained perception and reasoning tasks. Code is available at this https URL.
| Comments: | ICML 2026 |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV) |
| Cite as: | arXiv:2607.05716 [cs.CV] |
| (or arXiv:2607.05716v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2607.05716 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Zhiwei Yang [view email]
[v1]
Tue, 7 Jul 2026 01:00:51 UTC (19,044 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CV
See more →LLM-Guided ANN Index Optimization for Human-Object Interaction Retrieval
A phase-aware LLM agent optimizes human-object interaction retrieval, outperforming Optuna TPE by 33.3% and VDTuner by 34.2% on the HICO-DET benchmark. This method enhances throughput by 15.3x over UniIR and demonstrates strong transferability across vector database management systems.