Reasoning-Guided Part-Level Visual Grounding via Reinforcement Learning
Quick Answer
The proposed Object-Part Hierarchical Reflective Grounding (OP-HRG) method enhances part-level visual grounding in multimodal large language models by first localizing parent objects before their parts.
Quick Take
A 4B model trained with a part-aware framework outperforms 7B grounding models and SAM3 on benchmarks like PascalPart and PartImageNet, showcasing significant advancements in reasoning segmentation.
Key Points
- OP-HRG employs a coarse-to-fine strategy for part-level grounding.
- A self-check mechanism refines the localized part predictions.
- The 4B model outperforms larger 7B models on key benchmarks.
- Training utilizes stage-wise rewards via a part-aware GRPO framework.
- The approach significantly improves reasoning segmentation capabilities.
Paper Resources
📖 Reader Mode
~2 min readAbstract:Multimodal large language models (MLLMs) ground whole objects well from free-form language queries, but they struggle when the query names a part rather than the object. We trace this to a missing object-part hierarchy, since parts are localized in the same single step used for objects. We propose Object-Part Hierarchical Reflective Grounding (OP-HRG), a coarse-to-fine reasoning-guided grounding strategy that first localizes the parent object and then the part within it. A self-check then reflects on the result, with an extension to re-encode the predicted crop to inspect the region it is correcting. We introduce a part-aware GRPO framework to train our pipeline with stage-wise rewards. A 4B model trained this way outperforms 7B grounding LLMs and SAM3 across PascalPart, PartImageNet, and InstructPart, and transfers to reasoning segmentation.
| Comments: | ECCV 2026 |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV) |
| Cite as: | arXiv:2607.15374 [cs.CV] |
| (or arXiv:2607.15374v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2607.15374 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Kazi Sajeed Mehrab [view email]
[v1]
Thu, 16 Jul 2026 18:18:31 UTC (25,296 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CV
See more →ProMoE-FL: Prototype-conditioned Mixture of Experts for Multimodal Federated Learning with Missing Modalities
ProMoE-FL introduces a Prototype-conditioned Mixture-of-Experts framework for multimodal federated learning, effectively addressing missing modalities. It outperforms existing methods on four chest X-ray datasets, demonstrating superior feature synthesis capabilities in both homogeneous and heterogeneous settings.