Reasoning-Guided Part-Level Visual Grounding via Reinforcement Learning
Quick Answer
The proposed Object-Part Hierarchical Reflective Grounding (OP-HRG) method enhances part-level visual grounding in multimodal large language models by first localizing parent objects before their parts.
Quick Take
A 4B model trained with a part-aware framework outperforms 7B grounding models and SAM3 on benchmarks like PascalPart and PartImageNet, showcasing significant advancements in reasoning segmentation.
Key Points
- OP-HRG employs a coarse-to-fine strategy for part-level grounding.
- A self-check mechanism refines the localized part predictions.
- The 4B model outperforms larger 7B models on key benchmarks.
- Training utilizes stage-wise rewards via a part-aware GRPO framework.
- The approach significantly improves reasoning segmentation capabilities.
Paper Resources
Source Excerpt
Multimodal (MLLMs) ground whole objects well from free-form language queries, but they struggle when the query names a part rather than the object. We trace this to a missing object-part hierarchy, since parts are localized in the same single step used for objects. We propose Object-Part Hierarchical Reflective Grounding (OP-HRG), a coarse-to-fine reasoning-guided grounding strategy that first localizes the parent object and then the part within it. A self-check then reflec
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CV
See more →ProMoE-FL: Prototype-conditioned Mixture of Experts for Multimodal Federated Learning with Missing Modalities
ProMoE-FL introduces a Prototype-conditioned Mixture-of-Experts framework for multimodal federated learning, effectively addressing missing modalities. It outperforms existing methods on four chest X-ray datasets, demonstrating superior feature synthesis capabilities in both homogeneous and heterogeneous settings.