What Makes Synthetic Data Effective in Image Segmentation
Quick Take
Synthetic data from diffusion models enhances image segmentation through improved spatial representations.
Key Points
- SENSE framework improves segmentation performance significantly.
- Model-agnostic and compatible with various architectures.
- Validated on Cityscapes, COCO, and ADE20K datasets.
📖 Reader Mode
~2 min readAbstract:Driven by rapid advances in large-scale generative models, synthetic data has emerged as a promising solution for visual understanding. While modern diffusion models achieve remarkable photorealistic image synthesis, their potential in complex visual segmentation tasks remains underexplored. In this work, we conduct a systematic analysis of synthetic images from state-of-the-art diffusion models to uncover the factors governing their utility. In particular, synthetic images characterized by dense scene composition and fine instance fidelity demonstrate distinctive benefits, yielding significantly more discriminative spatial representations. Building on these insights, we propose SENSE, a unified framework that leverages flexible and scalable synthetic data to substantially enhance segmentation performance. Notably, SENSE is model-agnostic, compatible with diverse architectures (e.g., DPT and Mask2Former), and scales effectively across models with varying parameter capacities. Extensive experiments on Cityscapes, COCO, and ADE20K validate the effectiveness and generalization capability of our approach. Code is available at this https URL.
| Comments: | Accepted to ICML 2026 |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV) |
| Cite as: | arXiv:2605.19289 [cs.CV] |
| (or arXiv:2605.19289v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2605.19289 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Jinjin Zhang [view email]
[v1]
Tue, 19 May 2026 03:07:04 UTC (9,628 KB)
— Originally published at arxiv.org
More from arXiv cs.CV
See more →GeoSym127K: Scalable Symbolically-verifiable Synthesis for Multimodal Geometric Reasoning
GeoSym127K introduces a scalable neuro-symbolic framework for enhanced geometric reasoning in multimodal models.