GenSyn10: A Multi-Generative AI Dataset For Benchmarking Image Classification
Quick Answer
GenSyn10 introduces a synthetic image dataset of 60,000 images generated by FLUX.2-dev, HunyuanImage-3.0, and Qwen-Image-2512, aimed at improving AI-generated image detection.
Quick Take
Despite achieving 96.86% zero-shot accuracy, models struggle with unseen generators, highlighting challenges in out-of-distribution generalization.
Key Points
- GenSyn10 consists of 60,000 CIFAR-10-aligned synthetic images across 10 classes.
- Images generated using diverse models: FLUX.2-dev, HunyuanImage-3.0, and Qwen-Image-2512.
- CIFAR-10-trained models achieve up to 96.86% zero-shot accuracy on GenSyn10.
- Fine-tuning boosts accuracy to 99.88%, but performance drops on unseen generators.
- Establishes a benchmark for studying robustness and cross-generator generalization.
DeepSignal Analysis
What happened
GenSyn10 is a synthetic image dataset comprising 60,000 images generated by three different AI models. It aims to enhance the detection of AI-generated images, particularly in scenarios involving unseen generators. Despite achieving high accuracy in zero-shot settings, models show significant performance drops when faced with unfamiliar generators.
Key evidence
- GenSyn10 consists of 60,000 images generated by FLUX.2-dev, HunyuanImage-3.0, and Qwen-Image-2512, organized into 10 classes.
- CIFAR-10-trained models reached a zero-shot accuracy of 96.86% on GenSyn10, which improved to 99.88% after fine-tuning.
- Fine-tuned models achieved 97-99.9% accuracy on seen generators but dropped to 79-96% on images from an unseen generator.
Why it matters
The introduction of GenSyn10 highlights the ongoing challenges in detecting AI-generated images, especially when models encounter new generators. The dataset provides a structured approach to evaluate and improve out-of-distribution generalization, which is crucial for the reliability of AI systems in real-world applications. Understanding these limitations can guide future research in robustness and domain adaptation.
Paper Resources
📖 Reader Mode
~2 min readAbstract:The rapid advancement of generative AI has outpaced our ability to reliably detect its outputs, particularly when detectors encounter generators they have not seen before. We introduce GenSyn10, a CIFAR-10-aligned synthetic image dataset of 60,000 images (10 classes, 32$\times$32, 50k/10k split) generated using three architecturally diverse state-of-the-art models: FLUX.2-dev (Rectified Flow Transformer), HunyuanImage-3.0 (MoE Transformer), and Qwen-Image-2512 (Multimodal Diffusion Transformer), to advance research in AI-generated image detection. A central challenge in this domain is that detectors perform well on known generators but degrade on unseen ones. GenSyn10 addresses this limitation by curating data from multiple contemporary architectures under a standardized generation protocol, enabling controlled and systematic evaluation of out-of-distribution (OOD) generalization to novel generators. Images are generated using a template-based prompt engine and downsampled to ensure consistency. We evaluate 17 image classification models under a four-stage protocol: real-data baseline, zero-shot transfer, fine-tuning, and retention. Despite a measurable domain gap, CIFAR-10-trained models achieve up to 96.86\% zero-shot accuracy on GenSyn10, increasing to 99.88\% after fine-tuning. In binary real-vs-synthetic classification, fine-tuned models achieve 97-99.9\% accuracy on seen generators but drop to 79-96\% on images from an unseen generator, highlighting persistent limitations in OOD generalization. These results establish GenSyn10 as a controlled benchmark for studying synthetic image detection beyond single-generator settings, supporting research on robustness, domain adaptation, and cross-generator generalization.
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2607.16283 [cs.CV] |
| (or arXiv:2607.16283v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2607.16283 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Md Faraz Kabir Khan [view email]
[v1]
Fri, 10 Jul 2026 09:47:16 UTC (3,184 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CV
See more →ProMoE-FL: Prototype-conditioned Mixture of Experts for Multimodal Federated Learning with Missing Modalities
ProMoE-FL introduces a Prototype-conditioned Mixture-of-Experts framework for multimodal federated learning, effectively addressing missing modalities. It outperforms existing methods on four chest X-ray datasets, demonstrating superior feature synthesis capabilities in both homogeneous and heterogeneous settings.