GenSyn10: A Multi-Generative AI Dataset For Benchmarking Image Classification
Quick Answer
GenSyn10 introduces a synthetic image dataset of 60,000 images generated by FLUX.2-dev, HunyuanImage-3.0, and Qwen-Image-2512, aimed at improving AI-generated image detection.
Quick Take
Despite achieving 96.86% zero-shot accuracy, models struggle with unseen generators, highlighting challenges in out-of-distribution generalization.
Key Points
- GenSyn10 consists of 60,000 CIFAR-10-aligned synthetic images across 10 classes.
- Images generated using diverse models: FLUX.2-dev, HunyuanImage-3.0, and Qwen-Image-2512.
- CIFAR-10-trained models achieve up to 96.86% zero-shot accuracy on GenSyn10.
- Fine-tuning boosts accuracy to 99.88%, but performance drops on unseen generators.
- Establishes a benchmark for studying robustness and cross-generator generalization.
DeepSignal Analysis
What happened
GenSyn10 is a synthetic image dataset comprising 60,000 images generated by three different AI models. It aims to enhance the detection of AI-generated images, particularly in scenarios involving unseen generators. Despite achieving high accuracy in zero-shot settings, models show significant performance drops when faced with unfamiliar generators.
Key evidence
- GenSyn10 consists of 60,000 images generated by FLUX.2-dev, HunyuanImage-3.0, and Qwen-Image-2512, organized into 10 classes.
- CIFAR-10-trained models reached a zero-shot accuracy of 96.86% on GenSyn10, which improved to 99.88% after fine-tuning.
- Fine-tuned models achieved 97-99.9% accuracy on seen generators but dropped to 79-96% on images from an unseen generator.
Why it matters
The introduction of GenSyn10 highlights the ongoing challenges in detecting AI-generated images, especially when models encounter new generators. The dataset provides a structured approach to evaluate and improve out-of-distribution generalization, which is crucial for the reliability of AI systems in real-world applications. Understanding these limitations can guide future research in robustness and domain adaptation.
Paper Resources
Source Excerpt
The rapid advancement of generative AI has outpaced our ability to reliably detect its outputs, particularly when detectors encounter generators they have not seen before. We introduce GenSyn10, a CIFAR-10-aligned synthetic image dataset of 60,000 images (10 classes, 32$\times$32, 50k/10k split) generated using three architecturally diverse state-of-the-art models: FLUX. 2-dev (Rectified Flow Transformer), HunyuanImage-3. 0 (MoE Transformer), and Qwen-Image-2512 (Multimodal Diffusion Transformer),
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CV
See more →ProMoE-FL: Prototype-conditioned Mixture of Experts for Multimodal Federated Learning with Missing Modalities
ProMoE-FL introduces a Prototype-conditioned Mixture-of-Experts framework for multimodal federated learning, effectively addressing missing modalities. It outperforms existing methods on four chest X-ray datasets, demonstrating superior feature synthesis capabilities in both homogeneous and heterogeneous settings.