Unification of Closed-Open Industrial Detection Scenarios: New Large-Scale Benchmarks,Challenges and Baselines
Quick Answer
The MMIOC-1M benchmark introduces over one million samples for industrial defect detection, addressing dataset scarcity and prompt reliance.
Quick Take
The MMIOC-1M benchmark introduces over one million samples for industrial defect detection, addressing dataset scarcity and prompt reliance. The RTVPNet model enhances performance with innovations in domain adaptation and visual prompt generation, achieving state-of-the-art results across multiple benchmarks.
Key Points
- MMIOC-1M includes 1M samples across 14 super-categories and 351 defect subcategories.
- RTVPNet features expert-assisted domain projection for rapid adaptation to industrial domains.
- The model employs an energy-based sparse sampling strategy for refined visual prompts.
- RTVPNet achieves state-of-the-art performance on MMIOC-1M, LVIS, and COCO benchmarks.
- Dataset and code are publicly available at https://github.com/hellozzk/MMIO.
Article Content
From source RSS / original summaryarXiv:2606.
07953v1 Announce Type: new Abstract: Large-scale Visual-Language Models (LVLMs) have achieved remarkable success in natural visual tasks, yet their application to industrial defect detection remains challenging due to two fundamental limitations: (i) the scarcity of large-scale industrial datasets that cover diverse defect categories across multiple domains, and (ii) the reliance on manual prompts (points, boxes, masks) that introduce subjective noise and lack text-visual interaction for fine-grained understanding.
To address these challenges, we introduce a Large-Scale Multi-Modal Industrial Open-Closed benchmark (MMIOC-1M) containing over one million samples across $14$ super-categories, $29$ industrial scenes, and $351$ defect subcategories. To our knowledge, MMIOC-1M is the first unified largest benchmark supporting both open-vocabulary and closed-set industrial detection, providing valuable pre-training data for LVLMs in industrial scenarios.
Furthermore, we propose a Refined Text-Visual Prompt Network (RTVPNet) that incorporates three key innovations: (1) an expert-assisted domain projection mechanism that enables rapid adaptation of general vision models to industrial domains, (2) an energy-based sparse sampling strategy that automatically generates refined visual prompts without manual intervention, and (3) a bidirectional text-visual interaction module that enhances cross-modal semantic alignment and understanding.
Extensive experiments demonstrate that RTVPNet achieves state-of-the-art performance on MMIOC-1M, LVIS, and COCO benchmarks while maintaining computational efficiency. The dataset and code are available at https://github. com/hellozzk/MMIO.
Reader Mode unavailable (could not extract clean content).
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.AI
See more →The Sim-to-Real Gap of Foundation Model Agents: A Unified MDP Perspective
This paper addresses the sim-to-real gap for foundation model agents by framing it within a Markov Decision Process (MDP) structure. It advocates for established solutions like domain randomization to enhance agent robustness, aiming to create standardized benchmarks for reliable real-world applications.