Structuring Open-Ended NAS: Semi-Automated Design Knowledge Structuring with LLMs for Efficient Neural Architecture Search
Quick Take
This paper presents FairNAD, a semi-automated approach for efficient neural architecture search using structured design knowledge.
Key Points
- Addresses limitations of predefined NAS search spaces.
- Utilizes LLMs to create diverse architectural search spaces.
- Achieves significant performance improvements on benchmark datasets.
📖 Reader Mode
~2 min readAbstract:Current neural architecture search (NAS) methods are often limited by their predefined, restrictive search spaces. While recent large language model (LLM)-assisted NAS methods enable open-ended search spaces, they often suffer from inefficient exploration due to biased or low-quality design ideas. To address these issues, we propose to semi-automatically structure model design knowledge to guide the search process. Our approach first defines a high-level structural template of architectural attributes. An LLM then populates this template by analyzing papers, creating a rich and diverse search space that embodies this structured design knowledge. To efficiently explore this vast space, we introduce FairNAD, using a multi-type mutation that enables broad exploration through mutation with fair idea sampling, Pareto-aware mutation, LLM-driven iterative mutation, and a fine-grained feedback loop. We demonstrate the effectiveness of FairNAD in discovering high-performing architectures that yield 0.84, 2.17, and 2.35 points improvement on CIFAR-10, CIFAR-100, and ImageNet16-120, respectively, compared to current state-of-the-art methods.
| Comments: | 42 pages |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV) |
| Cite as: | arXiv:2605.19247 [cs.CV] |
| (or arXiv:2605.19247v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2605.19247 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yuiko Sakuma [view email]
[v1]
Tue, 19 May 2026 01:41:48 UTC (927 KB)
— Originally published at arxiv.org
More from arXiv cs.CV
See more →GeoSym127K: Scalable Symbolically-verifiable Synthesis for Multimodal Geometric Reasoning
GeoSym127K introduces a scalable neuro-symbolic framework for enhanced geometric reasoning in multimodal models.