Chat2Scenic: An Iterative RAG-Based Framework for Scenario Generation in Autonomous Driving
Quick Answer
Chat2Scenic is an innovative iterative retrieval-augmented framework for generating autonomous driving scenario scripts, achieving a 76.42% Compilation Success Rate (CSR) and 58.17% Framework Accuracy (FA).
Quick Take
Chat2Scenic is an innovative iterative retrieval-augmented framework for generating autonomous driving scenario scripts, achieving a 76.42% Compilation Success Rate (CSR) and 58.17% Framework Accuracy (FA). It outperforms existing methods significantly, with a benchmark of 123 scenarios from various regulations. The open-source code is available for further research.
Key Points
- Chat2Scenic integrates a chatbot interface for interactive scenario refinement.
- Achieves a 76.42% Compilation Success Rate, outperforming existing methods.
- Proposes an open benchmark with 123 scenarios from NHTSA and UN regulations.
- Utilizes (RAG) for regulatory knowledge grounding.
- Code is released as open source to facilitate future research.
Paper Resources
📖 Reader Mode
~2 min readAbstract:Validating autonomous driving systems requires diverse, regulation-compliant test scenarios. In simulation-based testing, scenarios are defined as executable scripts. Yet automatically generating such scripts from regulatory descriptions remains an open challenge, and existing approaches face fundamental trade-offs. Retrieval-assemble methods achieve reasonable compilation rates but lack scalability, whereas retrieval-based full-script generation suffers from low compilation success rates. We present Chat2Scenic, the first iterative retrieval-augmented framework to generate scenario scripts in Domain Specific Language (DSL). Specifically, Chat2Scenic provides a chatbot interface that supports interactive scenario refinement and integrates Retrieval-augmented Generation (RAG) to ground scenario generation in regulatory knowledge and DSL syntax. Furthermore, we propose an open benchmark for scenario generation comprising 123 scenarios from various regulations, including NHTSA and United Nations Vehicle Regulations, as well as other sources. Extensive evaluation with State-of-the-Art (SOTA) Large Language Models (LLMs) demonstrates that Chat2Scenic achieves 76.42% Compilation Success Rate (CSR) and 58.17% Framework Accuracy (FA), outperforming existing methods (Retrieval Assemble with 30.08% CSR, 11.03% FA and Retrieval full script generation with 16.26% CSR, 10.86% FA). To facilitate future research, we release our code as open source at this https URL.
| Comments: | Accepted at 2026 IEEE International Conference on Intelligent Robots and Systems (IROS) |
| Subjects: | Artificial Intelligence (cs.AI); Robotics (cs.RO) |
| Cite as: | arXiv:2607.14387 [cs.AI] |
| (or arXiv:2607.14387v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2607.14387 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yuan Gao [view email]
[v1]
Wed, 15 Jul 2026 22:02:03 UTC (2,257 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.AI
See more →Automatic Ordinary Differential Equations Discovery For Biological Systems Using Large Language Model Powered Agentic System
The MEDA system utilizes large language models and symbolic regression to autonomously discover ordinary differential equations for biological systems, achieving strong structural recovery and biologically plausible models. It outperforms existing methods by integrating domain knowledge and mechanistic constraints, demonstrating effective retrieval and extrapolation capabilities.