ScenePilot: Controllable Boundary-Driven Critical Scenario Generation for Autonomous Driving
Quick Answer
ScenePilot introduces a boundary-driven framework for generating safety-critical scenarios in autonomous driving, achieving a 6.2% increase in collision rates while maintaining physical validity.
Quick Take
ScenePilot introduces a boundary-driven framework for generating safety-critical scenarios in autonomous driving, achieving a 6.2% increase in collision rates while maintaining physical validity. This method utilizes constrained multi-objective reinforcement learning to explore feasible scenarios that challenge current autonomy systems, ultimately reducing downstream crash rates through adversarial fine-tuning.
Key Points
- ScenePilot targets physically solvable yet challenging scenarios for autonomous vehicles.
- Utilizes constrained multi-objective reinforcement learning for scenario generation.
- Achieved a 6.2 percentage point increase in collision rates on SafeBench.
- Maintains physical validity while exploring the feasibility boundary.
- Adversarial fine-tuning on boundary scenarios reduces downstream crash rates.
Paper Resources
Article Content
From source RSS / original summaryarXiv:2605. 21168v1 Announce Type: new Abstract: Safety-critical scenarios are central to evaluating autonomous driving systems, yet their rarity in naturalistic logs makes simulation-based stress testing indispensable.
Most scenario generation methods treat surrounding agents as adversaries, but they either (i) induce failures without explicitly modeling vehicle-road physical limits, yielding visually extreme yet physically unsolvable crashes, or (ii) enforce physical feasibility or policy feasibility in isolation, which can over-focus on aggressive maneuvers or remain tied to a controller-dependent capability boundary.
We propose ScenePilot, a feasibility-guided, boundary-driven framework that targets the boundary band: scenarios that are physically solvable in principle yet still cause the deployed autonomy stack to fail.
We formulate generation as constrained multi-objective reinforcement learning, combining an RSS-derived physical-feasibility score $\sigma$ with an online-learned AV-risk predictor $\Phi$, and introduce step-level feasibility-aware shielding to keep exploration near the feasibility boundary while avoiding infeasible artifacts. Experiments on SafeBench with multiple planners show that ScenePilot yields substantially higher collision rates (+6.
2 percentage points) while preserving physical validity, and that adversarial fine-tuning on these boundary-band scenarios consistently reduces downstream crash rates. The code is available at https://github. com/QiyuRuan/ScenePilot.
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.AI
See more →Procedural Memory Distillation: Online Reflection for Self-Improving Language Models
Procedural Memory Distillation (PMD) enhances reinforcement learning by converting cross-episode signals into reusable memory, improving Qwen3-8B and OLMo3-Instruct-7B models by 3.8-5.5% on SCIKNOWEVAL and 7.9-13.6% on . The co-evolution of policy and memory allows for more effective self-supervision, demonstrating significant performance gains when both components are active.