RT-VLA: Real-Time Vision-Language-Action Models via Knowledge Distillation
Quick Answer
RT-VLA is a lightweight, distilled Vision-Language-Action model that achieves a 44.8X reduction in inference time compared to SimLingo, while maintaining competitive performance in autonomous driving and language reasoning.
Quick Take
RT-VLA is a lightweight, distilled Vision-Language-Action model that achieves a 44.8X reduction in inference time compared to SimLingo, while maintaining competitive performance in autonomous driving and language reasoning. This model enables real-time, explainable decision-making without latency, making it suitable for deployment in autonomous vehicles.
Key Points
- RT-VLA reduces inference time by 44.8X in vision-only mode.
- Maintains competitive performance in closed-loop driving and language reasoning.
- Supports post-hoc explanation through offline language analysis.
- Utilizes multi-level supervised distillation from the SimLingo model.
- Enables real-time control without added latency.
Paper Resources
Article Content
From source RSS / original summaryarXiv:2606. 14010v1 Announce Type: new Abstract: Vision-Language-Action (VLA) models have shown strong potential for end-to-end autonomous driving by jointly modeling visual perception, language reasoning, explainability and action prediction. However, their large vision-language backbones and reasoning modules introduce substantial inference latency and thereby prevent their deployment in the unforgiving reality of the road networks.
We propose RT-VLA, a lightweight, distilled VLA model that transfers the driving and reasoning capabilities of the state-of-the-art SimLingo model into a compact student through multi-level supervised distillation. RT-VLA preserves language-based reasoning and supports post-hoc explanation through offline language analysis of safety-critical driving moments without adding latency to real-time control.
Compared to the SimLingo teacher, RT-VLA maintains competitive closed-loop driving and language reasoning performance while reducing inference time by 44. 8X in vision-only mode and 7. 9X in vision+language mode. These results suggest that supervised distillation is a practical approach for building real-time, explainable VLA-style autonomous driving models.
Reader Mode unavailable (could not extract clean content).
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CV
See more →LLM-Guided ANN Index Optimization for Human-Object Interaction Retrieval
A phase-aware LLM agent optimizes human-object interaction retrieval, outperforming Optuna TPE by 33.3% and VDTuner by 34.2% on the HICO-DET benchmark. This method enhances throughput by 15.3x over UniIR and demonstrates strong transferability across vector database management systems.