RadarTwin: Scene-Specific mmWave Radar Simulation and Learning for Mobile Indoor Perception
Quick Answer
RadarTwin is a novel framework that generates scene-specific mmWave radar training data using 3D reconstructions and vision-language models, improving object recognition accuracy to 95.3% with minimal real data.
Quick Take
RadarTwin is a novel framework that generates scene-specific mmWave radar training data using 3D reconstructions and , improving object recognition accuracy to 95.3% with minimal real data. This approach addresses the data scarcity issue in radar perception, enabling effective training before real data collection.
Key Points
- RadarTwin synthesizes radar data from 3D reconstructions and physics-based ray tracing.
- Achieves 2.5 times better recognition of real objects using only simulated training data.
- Real-simulated dataset includes various household objects and mobile sensing trajectories.
- Modeling multipath environments is crucial for matching real radar measurements.
- Enables radar perception training in new spaces prior to any real data collection.
Paper Resources
📖 Reader Mode
~2 min readAbstract:Millimeter-wave (mmWave) radar perception is limited by data scarcity: models trained on existing radar datasets fail to generalize to new objects, environments, and sensing trajectories. We present RadarTwin, a framework for generating deployment-specific radar training data before real data collection. Given a 3D reconstruction of a target space (phone LiDAR, robot-mounted sensing, or RGB-to-3D), RadarTwin uses a vision-language model to infer radar-relevant surface materials and a physics-based ray tracer to synthesize raw frequency-modulated continuous-wave (FMCW) radar measurements with multi-bounce propagation. To study what transfers from simulation to reality, we collect a paired real-simulated dataset spanning household objects, material classes, distances, rotations, translations, and mobile sensing trajectories. We show that simulated and real radar share the same object-discriminative shape and material features, and that modeling the environment's multipath is essential to matching real measurements. A representation trained on simulation alone recognizes real objects at 2.5 times chance with no real radar labels, and a few labeled examples raise this to 95.3% on a 12-way recognition task. RadarTwin enables training radar perception for a new space before any real radar data is collected there.
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG) |
| Cite as: | arXiv:2606.28396 [cs.CV] |
| (or arXiv:2606.28396v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2606.28396 arXiv-issued DOI via DataCite |
Submission history
From: Emily Bejerano [view email]
[v1]
Wed, 24 Jun 2026 05:50:26 UTC (4,960 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CV
See more →LLM-Guided ANN Index Optimization for Human-Object Interaction Retrieval
A phase-aware LLM agent optimizes human-object interaction retrieval, outperforming Optuna TPE by 33.3% and VDTuner by 34.2% on the HICO-DET benchmark. This method enhances throughput by 15.3x over UniIR and demonstrates strong transferability across vector database management systems.