Towards Compact Autonomous Driving Perception with Balanced Learning and Multi-sensor Fusion
Quick Take
A novel compact deep multi-task learning model enables simultaneous semantic segmentation, depth estimation, and LiDAR segmentation for autonomous driving, achieving superior performance with fewer parameters and faster inference. The model leverages adaptive loss weighting and multi-sensor fusion from RGB cameras and LiDAR, demonstrating consistent results across various datasets.
Key Points
- The model processes multiple perception tasks in one forward pass.
- Adaptive loss weighting addresses imbalanced learning across tasks.
- Achieves better performance with fewer parameters compared to recent models.
- Consistent results across 3 CARLA simulation datasets and 1 real-world dataset.
- Code and resources available at https://github.com/oskarnatan/compact-perception.
Article Content
From source RSS / original summaryarXiv:2606. 02979v1 Announce Type: new Abstract: We present a novel compact deep multi-task learning model to handle various autonomous driving perception tasks in one forward pass. The model performs multiple views of semantic segmentation, depth estimation, light detection and ranging (LiDAR) segmentation, and bird's eye view projection simultaneously without being supported by other models.
We also provide an adaptive loss weighting algorithm to tackle the imbalanced learning issue that occurred due to plenty of given tasks. Through data pre-processing and intermediate sensor fusion techniques, the model can process and combine multiple input modalities retrieved from RGB cameras, dynamic vision sensors (DVS), and LiDAR placed at several positions on the ego vehicle. Therefore, a better understanding of a dynamically changing environment can be achieved.
Based on the ablation study, the model variant trained with our proposed method achieves a better performance. Furthermore, a comparative study is also conducted to clarify its performance and effectiveness against the combination of some recent models. As a result, our model maintains better performance even with much fewer parameters. Hence, the model can inference faster with less GPU memory utilization.
Moreover, the result tends to be consistent in 3 different CARLA simulation datasets and 1 real-world nuScenes-lidarseg dataset. To support future research, we share codes and other files publicly at https://github. com/oskarnatan/compact-perception.
Reader Mode unavailable (could not extract clean content).
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CV
See more →Plan2Map: A Multimodal Benchmark for Document-Grounded Geospatial Boundary Reconstruction from Planning Records
Plan2Map introduces a 208-case benchmark for reconstructing geospatial boundaries from UK planning documents. The GeoPlanAgent system achieves a mean IoU of 0.736, significantly outperforming baseline models, highlighting the challenges in localization and map registration.
