SkySeg: Collaborative Onboard Semantic Segmentation with Heterogeneous UAVs in the Wild
Quick Take
SkySeg introduces a multi-UAV framework for onboard semantic segmentation, enhancing accuracy by 5.91% and reducing inference latency by 3.6x. This system addresses hardware constraints and environmental variations, making it suitable for real-time UAV missions.
Key Points
- SkySeg integrates low-cost sensors for efficient onboard semantic segmentation.
- Utilizes a cross-device test-time adaptation strategy to handle data distribution shifts.
- Achieves a 10.91% average accuracy gain in dynamic environments.
- Combines low-definition and high-definition images for improved performance.
- Accelerates inference latency significantly, enhancing real-time analysis capabilities.
Article Content
From source RSS / original summaryarXiv:2605. 24014v1 Announce Type: new Abstract: The demand for unmanned aerial vehicle (UAV)-based image acquisition and analysis has surged, with UAVs increasingly utilized for semantic segmentation tasks. To meet the real-time analysis requirements of UAV remote sensing missions, performing onboard computation and making decisions based on the results is a natural approach.
However, deploying semantic segmentation on resource-constrained UAV platforms presents two significant challenges: 1) hardware constraints limit the ability of UAVs to perform real-time semantic segmentation, and 2) environmental variations during flight cause data distribution shifts, deviating from the original training data.
To address these issues, this paper introduces SkySeg, a heterogeneous multi-UAV air-air cooperation framework that integrates computer vision and flight pattern to enable onboard semantic segmentation using low-cost sensors. SkySeg employs an efficient information fusion inference method, combining low-definition, wide-area images with high-definition, focused-area images.
Additionally, it incorporates a cross-device test-time adaptation (TTA) strategy to enhance segmentation performance in dynamic environments by collaboratively addressing distribution shifts of test data streams across UAVs. Experimental results demonstrate that our SkySeg framework accelerates inference latency by approximately 3. 6x, improves onboard segmentation accuracy by 5. 91\%, and achieves a 10. 91\% average accuracy gain in the wild.
Reader Mode unavailable (could not extract clean content).
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CV
See more →Evi-Steer: Learning to Steer Biomedical Vision-Language Models through Efficient and Generalizable Evidential Tuning
Evi-Steer introduces a novel evidential tuning framework for BiomedCLIP, achieving 0.11% parameter updates while enhancing uncertainty-aware fine-tuning. It outperforms state-of-the-art methods across 15 biomedical imaging datasets, proving effective in few-shot learning and domain shifts for clinical applications.
