HeteroPROPMT: A Real-time and Privacy-Preserving Heterogeneous Collaborative Perception Framework
Quick Answer
HeteroPROMPT is a real-time, privacy-preserving framework for heterogeneous collaborative perception, enhancing autonomous systems' awareness by aligning diverse sensor data with a unified feature space.
Quick Take
It outperforms existing methods in Average Precision on OPV2V-H and V2XSet datasets while using significantly fewer trainable parameters, achieving over 99.99% accuracy in modality classification without exposing proprietary information.
Key Points
- HeteroPROMPT rapidly aligns heterogeneous agent features using modular prompts.
- It maintains frozen encoders while improving collaborative fusion and detection.
- Achieves better Average Precision than state-of-the-art methods with fewer parameters.
- The modality classifier predicts agent modalities with over 99.99% accuracy.
- Designed for metadata-free deployment, enhancing privacy in collaborative perception.
Paper Resources
Source Excerpt
Collaborative Perception (CP) improves autonomous systems' awareness of their surroundings by sharing sensor data, intermediate features, and detection results. In real-world deployments, however, collaborating vehicles often use heterogeneous sensors, perception models, datasets, and training domains, creating feature-space shifts that degrade downstream fusion and detection. Existing approaches typically retrain fusion and detection components or introduce modality-specific feature interpreter
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CV
See more →ProMoE-FL: Prototype-conditioned Mixture of Experts for Multimodal Federated Learning with Missing Modalities
ProMoE-FL introduces a Prototype-conditioned Mixture-of-Experts framework for multimodal federated learning, effectively addressing missing modalities. It outperforms existing methods on four chest X-ray datasets, demonstrating superior feature synthesis capabilities in both homogeneous and heterogeneous settings.