DisasterTD: Disaster Toponym Disambiguation Using Multimodal LLMs and Cross-View Geolocalization
Quick Answer
This paper shows that DisasterTD integrates multimodal large language models with cross-view geolocalization to enhance disaster toponym disambiguation.
Quick Take
Evaluated on Hurricane Harvey data, it achieves geolocalization accuracies of 71.62% within 1000m, significantly improving performance on ambiguous toponyms by reducing candidate dispersion and errors.
Key Points
- DisasterTD uses MLLMs for semantic reasoning and candidate geolocation generation.
- Cross-view matching with SMI, RSI, and SVI refines geolocation accuracy.
- Achieves 71.62% accuracy within 1000m and 47.01% within 50m.
- Largest improvements noted in ambiguous toponyms, reducing errors significantly.
- Dataset categorized by toponym clarity allows for detailed performance analysis.
DeepSignal Analysis
What happened
DisasterTD is a framework designed to improve disaster toponym disambiguation by combining multimodal large language models with cross-view geolocalization. It was tested using data from Hurricane Harvey, achieving notable geolocalization accuracy rates.
Key evidence
- DisasterTD achieved geolocalization accuracies of 71.62% within 1000 meters when evaluated on the Hurricane Harvey dataset.
- The framework reduces candidate dispersion and errors, particularly for ambiguous toponyms, by integrating MLLM-based candidate generation with cross-view verification.
- The evaluation dataset was categorized into four groups based on toponym clarity and ambiguity, allowing for detailed performance analysis.
Why it matters
The integration of multimodal large language models with cross-view geolocalization represents a significant advancement in accurately identifying geographic references in social media imagery. This is crucial for emergency response and situational awareness during disasters, where timely and precise information can save lives and resources.
What to watch
Paper Resources
Source Excerpt
Social media imagery (SMI) provides timely and fine-grained ground perspectives that are valuable for situational awareness and emergency response. Unlike satellite or aerial imagery, SMI can capture disaster impacts and ground-level conditions in a timely manner. However, geographic references in SMI are often vague or ambiguous, making accurate geolocalization challenging. To address this issue, we propose DisasterTD, a disaster toponym disambiguation framework that integrates multimodal large
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CV
See more →ProMoE-FL: Prototype-conditioned Mixture of Experts for Multimodal Federated Learning with Missing Modalities
ProMoE-FL introduces a Prototype-conditioned Mixture-of-Experts framework for multimodal federated learning, effectively addressing missing modalities. It outperforms existing methods on four chest X-ray datasets, demonstrating superior feature synthesis capabilities in both homogeneous and heterogeneous settings.