PiCo: Physics-Informed Concept Probing for Geo-Foundation Models
Quick Answer
The PiCo framework enhances Geo-Foundation Models (GFMs) by identifying concept vectors in their embedding space, revealing physical sensitivities that inform model predictions.
Quick Take
Evaluated across three GFMs, PiCo retains 70.0-93.8% accuracy in hierarchical classification tasks while supporting hypothesis testing in ground-cover estimation, demonstrating its potential for broader applications in Earth observation.
Key Points
- PiCo identifies concept vectors using synthetic multispectral satellite observations.
- It reveals physical sensitivities that inform domain-based hypotheses.
- In ground-cover estimation, RGB restriction reduces dry-matter sensitivity.
- Concept representations retain 70.0-93.8% accuracy in hierarchical classification.
- PiCo's methodology is applicable beyond Earth observation with controlled simulations.
DeepSignal Analysis
What happened
The PiCo framework enhances Geo-Foundation Models (GFMs) by identifying concept vectors in their embedding space, which reveal physical sensitivities that inform model predictions. Evaluated on three GFMs, PiCo maintains 70.0-93.8% accuracy in hierarchical classification tasks and supports hypothesis testing in ground-cover estimation.
Key evidence
- PiCo identifies concept vectors in a GFM's embedding space, revealing physical sensitivities that inform domain-based hypotheses about downstream performance.
- In ground-cover estimation, RGB restriction reduces dry-matter sensitivity and disproportionately degrades predictions for dry vegetation and bare ground.
- Concept representations in hierarchical forest type and tree species classification retain 70.0-93.8% of full-embedding accuracy, indicating the models capture similar physical contrasts between classes.
Why it matters
The ability to probe GFMs for physical sensitivities enhances model interpretability and usability, allowing developers to test hypotheses about model behavior. This could lead to improved model selection for various tasks, particularly in Earth observation, where understanding physical features is crucial for accurate predictions.
Paper Resources
📖 Reader Mode
~2 min readAbstract:Geo-foundation models (GFMs) are typically evaluated by downstream task predictive performance, revealing little about which physical features they are sensitive to or use in prediction. This limits developers' ability to test hypotheses about model behaviour and users' ability to select models for new tasks. We propose Physics-Informed Concept Probing (PiCo), a post-hoc framework for explaining GFM predictions. PiCo identifies concept vectors in a GFM's embedding space by generating synthetic multispectral satellite observations from a 3D radiative transfer model coupled with a game engine. These vectors reveal physical sensitivities that inform domain-based hypotheses about downstream performance. Projecting embeddings onto these concept vectors produces low-dimensional representations for quantifying concept importance in making predictions. We evaluate PiCo on three GFMs with different input modalities using seven physical concepts covering forest leaf optics and canopy structure. In ground-cover estimation, RGB restriction reduces dry-matter sensitivity and disproportionately degrades predictions for dry vegetation and bare ground, supporting a hypothesis derived from sensitivity analysis. For hierarchical forest type and tree species classification, concept representations retain 70.0-93.8% of full-embedding accuracy. Models capture similar physical contrasts between classes, such as structural differences between broadleaf forest and cleared ground and leaf-water-content differences between oak and beech, yet differ in the strength of these contrasts and the concepts driving classification. PiCo thus provides a physical basis for hypothesis testing and model assessment, with potential applicability beyond Earth observation wherever forward models enable controlled simulation and pretrained embeddings are accessible for probing.
| Subjects: | Computer Vision and Pattern Recognition (cs.CV) |
| Cite as: | arXiv:2610.20973 [cs.CV] |
| (or arXiv:2610.20973v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2610.20973 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yihang She [view email]
[v1]
Wed, 14 Oct 2026 21:46:40 UTC (15,548 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CV
See more →ProMoE-FL: Prototype-conditioned Mixture of Experts for Multimodal Federated Learning with Missing Modalities
ProMoE-FL introduces a Prototype-conditioned Mixture-of-Experts framework for multimodal federated learning, effectively addressing missing modalities. It outperforms existing methods on four chest X-ray datasets, demonstrating superior feature synthesis capabilities in both homogeneous and heterogeneous settings.