Planktonzilla: Multimodal dataset and models for understanding plankton ecosystems
Quick Take
Planktonzilla-17M is a groundbreaking dataset with 17.4 million images, enhancing plankton classification across diverse environments. It reveals that supervised classifiers outperform CLIP-style models when using taxonomic lineage as text, highlighting limitations in current biological foundation models for marine imaging.
Key Points
- Planktonzilla-17M consolidates 17.4 million plankton images from 13 imaging systems.
- Dataset includes 3.74 million plankton images across 602 taxonomic classes.
- Supervised classifiers outperform CLIP-style training when using taxonomic lineage.
- BioCLIP and BioCLIP2 show poor performance in zero-shot and few-shot settings.
- Dataset enhances understanding of plankton ecosystems and ocean health.
Article Content
From source RSS / original summaryarXiv:2606. 00080v1 Announce Type: new Abstract: Marine plankton underpin aquatic food webs and play a key role in global CO2 sequestration, making reliable species identification critical for understanding ocean health and climate feedbacks. Existing classification models perform well on individual collections but fail to generalize across instruments and environments due to isolated training datasets and inconsistent labels.
To address this, we introduce Planktonzilla-17M, a unified dataset consolidating publicly available plankton image collections spanning thirteen imaging systems. It comprises 17. 4 million images with standardized taxonomy and geo-environmental metadata, including 3. 74 million plankton images spanning over 602 taxonomic classes, of which 201 are identified at the species level, making it the largest and most comprehensive plankton image dataset to date.
Using this large-scale dataset, we perform a controlled comparison between supervised and CLIP-style image--text training on a shared ViT backbone. We find that a supervised classifier matches or exceeds CLIP-style training when trained using taxonomic lineage as text. We further observe that BioCLIP and BioCLIP2 perform poorly on plankton in zero-shot and few-shot settings.
Leveraging Planktonzilla-17M improves plankton classification performance, highlighting the limitations of current biological foundation models in marine imaging domains.
Reader Mode unavailable (could not extract clean content).
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CV
See more →Evi-Steer: Learning to Steer Biomedical Vision-Language Models through Efficient and Generalizable Evidential Tuning
Evi-Steer introduces a novel evidential tuning framework for BiomedCLIP, enabling efficient fine-tuning with only 0.11% parameter updates. It significantly enhances performance in few-shot learning and domain shifts across 15 biomedical imaging datasets, demonstrating robustness for clinical applications.