Mind the Tail: Data Selection Through Sparse Concepts
Quick Answer
The study explores data selection strategies using sparse autoencoders to enhance model performance on tail concepts in visual datasets.
Quick Take
By focusing on under-supported concepts, the method shows consistent improvements over existing data selection baselines, enhancing robustness against out-of-distribution scenarios and spurious contexts.
Key Points
- Sparse autoencoders extract concept activations to guide data selection.
- Focus on tail concepts improves model robustness and performance.
- Empirical results show consistent improvements across all benchmarks.
- Method addresses issues with overemphasizing boundary samples.
- Concept support principle enhances data-efficient learning.
Paper Resources
📖 Reader Mode
~2 min readAbstract:Data selection reallocates training support across visual concepts, including objects, attributes, color, texture, and backgrounds. Different allocation strategies may produce varied downstream model behavior, especially for tail concepts supported by only a few samples. Discarding them can hurt test-time robustness, while overemphasizing samples lying on category boundaries may introduce unstable training signals. This work explicitly investigates this relationship across fine-grained, spurious-correlation, and general image classification, as well as multimodal pretraining. Specifically, we use sparse autoencoders (SAEs) to extract concept activations from individual images. During selection, we accumulate the dataset-level concept support for each added sample. Applying diminishing returns to this distribution down-weights well-represented concepts and shifts the focus to the distribution tail. We then incorporate local support to find samples that reliably strengthen under-supported concepts. Empirical results show consistent improvements over strong data selection baselines across all benchmarks. We also reveal the connection between concepts our method enhances and the induced behavioral changes, including improved robustness against out-of-distribution and spurious contexts. This work establishes concept support, beyond individual sample importance, as a useful principle for data-efficient and robust learning.
| Comments: | Project page: this https URL |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV) |
| Cite as: | arXiv:2610.21007 [cs.CV] |
| (or arXiv:2610.21007v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2610.21007 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Ziheng Zhang [view email]
[v1]
Wed, 14 Oct 2026 23:00:43 UTC (2,387 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CV
See more →ProMoE-FL: Prototype-conditioned Mixture of Experts for Multimodal Federated Learning with Missing Modalities
ProMoE-FL introduces a Prototype-conditioned Mixture-of-Experts framework for multimodal federated learning, effectively addressing missing modalities. It outperforms existing methods on four chest X-ray datasets, demonstrating superior feature synthesis capabilities in both homogeneous and heterogeneous settings.