Fisher-Guided Progressive Parameter Selection for Adaptive Fine-Tuning
Quick Answer
FisherAdapTune introduces a dynamic parameter selection method for fine-tuning pretrained models, leveraging Fisher geometry to enhance performance in segmentation tasks.
Quick Take
The framework shows improved in-distribution performance and zero-shot transfer, demonstrating the effectiveness of tracking Fisher structural drift for efficient adaptation.
Key Points
- FisherAdapTune tracks Fisher geometry to select parameter groups dynamically.
- The method reduces generalization error by freezing stabilized parameter groups.
- Evaluation on segmentation tasks shows significant performance improvements.
- Results indicate better zero-shot transfer capabilities across multiple settings.
- Code for FisherAdapTune is publicly available for further research.
Paper Resources
Source Excerpt
arXiv:2606. 10196v1 Announce Type: new Abstract: Parameter-efficient fine-tuning (PEFT) aims to adapt pretrained models with a small trainable parameter subset, however, most existing methods choose this subset from fixed architectural heuristics rather than using dynamic, task-aware criteria. We introduce \textbf{FisherAdapTune}, a Fisher-guided Adaptive Fine-Tuning framework that progressively selects parameter groups by tracking the temporal drift of their Fisher geometry. …
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CV
See more →ProMoE-FL: Prototype-conditioned Mixture of Experts for Multimodal Federated Learning with Missing Modalities
ProMoE-FL introduces a Prototype-conditioned Mixture-of-Experts framework for multimodal federated learning, effectively addressing missing modalities. It outperforms existing methods on four chest X-ray datasets, demonstrating superior feature synthesis capabilities in both homogeneous and heterogeneous settings.