GradShield: Alignment Preserving Finetuning
Quick Answer
GradShield introduces a filtering method for Large Language Models (LLMs) that identifies and removes harmful data during finetuning, achieving an Attack Success Rate (ASR) below 6% while maintaining utility performance across various tasks.
Key Points
- GradShield computes a Finetuning Implicit Harmfulness Score (FIHS) for data points.
- An adaptive thresholding algorithm is employed to filter harmful data effectively.
- Results show GradShield outperforms baseline methods in safety and utility.
- Maintains an Attack Success Rate (ASR) below 6% across multiple tasks.
- Evaluated across various utility fine-tuning tasks with varying harmful data levels.
Paper Resources
Article Excerpt
From source RSS / original summaryarXiv:2605. 14194v1 Announce Type: new Abstract: (LLMs) pose a significant risk of safety misalignment after finetuning, as models can be compromised by both explicitly and implicitly harmful data. Even some seemingly benign data can inadvertently steer a model towards misaligned behaviors. To address this, we introduce GradShield, a principled filtering method that safeguards LLMs during finetuning by identifying and removing harmful data points before they corrupt the model's alignment.
It removes potentially harmful data by computing a Finetuning Implicit Harmfulness Score (FIHS) for each data point and employs an adaptive thresholding algorithm. We apply GradShield to multiple utility fine-tuning tasks across varying levels of harmful data and evaluate the safety and utility performance of the resulting LLMs using various metrics.
The results show that GradShield outperforms all baseline methods, consistently maintaining an Attack Success Rate (ASR) below $6\%$ while preserving utility performance.
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →TriAgent: Divergence-Aware Committees for Cost-Efficient Financial Sentiment Analysis
TriAgent introduces a cost-efficient multi-agent system for financial sentiment analysis, combining VADER, FinBERT, and Qwen2.5. It achieves an F1 score of ~0.87 with significant savings of $9.3M/year at a 10M-user scale compared to GPT-4o-mini, while also detecting hallucinations with an AUC of 0.90.