DataRx: Missingness-Aware Sampling for Safer Large Language Model Task-Specific Fine-Tuning
Quick Answer
DataRx introduces a missingness-aware sampling method that enhances safety in large language model fine-tuning.
Quick Take
By selecting safety-critical examples based on high-dimensional hidden representations, it reduces the attack success rate of Llama3-8B-Instruct from 59.23% to 13.70% with just 1% additional safety samples. This method aims to inspire further research into data-centric defenses.
Key Points
- DataRx leverages high-dimensional representations to identify safety signal gaps.
- Only 1% additional safety samples significantly improve model safety.
- Attack success rate on Llama3-8B-Instruct dropped from 59.23% to 13.70%.
- Combines with existing safety data synthesis methods for enhanced defenses.
- Aims to inspire further research in data-centric safety strategies.
DeepSignal Analysis
What happened
DataRx introduces a method for selecting safety-critical examples during the fine-tuning of large language models (LLMs). This approach significantly reduces the attack success rate of the Llama3-8B-Instruct model from 59.23% to 13.70% with only 1% additional safety samples.
Key evidence
- DataRx is based on the hypothesis that effective safety samples fill gaps in LLMs' safety capabilities, leveraging high-dimensional hidden representations.
- The method was tested on seven downstream tasks, demonstrating a substantial reduction in attack success rates when using missingness-aware sampling.
- DataRx can be combined with existing safety data synthesis methods to further enhance safety defenses during fine-tuning.
Why it matters
The findings highlight a critical issue in task-specific fine-tuning, where safety guardrails can be compromised. By improving the selection of safety data, DataRx aims to enhance the robustness of LLMs against attacks, which is essential for their safe deployment in real-world applications. This research could pave the way for more effective data-centric defense strategies in AI.
What to watch
Paper Resources
📖 Reader Mode
~2 min readAbstract:Task-specific fine-tuning can improve the performance of large language models (LLMs) on downstream tasks. However, our study reveals that task-specific fine-tuning can also weaken the safety guardrails of aligned LLMs. A widely adopted strategy for preserving safety during fine-tuning is to incorporate safety data. Although previous studies have shown that randomly mixing safety data can alleviate safety degradation, the underlying principle determining why some safety examples are more effective than others still remains unclear. In this paper, we propose DataRx, a missingness-aware sampling method for selecting safety-critical examples. DataRx is based on the hypothesis that a safety sample is more effective when the selected examples provide safety signals that fill the missing parts of LLMs' safety capabilities. DataRx's key insight is leveraging high-dimensional hidden representations rather than discrete tokens to quantify the safety signal gap between the target model's native response and the safety reference response. The results show that, with only 1% additional safety samples from BeaverTails, DataRx reduces the average attack success rate of Llama3-8B-Instruct across seven downstream tasks from 59.23% under random sampling to 13.70%. In addition, DataRx can be combined with the existing safety data synthesis method to further enhance safety defenses during fine-tuning. We hope that DataRx will inspire more data-centric defense research.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2608.04322 [cs.CL] |
| (or arXiv:2608.04322v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2608.04322 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Junbo Zhang [view email]
[v1]
Wed, 5 Aug 2026 01:05:13 UTC (1,174 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →TriAgent: Divergence-Aware Committees for Cost-Efficient Financial Sentiment Analysis
TriAgent introduces a cost-efficient multi-agent system for financial sentiment analysis, combining VADER, FinBERT, and Qwen2.5. It achieves an F1 score of ~0.87 with significant savings of $9.3M/year at a 10M-user scale compared to GPT-4o-mini, while also detecting hallucinations with an AUC of 0.90.