Does Personalization Break Trust? Open-Set Rejection Calibration Under Few-Shot On-Device Intent Adaptation
Quick Answer
This paper shows that Naive few-shot LoRA personalization in virtual assistants degrades out-of-scope rejection performance, with AUROC dropping by 0.055 to 0.258 across five benchmarks, despite maintaining in-scope accuracy.
Quick Take
A Mahalanobis distance scoring method effectively recovers most lost quality in three datasets without accuracy loss.
Key Points
- Few-shot LoRA personalization reduces OOS rejection AUROC by 0.055 to 0.258.
- In-scope accuracy remains intact despite the drop in OOS rejection performance.
- Mahalanobis distance scoring recovers quality on 3 out of 5 datasets.
- The drop in OOS performance is partly due to miscalibration of the confidence scorer.
- All code for reproducibility has been released by the authors.
Paper Resources
📖 Reader Mode
~2 min readAbstract:Always-on virtual assistants increasingly personalize on-device to new user-specific commands via few-shot fine-tuning (e.g., LoRA), but this adaptation is evaluated almost exclusively on whether it learns the new commands, not on whether the assistant can still recognize an input that matches none of its intents - the out-of-scope (OOS) rejection problem. We show that naive few-shot LoRA personalization degrades OOS-rejection AUROC (area under the ROC curve) on every one of five public intent-classification benchmarks (mean drop $0.055$--$0.258$, five seeds each), while in-scope accuracy is preserved. A scorer-invariant check shows part of this drop is an artifact of the standard confidence scorer's own miscalibration under adaptation, though a smaller, genuine effect remains under any scorer; we trace it to a measurable shrinkage of the embedding margin around known-intent regions (pooled Spearman rho=0.61$). A deployment-cheap fix is scoring rejection by Mahalanobis distance instead of softmax confidence without extra training/retraining recovers most of the lost quality on 3 of 5 datasets at no accuracy cost, beats two further OOD scoring baselines, and generalizes across shot count, LoRA rank, and a second base encoder. We report honestly where the fix falls short and release all code for reproducibility.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.20652 [cs.CL] |
| (or arXiv:2610.20652v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.20652 arXiv-issued DOI via DataCite |
Submission history
From: Pushp Garg [view email]
[v1]
Mon, 12 Oct 2026 18:54:51 UTC (210 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →The "10th Juror": Open-Set Standpoint Screening for Bureaucratic Bias Detection
MARS-Gov introduces a framework for detecting bureaucratic bias in Dutch government documents, achieving a new state-of-the-art F1 score of 0.880. This model outperforms existing zero-shot detectors by 20.2 points and reduces unnecessary interventions to just 2.5%. The framework's dynamic '10th juror' adapts to emerging biases, enhancing legal language processing.