Vectors Are Not Neutral: Sensitive-Information Inference from Exported LLM Representations in Summarization
Quick Answer
This study reveals that exported vector representations from LLM summarization systems can infer sensitive information, such as race from EHRs, despite access restrictions.
Quick Take
This study reveals that exported vector representations from LLM summarization systems can infer sensitive information, such as race from EHRs, despite access restrictions. The proposed SurfaceLoRA method effectively reduces the recoverability of sensitive labels while maintaining summarization quality, highlighting the need for targeted privacy auditing.
Key Points
- Sensitive information can be inferred from LLM-generated vector representations.
- SurfaceLoRA reduces recoverability of sensitive labels while preserving summarization utility.
- Privacy auditing must focus on specific vector artifacts retained for downstream use.
- EHR-recorded race recoverability remains high from untargeted pooled artifacts.
- The study emphasizes risks in clinical discharge-summary generation.
Paper Resources
Article Content
From source RSS / original summaryarXiv:2605. 26433v1 Announce Type: new Abstract: Large language model (LLM) summarization systems may pass compact vector representations of private inputs to downstream retrieval, monitoring, audit, or analytic workflows. Even when source documents remain access-restricted, derived vectors may be handled under different access controls and still support sensitive-information inference, creating a residual information-disclosure risk.
We study this issue in clinical discharge-summary generation as a high-stakes case study, using electronic health record (EHR)-recorded race as a controlled sensitive-label audit. We audit two artifacts that a system might retain or expose to downstream components: the final prompt-token hidden state and the mean-pooled prompt representation. Our results show that reducing recoverability of the case-study sensitive label from one exported artifact does not necessarily reduce recoverability from another.
As a mitigation case study, we introduce SurfaceLoRA, an exported-vector-targeted parameter-efficient fine-tuning method that uses a gradient-reversal discriminator attached to a designated exported vector. Under a balanced five-way probing protocol, SurfaceLoRA reduces EHR-recorded race recoverability from the targeted final-token artifact toward chance while preserving summarization utility, yet recoverability remains substantially higher from untargeted pooled artifacts.
These findings show that privacy auditing and mitigation should be performed on the exact vector artifact retained or exposed to downstream components.
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →Letting the Data Speak: Extracting Keywords from Crowdsourced Collections with AI
The study evaluates three NLP approaches—Named Entity Recognition, Keyword Extraction, and Topic Modelling—using the Their Finest Hour Online Archive to automate keyword extraction from crowdsourced WWII collections. Findings suggest that while NLP methods show promise, no single approach is sufficient, and ethical considerations in automated keyword extraction are crucial for responsible stewardship.