Deployment-Time Memorization in Foundation-Model Agents
Quick Answer
This paper shows that Foundation-model agents, such as Gemma 3 12B and GPT-4o-mini, utilize deployment-time memorization, balancing personalization recall and extraction risk.
Quick Take
Foundation-model agents, such as Gemma 3 12B and GPT-4o-mini, utilize deployment-time memorization, balancing personalization recall and extraction risk. Key findings reveal that summarization reduces adversarial extraction by up to 76%, but deletion fidelity suffers, with 20% recoverability of deleted data unless comprehensive purging is applied.
Key Points
- Summarization reduces adversarial extraction by 76% on Gemma 3 12B.
- Personalization recall remains nearly intact despite aggressive summarization.
- Increasing retrieval breadth (k) post-compression does not restore leakage.
- 20% of deleted information remains recoverable without full-pipeline purge.
- Forgetting Residue Score (FRS) quantifies recoverability of deleted data.
Paper Resources
Article Content
From source RSS / original summaryarXiv:2606. 10062v1 Announce Type: new Abstract: Foundation-model agents are increasingly long-lived systems that remember users across interactions, making memorization an explicit deployment-time function rather than solely a property of model weights. Existing work addresses parametric memorization or audits fixed memory configurations, but does not characterize how memory-design choices jointly shape personalization utility, extraction risk, and deletion fidelity.
We study this surface as deployment-time memorization, formulating as a privacy-utility frontier measured by Personalization Recall (PR) and Adversarial Extraction Rate (AER), and sweeping three memory-design knobs: summarization aggressiveness, retrieval breadth (k), and deletion mode. We further introduce the Forgetting Residue Score (FRS) to quantify whether deleted information remains recoverable from derived memory tiers.
On LongMemEval, key-fact summarization reduces canary extraction by 76% on Gemma 3 12B and 64% on GPT-4o-mini while preserving nearly all personalization recall; critically, once content is compressed away, increasing k no longer restores leakage. The same compression, however, induces a deletion-fidelity failure: raw-only deletion leaves derived summary copies recoverable in approximately 20% of instances, and only full-pipeline purge or tombstone redaction drives worst-tier residue to zero.
Together, these results establish that persistent agent memory must be evaluated as a first-class memorization mechanism -- assessed by what it helps agents recall, what it makes extractable, and what it can truly erase.
Reader Mode unavailable (could not extract clean content).
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.AI
See more →Arbor: Tree Search as a Cognition Layer for Autonomous Agents
Arbor introduces a multi-agent framework utilizing structured tree search for optimizing LLM inference, achieving up to 193% throughput-latency improvement compared to vendor-optimized systems. It employs an Orchestrator and Critic agent for stability and coordination, demonstrating hardware-agnostic performance with minimal variance.