TF-Engram: A Train-Free Engram with SSD-Backed Memory for Large Language Models
Quick Answer
TF-Engram is a train-free memory system for large language models, enhancing Qwen3-0.6B's performance from 57.6 to 59.4 on downstream tasks.
Quick Take
TF-Engram is a train-free memory system for large language models, enhancing Qwen3-0.6B's performance from 57.6 to 59.4 on downstream tasks. It utilizes SSD-backed storage to reduce GPU memory demands and employs predictive prefetching to mitigate latency during decoding, demonstrating efficient integration of static phrase memory.
Key Points
- TF-Engram constructs phrase-specific memory offline from external corpora.
- It improves downstream performance on Qwen3-0.6B by 1.8 points.
- SSD-backed storage significantly reduces GPU memory requirements.
- Early-Exit Guided Predictive Prefetching hides external memory latency.
- Static phrase memory can be integrated into LLM inference with low overhead.
Paper Resources
📖 Reader Mode
~2 min readAbstract:Large Language Models (LLMs) store factual knowledge and domain-specific patterns implicitly in dense Transformer parameters, making knowledge expansion costly through pretraining, fine-tuning, retrieval augmentation, or longer contexts. Engram-style memory offers a compact hidden-state injection pathway, but existing GPU-resident designs often rely on hash-based compression, causing unrelated phrases to collide in shared slots and weakening phrase-level semantic fidelity. We present TF-Engram, a train-free Engram system that constructs phrase-specific semantic memory offline from external corpora, stores large memory tables across a GPU--DRAM--SSD hierarchy, and uses Early-Exit Guided Predictive Prefetching to hide external-memory latency during autoregressive decoding. On Qwen3-0.6B, TF-Engram improves the average downstream score from 57.6 to 59.4, outperforming both the frozen backbone and a parameter-matched LoRA baseline. System evaluation shows that large TF-Engram tables can be built with moderate offline cost, SSD-backed storage substantially reduces GPU memory demand, and predictive prefetching recovers much of the throughput loss caused by external memory access. These results demonstrate that static phrase memory can be integrated into LLM inference as a scalable, train-free, and low-overhead system component.
| Comments: | 13 pages, 2 figures |
| Subjects: | Computation and Language (cs.CL); Machine Learning (cs.LG) |
| Cite as: | arXiv:2607.07388 [cs.CL] |
| (or arXiv:2607.07388v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2607.07388 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yutang Ma [view email]
[v1]
Wed, 8 Jul 2026 13:19:52 UTC (3,304 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →Quantifying Prior Dominance in Systems
The study introduces the Normalized Context Utilization (NCU) metric to evaluate Retrieval-Augmented Generation (RAG) systems, revealing that Small Language Models (SLMs) outperform larger models in factual extraction. The findings indicate that traditional scaling laws yield diminishing returns, with a commercial API frequently failing against adversarial evidence due to systemic confidence collapse.
