Mergeable Model-Side Aggregation States for Long-Context Language Models
Quick Answer
The paper introduces a model-side aggregation interface using Hash-based HyperLogLog (HLL) states to enhance long-context language models like Gemma 4 and Qwen, achieving up to 99.3% accuracy on aggregate-then-reason tasks.
Quick Take
This approach maintains a fixed HLL state size, allowing for efficient merging of states across segments, significantly improving performance over traditional reasoning methods by up to 63.2 points.
Key Points
- Introduces a compact aggregation interface for long-context models using HLL sketch states.
- Achieved 99.2% accuracy on 3,969 aggregate-then-reason tasks with Gemma 4.
- Fixed HLL state size of 2 KiB remains constant regardless of context length.
- Improved performance by 63.2 points over direct full-context reasoning on matched sets.
- Code for the proposed method is publicly available.
DeepSignal Analysis
What happened
The paper presents a model-side aggregation interface that utilizes Hash-based HyperLogLog (HLL) states to improve long-context language models. This method allows for efficient merging of states across segments, achieving high accuracy on various reasoning tasks.
Key evidence
- The proposed method reached 99.2% accuracy on 3,969 aggregate-then-reason tasks using Gemma 4, with a minimal gap of 0.8 percentage points compared to exact aggregation.
- In distinct-count experiments with one million records, the mean relative error was reported at 1.6%, demonstrating the effectiveness of the HLL state.
- The approach improved performance over direct full-context reasoning by 63.2 points on Qwen and 56.3 points on Gemma, indicating significant enhancements in reasoning tasks.
Why it matters
This advancement addresses a known limitation of long-context language models, which struggle with set-based aggregation as context length increases. By maintaining a fixed HLL state size, the method enhances efficiency and accuracy, making it applicable to various real-world tasks such as logs and multi-turn conversations.
What to watch
Paper Resources
📖 Reader Mode
~2 min readAbstract:A known limitation of long-context language models is their increasingly unreliable performance in non-additive, set-based aggregation as context length grows. Examples include cardinality estimation, set relationships, and grouped statistics, which widely exist in logs, program outputs, tables, and multi-turn conversations. To provide the aggregation state required by these tasks, we introduce a model-side aggregation interface that maintains compact Hash-based HyperLogLog (HLL) sketch states alongside a frozen language model. While the model processes the context, an extractor maps each relevant record to a canonical identity. The identity is then hashed and updates the HLL state. These states can be merged across context segments and/or read out directly for downstream reasoning, avoiding an additional generate-execute-return cycle. We validate the proposed approach by setting the HLL state size as 2 KiB (2,048 registers), which does not increase with context length or set cardinality. In a distinct-count experiment involving one million records, the mean relative error was 1.6%. In a separate merge test, states built from as many as 256 segments produced exactly the same readout as a single pass over the same stream. On 3,969 aggregate-then-reason tasks from 174 source windows, the fixed-budget interface reached 99.2% accuracy on Gemma 4 (31B, BF16), compared with 100.0% under exact aggregation; the paired gap was 0.8 percentage points (95% window-cluster CI: 0.5-1.3 points). On a matched set of 174 items, our method improved over direct full-context reasoning by 63.2 points on Qwen and 56.3 points on Gemma. The corresponding gains over chain-of-thought (CoT) reasoning were 60.9 and 63.2 points, respectively. On a fixed 1,200-task Oolong-Synth subset, our method reached 91.1% on Qwen and 99.3% on Gemma. Code is available at this https URL.
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2607.26448 [cs.CL] |
| (or arXiv:2607.26448v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2607.26448 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Dachuan Song [view email]
[v1]
Wed, 29 Jul 2026 03:56:12 UTC (187 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →TriAgent: Divergence-Aware Committees for Cost-Efficient Financial Sentiment Analysis
TriAgent introduces a cost-efficient multi-agent system for financial sentiment analysis, combining VADER, FinBERT, and Qwen2.5. It achieves an F1 score of ~0.87 with significant savings of $9.3M/year at a 10M-user scale compared to GPT-4o-mini, while also detecting hallucinations with an AUC of 0.90.