Shapley Context Pruning: A Cooperative Game Perspective for Context Reranking and Pruning
Quick Answer
The paper introduces Shapley Context Pruning (SCP), a novel framework for context reranking in Retrieval-Augmented Generation (RAG) systems, leveraging cooperative game theory for importance attribution.
Quick Take
SCP employs a Deep Sets architecture and Monte-Carlo sampling to achieve competitive performance in downstream QA tasks, demonstrating formal theoretical guarantees for preserving Top-K rankings.
Key Points
- SCP models context as a cooperative game for effective importance attribution.
- Utilizes Deep Sets architecture to approximate permutation-invariant value functions.
- Employs Monte-Carlo sampling for efficient training and inference.
- Achieves competitive QA performance against robust baselines in various evaluations.
- Provides theoretical error bounds and sample complexity guarantees for Top-K rankings.
Paper Resources
📖 Reader Mode
~2 min readAbstract:Context reranking and pruning have become essential for improving the efficiency of modern Retrieval-Augmented Generation (RAG) systems, yet an interpretable and unified framework remains underexplored. Previous work has primarily emphasized lexical retrieval, cross-encoder architectures, model distillation, and Low-Rank Adaptation (LoRA), mostly relying on heuristic loss functions and empirical attribution. This paper presents Shapley Context Pruning (SCP), a novel framework for context reranking that establishes a cooperative-game-theory perspective for importance attribution by modeling the context as a cooperative game. Balancing the trade-off between fine-grained and coarse-grained representations, we employ a Deep Sets architecture to approximate a permutation-invariant value function at the sentence level, utilizing pre-trained language models as sentence embedders and optimizing via a pairwise margin ranking loss. To ensure practical scalability without sacrificing mathematical rigor, we leverage Monte-Carlo sampling for efficient training and inference, providing formal theoretical error bounds and sample complexity guarantees for preserving Top-K subset rankings. Furthermore, we conduct comprehensive experiments-spanning supporting-sentence recall, Needle-in-the-Haystack (NIAH) evaluations, long-context QA, and multi-hop reasoning-alongside rigorous ablation studies on embedding quality and attribution strategies. The model achieves competitive downstream QA performance against robust baselines.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2607.16209 [cs.AI] |
| (or arXiv:2607.16209v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2607.16209 arXiv-issued DOI via DataCite |
Submission history
From: Yepang Liu [view email]
[v1]
Sun, 10 May 2026 13:27:01 UTC (2,346 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.AI
See more →HOBA: Hierarchical On-Policy Bidding Agents for Adaptive Online Advertising
HOBA (Hierarchical On-policy Bidding Agents) is a novel hierarchical reinforcement learning framework that enhances online advertising bidding systems by improving adaptability and reducing hyperparameter tuning costs. It utilizes a for hyperparameter inference, a SARSA agent for expert model selection, and a dynamic expert pool for bid execution, achieving a +3.6% increase in target cost during large-scale deployment and outperforming state-of-the-art baselines on AuctionNet.