Shapley Context Pruning: A Cooperative Game Perspective for Context Reranking and Pruning
Quick Answer
The paper introduces Shapley Context Pruning (SCP), a novel framework for context reranking in Retrieval-Augmented Generation (RAG) systems, leveraging cooperative game theory for importance attribution.
Quick Take
SCP employs a Deep Sets architecture and Monte-Carlo sampling to achieve competitive performance in downstream QA tasks, demonstrating formal theoretical guarantees for preserving Top-K rankings.
Key Points
- SCP models context as a cooperative game for effective importance attribution.
- Utilizes Deep Sets architecture to approximate permutation-invariant value functions.
- Employs Monte-Carlo sampling for efficient training and inference.
- Achieves competitive QA performance against robust baselines in various evaluations.
- Provides theoretical error bounds and sample complexity guarantees for Top-K rankings.
Paper Resources
Source Excerpt
Context reranking and pruning have become essential for improving the efficiency of modern (RAG) systems, yet an interpretable and unified framework remains underexplored. Previous work has primarily emphasized lexical retrieval, cross-encoder architectures, model distillation, and Low-Rank Adaptation (LoRA), mostly relying on heuristic loss functions and empirical attribution. This paper presents Shapley Context Pruning (SCP), a novel framework for context reranki
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.AI
See more →RAIL Guard: Closing the Evaluation-to-Remediation Gap in Responsible AI for Agents
RAIL Guard introduces a closed-loop AI pipeline for large language models (LLMs) that evaluates outputs across eight dimensions and iteratively remediates failures, achieving 96.9% convergence compared to 49.1% for traditional block-and-retry methods. The system reduces unsafe agent executions by 33% without impacting task completion and is available as open-source SDKs.