RIMS: Preference Optimization via Smoothed Multi-pair Aggregation for Small-Scale LLM Retrieval-Augmented Generation
Quick Answer
RIMS introduces a three-stage preference optimization framework for small-scale language models (SLMs) in retrieval-augmented generation, enhancing performance under noisy conditions.
Quick Take
It outperforms existing methods like RoseRAG on multi-hop question answering benchmarks, achieving significant improvements in Exact Match and F1 scores. The approach leverages synthetic data generation and a differentiable soft aggregation mechanism to optimize preference selection.
Key Points
- RIMS utilizes synthetic chain-of-thought preference data generation for improved SLM performance.
- The framework includes a differentiable soft aggregation mechanism for better gradient alignment.
- Experiments show RIMS outperforms state-of-the-art methods on four multi-hop question answering benchmarks.
- Significant gains in Exact Match and F1 scores were achieved under noisy retrieval conditions.
- Implementation details are available at the provided URL.
DeepSignal Analysis
What happened
RIMS presents a three-stage framework aimed at optimizing preferences for small-scale language models in retrieval-augmented generation scenarios. This method reportedly enhances performance in noisy conditions, surpassing existing models like RoseRAG on multi-hop question answering benchmarks.
Key evidence
- RIMS employs synthetic data generation through rejection sampling using the target small-scale language model, avoiding reliance on proprietary models.
- The framework introduces a differentiable soft aggregation mechanism, which maintains gradient signals from all preference pairs while preserving the structure of margin-aware selection.
- Experiments demonstrate that RIMS achieves significant improvements in Exact Match and F1 scores across four multi-hop question answering benchmarks under noisy retrieval conditions.
Why it matters
The development of RIMS is significant as it addresses the limitations of existing preference-based methods that either discard useful signals or treat comparisons independently. By optimizing preference selection in a more nuanced way, RIMS could enhance the utility of small-scale language models in practical applications, especially in environments where resources are limited and noise is prevalent.
Paper Resources
Source Excerpt
Small-scale language models (SLMs) are attractive for (RAG) in resource-constrained settings, but their limited capacity makes them highly sensitive to noisy or spurious retrieved evidence. Existing preference-based methods such as RoseRAG select only the hardest single preference pair via hard argmin/argmax, discarding the remaining signal; others treat multiple pairs as independent binary comparisons, resulting in low data utilization. We propose RIMS, a three-st
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →Letting the Data Speak: Extracting Keywords from Crowdsourced Collections with AI
The study evaluates three NLP approaches—Named Entity Recognition, Keyword Extraction, and Topic Modelling—using the Their Finest Hour Online Archive to automate keyword extraction from crowdsourced WWII collections. Findings suggest that while NLP methods show promise, no single approach is sufficient, and ethical considerations in automated keyword extraction are crucial for responsible stewardship.