RubricReviewer: From Direct Critique to Objective and Comprehensive Rubric-Driven Peer Review
Quick Answer
RubricReviewer introduces a rubric-driven framework for peer review, combining a training-free agent (Scout) and a human-aligned model (Aligner) to enhance review comprehensiveness and robustness against adversarial attacks.
Quick Take
Experiments show it outperforms prior systems in generating more discriminative reviews.
Key Points
- RubricReviewer explicitly generates adaptive rubrics for peer review.
- Combines training-free evidence gathering with human-aligned review generation.
- Produces more comprehensive and discriminative reviews than previous systems.
- Demonstrates strong robustness against adversarial prompt-injection attacks.
- Ablation studies confirm the necessity of each component in the framework.
Paper Resources
Source Excerpt
Peer review at major venues is under unprecedented submission pressure, motivating the use of (LLMs) as review assistants. Existing LLM-based reviewers, however, face two structural limitations. First, they map manuscripts directly to reviews, leaving the underlying rubric implicit and entangling its derivation with the judgement. Second, the prevailing paradigms each capture only half of a good review: training-free agents gather broad evidence but produce undirected criti
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →TriAgent: Divergence-Aware Committees for Cost-Efficient Financial Sentiment Analysis
TriAgent introduces a cost-efficient multi-agent system for financial sentiment analysis, combining VADER, FinBERT, and Qwen2.5. It achieves an F1 score of ~0.87 with significant savings of $9.3M/year at a 10M-user scale compared to GPT-4o-mini, while also detecting hallucinations with an AUC of 0.90.