DeepSciVerify: Verifying Scientific Claim--Citation Alignment via LLM-Driven Evidence Escalation
Quick Take
DeepSciVerify enhances scientific claim verification by combining abstract reasoning with selective evidence escalation.
Key Points
- Two-stage pipeline for claim-citation verification.
- Achieves 86.7 Micro-F1 on SCitance benchmark.
- Resolves 67% of cases without full-text retrieval.
Article Excerpt
From source RSS / original summaryarXiv:2605. 27710v1 Announce Type: new Abstract: Misalignment between claims and their cited evidence is a common failure mode in reports generated by large language models, limiting their reliability in scientific and other high-stakes settings. We present DeepSciVerify, a two-stage pipeline for scientific claim-citation verification that combines abstract-level reasoning with selective escalation to passage-level evidence.
The system first verifies claims using the abstract and defers uncertain cases, retrieving and analyzing full-text passages only when necessary. This design leverages complementary behaviors across LLMs, as some models are more conservative while others are more decisive under uncertainty. On the SCitance benchmark, DeepSciVerify achieves 86. 7 Micro-F1, outperforming strong abstract-only baselines by +4. 5 points while resolving 67% of instances without full-text retrieval.
These results suggest that selective evidence escalation improves both accuracy and efficiency in claim-citation verification.
Reader Mode unavailable (could not extract clean content).
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.AI
See more →Got a Secret? LLM Agents Can't Keep It: Evaluating Privacy in Multi-Agent Systems
LLM agents struggle with privacy in multi-agent systems, revealing significant risks under social pressure.