Beyond Similarity: Grounded Agentic Extraction and Expert-Adjudicated Evaluation of Intertextuality in Classical Chinese Histories
Quick Answer
This study introduces a novel agentic extraction method for intertextuality in classical Chinese texts using large language models (LLMs), achieving precision rates between 56%-93% across 12 models.
Quick Take
It validates findings against a benchmark of 2,533 intertextual pairs, revealing a cost spread of 51x at comparable quality. The approach uncovers deeper structural insights beyond similarity scores, demonstrating stability in citation practices over 18 centuries.
Key Points
- Introduces agentic extraction for intertextuality using , enhancing traditional methods.
- Validated against a benchmark of 2,533 intertextual pairs with expert adjudication.
- Achieves precision rates from 56% to 93% across twelve LLMs.
- Demonstrates a 51x cost spread at similar quality levels.
- Reveals stability in citation practices over 18 centuries despite less literal quoting.
Paper Resources
📖 Reader Mode
~2 min readAbstract:Computational approaches to intertextuality have advanced from string matching to neural retrieval, yet their outputs, similarity scores and parallel-passage lists, identify where texts reuse one another without characterizing how or why. We recast fine-grained intertextuality extraction as an agentic task in which a large language model (LLM) reads two text units in full and, through a constrained tool interface, must ground each proposed reuse in exact character spans on both sides and label it under a five-dimension typology of reuse (form, aspect, source-marking, function, stance). We validate the approach on an exhaustive comparison of the Analects with the Book of Han, where three domain experts adjudicate a pooled multi-model candidate set into a benchmark of 2,533 intertextual pairs. Against this standard we study twelve LLMs, reporting precision (56%-93%), a 51$\times$ cost spread at comparable quality, and how well their confidence is calibrated. Expert agreement traces a reliability gradient: dimensions legible on the textual surface are annotated consistently, while those requiring inference of intent are contested, delimiting the claims such annotation supports. Scaling the validated extractor to the full Twenty-Four Histories (65,380 comparisons, 5,766 pairs) recovers corpus-level structure a similarity score cannot express. The interpretive composition of citation shows no systematic change across eighteen centuries, yet the same passage is quoted less and less literally. Stability in the aggregate with drift in the individual case is what a cultural-attraction account expects. We release the extraction protocol and the expert-adjudicated benchmark.
| Comments: | 9 pages, 4 figures, 3 tables |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Digital Libraries (cs.DL) |
| Cite as: | arXiv:2607.27595 [cs.CL] |
| (or arXiv:2607.27595v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2607.27595 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Zhaoji Wang [view email]
[v1]
Thu, 30 Jul 2026 02:34:23 UTC (661 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →TriAgent: Divergence-Aware Committees for Cost-Efficient Financial Sentiment Analysis
TriAgent introduces a cost-efficient multi-agent system for financial sentiment analysis, combining VADER, FinBERT, and Qwen2.5. It achieves an F1 score of ~0.87 with significant savings of $9.3M/year at a 10M-user scale compared to GPT-4o-mini, while also detecting hallucinations with an AUC of 0.90.