The Role of Prompt Language and Translation-Theory-Driven Prompts in Large Language Models: A Case Study on Spanish-Chinese Journalistic Translation
Quick Answer
This study evaluates how prompt language and translation theory-driven prompts affect Spanish-Chinese translations by GPT-5.2, revealing that while automated metrics favored baseline prompts, human evaluations preferred brief-oriented prompts.
Quick Take
Translation theory-driven prompts reduced awkward style errors but did not significantly impact unidiomatic errors, suggesting potential quality improvements in expert evaluations.
Key Points
- Study involved 48 experimental conditions across four Spanish editorials.
- Automated metrics ranked baseline prompts highest, while human evaluation favored brief-oriented prompts.
- Translation theory-driven prompts reduced awkward style errors effectively.
- Unidiomatic style errors persisted despite prompt variations.
- Implications for language learners require further validation through user studies.
Paper Resources
📖 Reader Mode
~2 min readAbstract:This study examines how prompt language and translation theory-driven prompt design influence the quality of Spanish-Chinese journalistic translations generated by GPT-5.2. A parallel corpus of four editorials from El Pais was translated under 48 experimental conditions (4 prompt types, 3 prompt languages, and 4 articles). Translation quality was assessed using BLEU and BERTScore-F1 for automated evaluation, alongside human evaluation based on the Multidimensional Quality Metrics (MQM) framework. Automated metrics identified the baseline prompt (BASE) as the best-performing condition, whereas human evaluation ranked the brief-oriented prompt (BRIEF) highest (MQM: 8.66 vs. 7.84), a reversal likely attributable to the single-reference constraint inherent in automated measures. Sub-error type analysis revealed that translation theory-driven prompts selectively reduced Awkward style errors, while Unidiomatic style errors persisted across conditions. Prompt language had a negligible impact under both evaluation paradigms. These results indicate that translation theory-driven prompts can yield measurable quality gains under expert evaluation of journalistic translations, although their pedagogical implications for language learners remain suggestive and require validation through user-based studies.
| Comments: | Published in the Proceedings of the 27th Annual Conference of the European Association for Machine Translation (EAMT 2026), pp. 927-945. ACL Anthology entry forthcoming |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2607.03160 [cs.CL] |
| (or arXiv:2607.03160v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2607.03160 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Haohong Lai [view email]
[v1]
Fri, 3 Jul 2026 09:59:36 UTC (508 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →TriAgent: Divergence-Aware Committees for Cost-Efficient Financial Sentiment Analysis
TriAgent introduces a cost-efficient multi-agent system for financial sentiment analysis, combining VADER, FinBERT, and Qwen2.5. It achieves an F1 score of ~0.87 with significant savings of $9.3M/year at a 10M-user scale compared to GPT-4o-mini, while also detecting hallucinations with an AUC of 0.90.