Translating Classical Poetry into Modern Prose
Quick Take
The Padyam2Gadyam dataset introduces 600 translations of 13th-17th Century Telugu poetry into modern Telugu and English. Evaluation of five contemporary Large Language Models reveals significant room for improvement in poem-to-prose translation capabilities across both languages.
Key Points
- Dataset includes 600 human-verified translations of classical Telugu poetry.
- Evaluated five contemporary Large Language Models for translation accuracy.
- Results indicate significant performance gaps across models in both languages.
- Qualitative analysis highlights limitations of current MT evaluation methods.
- Study emphasizes the need for improved translation techniques in classical literature.
Article Excerpt
From source RSS / original summaryarXiv:2606. 02806v1 Announce Type: new Abstract: We introduce Padyam2Gadyam, a dataset for the task of poem-to-prose translation from 13th-17th Century Telugu Classical Poetry to contemporary Telugu and English prose. The dataset consists of 600 poems and their human-verified Telugu and English prose translations. We evaluated 5 contemporary Large Language Models (LLMs) on their ability to do poem-to-prose translation into Telugu and English.
Our results indicate that while there are differences across LLMs, their overall performance leave a large room for improvement in both languages. Through qualitative analysis, we discuss the the capabilities and limitations of contemporary MT evaluation approaches for this task.
Reader Mode unavailable (could not extract clean content).
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →Time to REFLECT: Can We Trust LLM Judges for Evidence-based Research Agents?
The REFLECT benchmark reveals that current LLM judges are unreliable, achieving below 55% accuracy in evaluating reasoning and evidence use, highlighting the need for improved evaluation methods for deep research agents.