Refining Word-Based Grammatical Error Annotation for L2 Korean
Quick Take
This study enhances Korean grammatical error correction (K-GEC) by refining word-based annotation to address morpheme-level errors, improving evaluation metrics and model performance. The new ERRANT-style scheme and multi-reference KoLLA corpus yield lower perplexity and better agreement in edit representations, benefiting neural GEC systems.
Key Points
- Refined annotation addresses structural mismatches in Korean grammatical errors.
- New ERRANT-style scheme distinguishes various error types effectively.
- Multi-reference evaluation reduces penalties for valid corrections in GEC.
- Empirical validation shows improved performance for KoBART-based correction.
- Enhanced resources reflect Korean morphology and correction variability.
Article Content
From source RSS / original summaryarXiv:2605. 30545v1 Announce Type: new Abstract: Korean grammatical error correction (K-GEC) presents a structural mismatch between word-based evaluation and the morpheme-level locus of many learner errors. Postpositions and verbal endings are bound to lexical hosts, but they encode grammatical relations that must be represented in correction and evaluation.
This paper refines word-based grammatical error annotation for L2 Korean by addressing three connected problems in existing resources: surface target realization, Korean-specific edit annotation, and single-reference evaluation. We reconstruct target sentences from the National Institute of Korean Language (NIKL) L2 corpus under morphologically constrained realization rules and convert its morpheme-level annotations into word-level \texttt{m2} edits.
We then define a Korean ERRANT-style annotation scheme that preserves the MRU core while distinguishing functional morpheme errors, spelling errors, word boundary errors, and word order errors. We also augment the KoLLA corpus with an additional reference correction, yielding a multi-reference evaluation setting for Korean GEC.
Empirical validation shows that the refined NIKL targets yield lower perplexity, the converted \texttt{m2} files achieve higher agreement with source-target edit representations, and the refined resources improve KoBART-based correction under the same model setting. Multi-reference KoLLA evaluation further reduces the penalty imposed on valid corrections that diverge from a single reference, especially for neural and prompted GEC systems.
These results show that Korean GEC evaluation depends not only on correction models, but also on reference data and edit annotations that reflect Korean morphology, spacing, and correction variability.
Reader Mode unavailable (could not extract clean content).
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →Time to REFLECT: Can We Trust LLM Judges for Evidence-based Research Agents?
The REFLECT benchmark reveals that current LLM judges are unreliable, achieving below 55% accuracy in evaluating reasoning and evidence use, highlighting the need for improved evaluation methods for deep research agents.