GLAN-QnA-KR: A Seedless Taxonomy-Driven Korean Instruction Corpus
Quick Answer
GLAN-QnA-KR is a newly released synthetic Korean instruction-QA corpus with 303,581 rows, generated using Microsoft's Phi-3.5-MoE-instruct.
Quick Take
It features a unique seedless taxonomy-driven approach, ensuring minimal duplication and contamination, making it the largest of its kind on the Hugging Face Hub.
Key Points
- Corpus contains 303,581 rows with a flat taxonomy of 1,084 disciplines.
- Only one exact duplicate question found across the entire dataset.
- Contamination audit shows maximum Jaccard index of 0.163 with no high duplicates.
- Median length of questions is 313 characters; answers average 1,098 characters.
- Released under OpenRAIL license, suitable for downstream citation.
DeepSignal Analysis
What happened
The GLAN-QnA-KR corpus has been released, consisting of 303,581 rows of synthetic Korean instruction-QA data. It was generated using Microsoft's Phi-3.5-MoE-instruct model and employs a seedless taxonomy-driven approach to minimize duplication.
Key evidence
- The corpus includes 1,084 English-labelled disciplines paired with Korean question/answer text, with a median of 313 question characters and 1,098 answer characters per record.
- A contamination audit revealed a maximum character-trigram Jaccard of 0.163, indicating minimal overlap with existing datasets, and no test items exceeded a Jaccard of 0.7.
- At the time of release, GLAN-QnA-KR is the largest synthetic Korean instruction corpus verifiable on the Hugging Face Hub, built under a seedless taxonomy-driven protocol.
Why it matters
This release is significant as it provides a large-scale, openly redistributable resource for Korean instruction-based question-answering tasks. The unique generation method aims to reduce contamination from existing datasets, which is crucial for training robust AI models. The corpus's size and structure could enhance research and development in natural language processing for the Korean language.
Paper Resources
📖 Reader Mode
~2 min readAbstract:We release GLAN-QnA-KR, a 303,581-row openly redistributable Korean instruction-QA corpus produced via the seedless taxonomy-driven GLAN synthesis pipeline with Microsoft's Phi-3.5-MoE-instruct as the producer model (generation: 2024-12; release: 2024-12; licence: OpenRAIL). The corpus spans a flat taxonomy of 1,084 English-labelled disciplines paired with Korean question/answer text, a 100-900 difficulty scale, and a median of 313 question characters and 1,098 answer characters per record. Two properties are atypical for synthetic instruction data at this scale: (i) exact duplicate questions number only 1 in 303,581 rows and character-trigram near-duplicate clusters at Jaccard >= 0.9 number zero in a 5,000-sample probe, and (ii) a two-layer contamination audit against KMMLU, KoBEST (five sub-tasks), and HAE-RAE-Bench shows a maximum test-vs-corpus question-level character-trigram Jaccard of 0.163 with zero test items at Jaccard >= 0.7, and a maximum multilingual-E5 cosine of 0.901 with a single test item at cosine >= 0.90 and zero at >= 0.95, across 20,000 sampled GLAN questions and seven evaluation sets. At the time of release, this is, to our knowledge, the largest single-pipeline synthetic Korean instruction corpus verifiable on the Hugging Face Hub and the only Korean >=100k-row corpus built under a seedless taxonomy-driven protocol. This note documents the generation protocol, corpus statistics, the contamination audit, and the licensing boundary in a form suitable for downstream citation.
| Comments: | Technical report; 7 pages, 4 tables. Dataset: this https URL |
| Subjects: | Computation and Language (cs.CL); Machine Learning (cs.LG) |
| Cite as: | arXiv:2607.20443 [cs.CL] |
| (or arXiv:2607.20443v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2607.20443 arXiv-issued DOI via DataCite |
Submission history
From: Daekeun Kim [view email]
[v1]
Tue, 12 May 2026 15:23:59 UTC (12 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →TriAgent: Divergence-Aware Committees for Cost-Efficient Financial Sentiment Analysis
TriAgent introduces a cost-efficient multi-agent system for financial sentiment analysis, combining VADER, FinBERT, and Qwen2.5. It achieves an F1 score of ~0.87 with significant savings of $9.3M/year at a 10M-user scale compared to GPT-4o-mini, while also detecting hallucinations with an AUC of 0.90.