Khondo: A Multimodal Benchmark for Document Packet Splitting of Bangla Forms
Quick Answer
Khondo is the first benchmark for document packet splitting of Bangladeshi government forms, focusing on bilingual (Bangla-English) and vision-native approaches.
Quick Take
It reveals that while MLLMs cluster pages effectively, they struggle with restoring original page order, highlighting page-order reconstruction as a key challenge in low-resource document understanding.
Key Points
- Khondo includes five concatenation schemes across 14 administrative domains.
- Zero-shot evaluation shows MLLMs cluster pages well but struggle with page order.
- Explicit page-order instructions are necessary but not sufficient for success.
- English packets are ordered more reliably than Bangla, complicating arrangement.
- The dataset and code are publicly available for further research.
DeepSignal Analysis
What happened
Khondo is introduced as the first benchmark for splitting document packets of Bangladeshi government forms, focusing on bilingual and vision-native approaches. The benchmark reveals challenges in restoring original page order, particularly for low-resource languages like Bangla.
Key evidence
- Khondo is a bilingual benchmark that operates directly on page images, unlike previous datasets that were primarily English and OCR-text-based.
- The benchmark includes five concatenation schemes and covers 14 administrative domains, providing ground-truth boundaries and page order.
- Zero-shot evaluation of MLLMs indicates they effectively cluster pages but struggle with restoring the original page order, highlighting this as a significant challenge.
Why it matters
The introduction of Khondo addresses a gap in document understanding for low-resource languages, particularly in administrative contexts. By focusing on page-order reconstruction, it sets a clear direction for future research and development in this area. This benchmark could lead to improved document processing technologies that are more inclusive of diverse languages.
What to watch
Paper Resources
Source Excerpt
Document packets, multiple documents concatenated into a single file, are common in government and administrative workflows, yet splitting them into their constituent documents is difficult, especially for low-resource languages. We introduce Khondo (Bangla for split/segment), the first benchmark for document packet splitting on Bangladeshi government forms. Unlike prior English and OCR-text-based datasets, Khondo is bilingual (Bangla--English) and vision-native; where models operate directly on
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →TriAgent: Divergence-Aware Committees for Cost-Efficient Financial Sentiment Analysis
TriAgent introduces a cost-efficient multi-agent system for financial sentiment analysis, combining VADER, FinBERT, and Qwen2.5. It achieves an F1 score of ~0.87 with significant savings of $9.3M/year at a 10M-user scale compared to GPT-4o-mini, while also detecting hallucinations with an AUC of 0.90.