BaFCo: A Document Understanding Benchmark for Complex Bangla Form Comprehension
Quick Answer
BaFCo introduces a benchmark dataset for Bangla form comprehension, featuring 200 complex government forms and a detailed annotation schema.
Quick Take
BaFCo introduces a benchmark dataset for Bangla form comprehension, featuring 200 complex government forms and a detailed annotation schema. Evaluations of MLLMs like ChatGPT and Gemini reveal significant limitations in understanding Bangla forms, particularly in localizing granular entities.
Key Points
- BaFCo focuses on Document Layout Analysis and Key Information Extraction.
- The dataset includes 200 multi-page forms from various sectors like agriculture and education.
- A fine-grained annotation schema defines 26 entity types for accurate comprehension.
- Current MLLMs struggle with localizing complex form entities in Bangla.
- The dataset and code are publicly available for further research.
Paper Resources
📖 Reader Mode
~2 min readAuthors:Abu Tyeb Azad, Ishita Sur Apan, Fahim Ahmed, Sumaiya Karim Katha, Ezharuddin Jubaer, Armun Alam, Pranjal Kumar Nandi, Amin Ahsan Ali, Aman Chadha, Md Mofijul Islam, AKM Mahbubur Rahman
Abstract:Document comprehension is a challenging yet impactful task for Multimodal Large Language Models, especially as these systems see growing adoption in real-world, human-centric applications. However, this adoption is limited for low-resource languages such as Bangla due to the scarcity of high-quality annotated data. To address this gap, we introduce BaFCo, a benchmark dataset for Bangla form comprehension with a focus on Document Layout Analysis (DLA) and Key Information Extraction (KIE). BaFCo curates 200 multi-page complex Bangladeshi government forms, sourced from across diverse sectors including agriculture, education, banking, and land management. To accurately capture the structural and contextual complexity of these forms, we define a fine-grained annotation schema comprising 26 types of form entities, along with a separate coarse form entity set consisting of 5 types. We evaluate the latest MLLMs from the ChatGPT, Gemini, Claude, Qwen, and Kimi series using zero-shot and chain-of-thought prompts under both low and high reasoning setups. Our results reveal limitations in current MLLMs' ability in comprehending Bangla forms, particularly in accurately localizing highly granular form entities. Our dataset and code is available at: this https URL
| Comments: | Accepted at the 19th European Conference on Computer Vision (ECCV), 2026 |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV) |
| Cite as: | arXiv:2607.05614 [cs.CL] |
| (or arXiv:2607.05614v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2607.05614 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Fahim Ahmed [view email]
[v1]
Mon, 6 Jul 2026 20:17:38 UTC (23,180 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →Quantifying Prior Dominance in Systems
The study introduces the Normalized Context Utilization (NCU) metric to evaluate Retrieval-Augmented Generation (RAG) systems, revealing that Small Language Models (SLMs) outperform larger models in factual extraction. The findings indicate that traditional scaling laws yield diminishing returns, with a commercial API frequently failing against adversarial evidence due to systemic confidence collapse.