Simorgh at SemEval-2026 task 7: Region-Aware Hybrid Retrieval for Low-Resource Cultural Reasoning in Multilingual Question Answering
Quick Take
The paper presents a region-aware hybrid retrieval method for culturally grounded multilingual question answering using the Qwen3-14B model, improving cross-lingual stability over traditional parametric inference. Despite advancements, performance disparities remain between languages with varying training data availability.
Key Points
- Introduces a hybrid retrieval approach combining BM25 and dense semantic similarity.
- Utilizes the BLEnD benchmark with a multilingual corpus of 30 languages.
- Improvements noted in culturally grounded question answering with Qwen3-14B model.
- Performance gaps persist between high and low resource languages.
- Retrieval augmentation does not fully address training data imbalance issues.
Article Content
From source RSS / original summaryarXiv:2605. 27636v1 Announce Type: new Abstract: Although Large Language Models (LLMs) demonstrate excellent capabilities and performance for general reasoning tasks within the general public domain, they may face challenges with culturally grounded knowledge within languages with limited digital and textual data.
In this paper, we investigate culturally grounded multiple-choice question answering with the BLEnD benchmark, which consists of a multilingual corpus of 30 languages and covers various socio-cultural domains, such as cuisine, sports, family, etc. We propose a region-aware hybrid retrieval approach that combines BM25 lexical matching and dense semantic similarity with regional weighting heuristics to improve the relevance of the answer.
The retrieved documents are used to construct a structured prompt for the Qwen3-14B quantized model with logit-based deterministic answer selection. The experimental results show improvements to cross-lingual stability with the hybrid retrieval approach over pure parametric inference for culturally grounded question answering. However, there are still notable performance gaps between languages with more and less training data.
This shows that the limitations of the retrieval augmentation approach are not entirely overcome by the training data imbalance problem.
Reader Mode unavailable (could not extract clean content).
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →Time to REFLECT: Can We Trust LLM Judges for Evidence-based Research Agents?
The reliability of LLM judges for evaluating deep research agents is critically assessed using the REFLECT benchmark.