Soro: A Lightweight Foundation Model and Chatbot for Tajik
Quick Take
Soro is a Tajik-specialized conversational LLM that significantly outperforms Gemma 3 on new benchmarks while maintaining strong English performance. It was pretrained on a 1.9-billion-token corpus and tuned with 40K examples, enabling deployment in Tajikistan's educational sector with reduced memory requirements through FP8 and INT4 quantization.
Key Points
- Soro is based on open-weight Gemma 3 checkpoints.
- Pretrained on a 1.9-billion-token corpus specific to Tajik language.
- Outperforms Gemma 3 in Tajik benchmarks while retaining English performance.
- Introduces new benchmarks for evaluating Tajik language capabilities.
- Utilizes FP8 and INT4 quantization for efficient edge deployment.
Article Content
From source RSS / original summaryarXiv:2605. 27379v1 Announce Type: new Abstract: We present Soro, a family of Tajik-specialized conversational large language models (LLMs) designed for real-world deployment under tight compute and connectivity constraints in Tajikistan. Starting from open-weight Gemma 3 checkpoints, we perform Tajik-only continual pretraining on a curated 1.
9-billion-token corpus spanning filtered web text, PDF documents, and curriculum-aligned educational materials, followed by supervised instruction tuning on 40K Tajik teacher-style examples. To enable rigorous evaluation despite the limited coverage of Tajik in standard benchmarks, we introduce a suite of Tajik benchmarks covering general knowledge, linguistic competence, and school- and university entrance-exam domains, and we open-source them on Hugging Face.
Across these Tajik benchmarks, Soro substantially outperforms same-size Gemma 3 baselines while retaining strong English performance on standard datasets. We further show that FP8 and INT4 quantization of Soro preserves most Tajik-language gains while reducing memory requirements for edge deployment, supporting an ongoing education-sector pilot and planned scale-out across schools in Tajikistan.
Reader Mode unavailable (could not extract clean content).
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.AI
See more →The Importance of Out-of-Band Metadata for Safe Autonomous Agents: The Redpanda Agentic Data Plane
The Redpanda Agentic Data Plane (ADP) introduces out-of-band metadata channels to enhance the safety of autonomous AI agents, ensuring secure data access and tamper-proof audit trails. This architecture mitigates risks associated with unpredictable AI behavior by enforcing governance throughout the agent lifecycle, demonstrated in a multi-agent trading system with strict data scoping and approval thresholds.