Workload-Driven Optimization for On-Device Real-Time Subtitle Translation
Quick Answer
The study presents a workload-driven optimization for on-device English-to-Traditional-Chinese subtitle translation, achieving a 59.2% win rate against Google Translate using LocalSubs on a 500-example subset.
Quick Take
The model, based on LMT-60-0.6B, shows a 1.63x speedup with a 64k-token vocabulary compared to a 151k-token baseline, particularly excelling in short input scenarios.
Key Points
- LocalSubs achieves a 59.2% win rate against Google Translate on OpenSubtitles2024.
- Model optimization includes a shift to a 64k-token subtitle-domain tokenizer.
- 1.63x speedup observed on Apple M2 Metal with the optimized model.
- Performance declines as cue length increases, strongest on short inputs.
- Preliminary latency results indicate incomplete benchmark configuration.
Paper Resources
📖 Reader Mode
~2 min readAbstract:This report studies on-device English-to-Traditional-Chinese subtitle translation for Taiwan under short inputs, short outputs, batch-size-one inference, low latency, and privacy constraints. These conditions limit the value of optimizations designed for long-context or high-throughput language-model serving.
Starting from LMT-60-0.6B, preliminary profiling suggests that vocabulary projection becomes a more important decode-time cost after GGUF quantization reduces the relative cost of Transformer blocks. We replace the original 151k-token vocabulary with a 64k-token subtitle-domain tokenizer, migrate the embedding space, and adapt the model through embedding calibration followed by full supervised fine-tuning.
On a fixed 500-example subset of the OpenSubtitles2024 test set, the LocalSubs achieves a 59.2% tie-excluded win rate against Google Translate under GPT-4o pairwise judging. Performance is strongest on short cues and declines as cue length increases. Preliminary Apple M2 Metal measurements on a 64k-vocabulary model show a 1.63$\times$ speedup over a 151k-vocabulary profiling baseline. The raw benchmark configuration is incomplete, so the latency result is treated as preliminary.
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2607.09957 [cs.CL] |
| (or arXiv:2607.09957v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2607.09957 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Tsz-To Wong [view email]
[v1]
Fri, 10 Jul 2026 20:22:13 UTC (12 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →TriAgent: Divergence-Aware Committees for Cost-Efficient Financial Sentiment Analysis
TriAgent introduces a cost-efficient multi-agent system for financial sentiment analysis, combining VADER, FinBERT, and Qwen2.5. It achieves an F1 score of ~0.87 with significant savings of $9.3M/year at a 10M-user scale compared to GPT-4o-mini, while also detecting hallucinations with an AUC of 0.90.