Workload-Driven Optimization for On-Device Real-Time Subtitle Translation
Quick Answer
The study presents a workload-driven optimization for on-device English-to-Traditional-Chinese subtitle translation, achieving a 59.2% win rate against Google Translate using LocalSubs on a 500-example subset.
Quick Take
The study presents a workload-driven optimization for on-device English-to-Traditional-Chinese subtitle translation, achieving a 59.2% win rate against Google Translate using LocalSubs on a 500-example subset. The model, based on LMT-60-0.6B, shows a 1.63x speedup with a 64k-token vocabulary compared to a 151k-token baseline, particularly excelling in short input scenarios.
Key Points
- LocalSubs achieves a 59.2% win rate against Google Translate on OpenSubtitles2024.
- Model optimization includes a shift to a 64k-token subtitle-domain tokenizer.
- 1.63x speedup observed on Apple M2 Metal with the optimized model.
- Performance declines as cue length increases, strongest on short inputs.
- Preliminary latency results indicate incomplete benchmark configuration.
Paper Resources
📖 Reader Mode
~2 min readAbstract:This report studies on-device English-to-Traditional-Chinese subtitle translation for Taiwan under short inputs, short outputs, batch-size-one inference, low latency, and privacy constraints. These conditions limit the value of optimizations designed for long-context or high-throughput language-model serving.
Starting from LMT-60-0.6B, preliminary profiling suggests that vocabulary projection becomes a more important decode-time cost after GGUF quantization reduces the relative cost of Transformer blocks. We replace the original 151k-token vocabulary with a 64k-token subtitle-domain tokenizer, migrate the embedding space, and adapt the model through embedding calibration followed by full supervised fine-tuning.
On a fixed 500-example subset of the OpenSubtitles2024 test set, the LocalSubs achieves a 59.2% tie-excluded win rate against Google Translate under GPT-4o pairwise judging. Performance is strongest on short cues and declines as cue length increases. Preliminary Apple M2 Metal measurements on a 64k-vocabulary model show a 1.63$\times$ speedup over a 151k-vocabulary profiling baseline. The raw benchmark configuration is incomplete, so the latency result is treated as preliminary.
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2607.09957 [cs.CL] |
| (or arXiv:2607.09957v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2607.09957 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Tsz-To Wong [view email]
[v1]
Fri, 10 Jul 2026 20:22:13 UTC (12 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →Letting the Data Speak: Extracting Keywords from Crowdsourced Collections with AI
The study evaluates three NLP approaches—Named Entity Recognition, Keyword Extraction, and Topic Modelling—using the Their Finest Hour Online Archive to automate keyword extraction from crowdsourced WWII collections. Findings suggest that while NLP methods show promise, no single approach is sufficient, and ethical considerations in automated keyword extraction are crucial for responsible stewardship.