StepX-Edge: An On-Device UI Vision-Language Model via Architecture-Training-Deployment Co-Design
Quick Answer
StepX-Edge is a 0.9B-parameter on-device UI vision-language model that achieves superior UI understanding, surpassing 2B-2.3B baselines on benchmarks like ScreenQA and OCRBench.
Quick Take
It employs a unique architecture and training framework, maintaining accuracy within 1% post-quantization, and runs efficiently on Snapdragon 8 Gen5 devices.
Key Points
- Achieves 88.76 F1 on ScreenQA and 57.25 on Chinese OCRBench v2.
- Utilizes UI-aware Layered Visual Encoding and Progressive Dimensionality Projection.
- Employs a five-stage StepX-Curriculum framework for synergistic training.
- Maintains post-quantization accuracy loss within 1% using a two-stage quantization scheme.
- Runs on Snapdragon 8 Gen5 with ~0.84 s TTFT and 1.4 GB peak memory.
DeepSignal Analysis
What happened
StepX-Edge is a 0.9B-parameter on-device UI vision-language model that achieves high performance in UI understanding tasks. It surpasses larger models with 2B-2.3B parameters on benchmarks such as ScreenQA and OCRBench. The model maintains accuracy within 1% after quantization and operates efficiently on Snapdragon 8 Gen5 devices.
Key evidence
- StepX-Edge is a 0.9B-parameter model that outperforms 2B-2.3B baselines on benchmarks like ScreenQA (88.76 F1) and Chinese OCRBench v2 (57.25).
- The model employs a unique architecture with UI-aware Layered Visual Encoding (ULVE) and a Progressive Dimensionality Projection (PDP) connector.
- After W4A16+KV8 quantization, StepX-Edge achieves ~0.84 s TTFT and 98 tok/s decode on Snapdragon 8 Gen5 devices.
Why it matters
The development of StepX-Edge addresses the longstanding challenge of balancing accuracy and efficiency in deploying vision-language models on mobile devices. By achieving superior UI understanding with fewer parameters, it opens up possibilities for more efficient applications in mobile computing. The model's architecture and training framework could influence future designs in the field, particularly for resource-constrained environments.
Paper Resources
📖 Reader Mode
~2 min readAbstract:Deploying a vision-language model with full UI understanding on end devices has long been trapped between accuracy and efficiency: on one side is the accuracy bar for OCR, screen understanding, visual question answering, and element grounding; on the other is the strict compute, memory, and power budget of mobile chips. Existing work either trades one for the other, or stops at simulation without real-device validation. We present StepX-Edge, a 0.9B-parameter on-device UI vision-language model that resolves this tension through three-layer co-design of architecture, training, and deployment. Architecturally, UI-aware Layered Visual Encoding (ULVE) and a Progressive Dimensionality Projection (PDP) connector target the extreme aspect ratios and fine-grained perception of screens, while standard full attention throughout ensures native compatibility with mainstream mobile NPU operators. For training, the five-stage StepX-Curriculum framework is designed around our observation of mutual-promotion effects among UI subtasks, so that all four capabilities grow synergistically under a tight parameter budget rather than interfering. For deployment, a module-wise differentiated two-stage PTQ-to-QAT quantization scheme keeps the post-quantization accuracy loss within 1%. StepX-Edge achieves the strongest overall UI understanding among <=1B models, surpassing all 2B-2.3B baselines on ScreenQA (88.76 F1) and Chinese OCRBench v2 (57.25), and matching 1.3B-2.3B general VLMs on RefCOCO (92.0%) and OCRBench v1 (831) with far fewer parameters. After W4A16+KV8 quantization, the model runs stably on Snapdragon 8 Gen5 devices with ~0.84 s TTFT, 98 tok/s decode, and 1.4 GB peak memory. We will open-source the training data, the full training recipe, and the quantization deployment pipeline.
| Subjects: | Computer Vision and Pattern Recognition (cs.CV) |
| Cite as: | arXiv:2607.22708 [cs.CV] |
| (or arXiv:2607.22708v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2607.22708 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Haotian Hu [view email]
[v1]
Mon, 20 Jul 2026 14:14:44 UTC (6,258 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CV
See more →ProMoE-FL: Prototype-conditioned Mixture of Experts for Multimodal Federated Learning with Missing Modalities
ProMoE-FL introduces a Prototype-conditioned Mixture-of-Experts framework for multimodal federated learning, effectively addressing missing modalities. It outperforms existing methods on four chest X-ray datasets, demonstrating superior feature synthesis capabilities in both homogeneous and heterogeneous settings.