StepX-Edge: An On-Device UI Vision-Language Model via Architecture-Training-Deployment Co-Design
Quick Answer
StepX-Edge is a 0.9B-parameter on-device UI vision-language model that achieves superior UI understanding, surpassing 2B-2.3B baselines on benchmarks like ScreenQA and OCRBench.
Quick Take
It employs a unique architecture and training framework, maintaining accuracy within 1% post-quantization, and runs efficiently on Snapdragon 8 Gen5 devices.
Key Points
- Achieves 88.76 F1 on ScreenQA and 57.25 on Chinese OCRBench v2.
- Utilizes UI-aware Layered Visual Encoding and Progressive Dimensionality Projection.
- Employs a five-stage StepX-Curriculum framework for synergistic training.
- Maintains post-quantization accuracy loss within 1% using a two-stage quantization scheme.
- Runs on Snapdragon 8 Gen5 with ~0.84 s TTFT and 1.4 GB peak memory.
DeepSignal Analysis
What happened
StepX-Edge is a 0.9B-parameter on-device UI vision-language model that achieves high performance in UI understanding tasks. It surpasses larger models with 2B-2.3B parameters on benchmarks such as ScreenQA and OCRBench. The model maintains accuracy within 1% after quantization and operates efficiently on Snapdragon 8 Gen5 devices.
Key evidence
- StepX-Edge is a 0.9B-parameter model that outperforms 2B-2.3B baselines on benchmarks like ScreenQA (88.76 F1) and Chinese OCRBench v2 (57.25).
- The model employs a unique architecture with UI-aware Layered Visual Encoding (ULVE) and a Progressive Dimensionality Projection (PDP) connector.
- After W4A16+KV8 quantization, StepX-Edge achieves ~0.84 s TTFT and 98 tok/s decode on Snapdragon 8 Gen5 devices.
Why it matters
The development of StepX-Edge addresses the longstanding challenge of balancing accuracy and efficiency in deploying vision-language models on mobile devices. By achieving superior UI understanding with fewer parameters, it opens up possibilities for more efficient applications in mobile computing. The model's architecture and training framework could influence future designs in the field, particularly for resource-constrained environments.
Paper Resources
Source Excerpt
Deploying a with full UI understanding on end devices has long been trapped between accuracy and efficiency: on one side is the accuracy bar for OCR, screen understanding, visual question answering, and element grounding; on the other is the strict compute, memory, and power budget of mobile chips. Existing work either trades one for the other, or stops at simulation without real-device validation. We present StepX-Edge, a 0. 9B-parameter on-device UI vision-language model t
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CV
See more →ProMoE-FL: Prototype-conditioned Mixture of Experts for Multimodal Federated Learning with Missing Modalities
ProMoE-FL introduces a Prototype-conditioned Mixture-of-Experts framework for multimodal federated learning, effectively addressing missing modalities. It outperforms existing methods on four chest X-ray datasets, demonstrating superior feature synthesis capabilities in both homogeneous and heterogeneous settings.