Xiaomi MiMo and TileRT Push a 1-Trillion-Parameter Model Past 1000 Tokens Per Second on Commodity GPUs
Quick Answer
Xiaomi's MiMo team, in collaboration with TileRT, has launched MiMo-V2.5-Pro-UltraSpeed, achieving over 1000 tokens per second decoding on a 1-trillion-parameter model using a single 8-GPU commodity node.
Quick Take
This advancement significantly enhances performance for AI applications, making high-capacity models more accessible on standard hardware.
Key Points
- MiMo-V2.5-Pro-UltraSpeed decodes over 1000 tokens per second.
- Utilizes a single 8-GPU commodity node for processing.
- Targets a 1-trillion-parameter model for enhanced performance.
- Collaboration between Xiaomi's MiMo team and TileRT.
- Significant implications for AI application accessibility.
Source Excerpt
Xiaomi MiMo-V2. 5-Pro-UltraSpeed decodes past 1000 tokens per second on commodity GPUs using FP4 quantization and DFlash speculative decoding.
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from MarkTechPost
See more →Meet Flash-KMeans: An IO-Aware, Exact K-Means That Runs Over 200× Faster Than FAISS on GPUs
Flash-KMeans is an open-source, IO-aware k-means implementation that operates over 200× faster than FAISS on NVIDIA H200 GPUs. It achieves 17.9× end-to-end and 33× speedup over cuML by optimizing distance calculations and updating mechanisms without approximating results. This advancement significantly enhances performance for data scientists and machine learning practitioners.
