We brought SGLang to NVIDIA Vera Rubin and ...
Quick Answer
SGLang has been integrated with NVIDIA's Vera Rubin, achieving up to 20% faster FP8 MLA and KDA verification.
Quick Take
The enhancements include a 5.9% end-to-end inference speedup through MoE tail fusion, significantly optimizing performance for Kimi K3 inference and RL training.
Key Points
- Achieved 20% faster FP8 MLA at batch size 1 with 128K context.
- KDA verification speed improved by 20% with identical output.
- MoE tail fusion resulted in a 5.9% end-to-end inference speedup.
- Reduced kernel launches by 276 per decode step.
- SGLang supports Miles' end-to-end RL training on Rubin.
📖 Reader Mode
~1 min read
We brought SGLang to NVIDIA Vera Rubin and accelerated Kimi K3 inference. Working closely with @NVIDIA, we optimized attention, MoE, and speculative verification kernels on early-access Rubin hardware. Highlights: • Up to 20% faster FP8 MLA at batch 1 / 128K context • 20% faster KDA verification, with bitwise-identical output • 5.9% end-to-end inference speedup from MoE tail fusion, removing 276 kernel launches per decode step SGLang also powers rollouts for Miles' end-to-end RL training on Rubin, including agentic RL with 64 concurrent sandboxes on the Vera CPU. Full results and engineering details 👉 lmsys.org/blog/2026-10-0…
— Originally published at x.com
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from WebSearch (Tavily)
See more →全球AI芯片峰会，9月上海见！
The 2026 Global AI Chip Summit will take place in Shanghai on September 22-23, focusing on the evolving AI chip landscape, including the shift from training to inference, the rise of diverse chip technologies, and the restructuring of industry competition. Notable speakers include experts from leading universities and companies, discussing advancements in AI chip architecture and applications.