NVIDIA AI on X: "SGLang is hitting 180 tok/s/GPU on DeepSeek-V4 decode with ~1M context on Blackwell. Good to see fast progress in open source DeepSeek-V4 inference on new hardware. This comes from Blackwell-specific optimizations by @lmsysorg that better use the model’s hybrid sparse" / X
Quick Answer
NVIDIA's SGLang achieves 180 tok/s/GPU on DeepSeek-V4 decoding with ~1M context on Blackwell, showcasing significant advancements in open-source inference.
Quick Take
NVIDIA's SGLang achieves 180 tok/s/GPU on DeepSeek-V4 decoding with ~1M context on Blackwell, showcasing significant advancements in open-source inference. Optimizations by @lmsysorg enhance the model's hybrid sparse attention capabilities, ensuring robust performance from launch.
Key Points
- SGLang reaches 180 tok/s/GPU on DeepSeek-V4 with ~1M context.
- Optimizations by @lmsysorg improve hybrid sparse attention on Blackwell.
- DeepSeek V4 launched with full stack optimizations from architecture to kernels.
- Verified RL training pipeline available for V4 at launch.
📖 Reader Mode
~1 min readPost
Post
SGLang is hitting 180 tok/s/GPU on DeepSeek-V4 decode with ~1M context on Blackwell. Good to see fast progress in open source DeepSeek-V4 inference on new hardware. This comes from Blackwell-specific optimizations by
@lmsysorgthat better use the model’s hybrid sparse attention.
DeepSeek V4 by
@deepseek_aijust dropped! SGLang is ready on Day 0 with a full stack of optimizations from architectures to low-level kernels. We also deliver a verified RL training pipeline in Miles (by
@radixark) for V4 at launch: 1️⃣ Native "ShadowRadix" Design: DeepSeek V4's
Don't miss what's happening
People on X are the first to know.
— Originally published at x.com
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from WebSearch (Tavily)
See more →Stop just chatting with AI. Learn to build production-ready software in ...
The 2026 Bootcamp offers hands-on training in building production-ready software using Generative AI, LLM applications, and AI agents, emphasizing practical skills over casual interaction with AI. Participants will learn to develop applications like Cursor AI, preparing them for real-world challenges in AI development.
