DeepSeek open-sources DeepInfer, the Apache License 2.0 inference framework that powers DeepSeek-V4.1-Flash, on Oct 20, 2026
DeepSeek's Oct 20, 2026 GitHub release publishes DeepInfer, its high performance large scale production inference framework that powers DeepSeek V4.1 Flash, un…
Published on MyPrivateClaw
Oct 21, 2026, 4:22 PM UTC
Coverage date
Oct 20, 2026
Last updated
Oct 21, 2026, 4:22 PM UTC
News summary
DeepSeek open sourced DeepInfer on GitHub on Oct 20, 2026 under the Apache License 2.0. The framework is described as a high performance, large scale production inference framework that powers DeepSeek V4.1 Flash; it is an open source release of an internal inference framework, not a model or dataset release. DeepInfer was originally built primarily on vLLM, with additional inspiration from SGLang and TensorRT LLM. It accelerates decoding via DSpark draft token decoding with confidence scheduled verification, SWA bounded replay (a YOCO prefill truncation keeping 140 tokens), and optimized kernels including FlashMLA, DeepGEMM, and DeepEP. DeepInfer reports benchmark throughput measured on 4 SM100 GPUs for prefill and 64 SM100 GPUs for decode, each within one scale up domain. Those figures are DeepSeek's own single vendor benchmarks; no third party reproduction or comparison to vLLM/SGLan…