DeepSeek open-sources DeepInfer, the inference engine behind its production models

DeepSeek has open-sourced DeepInfer, the high-performance inference framework that runs its own models in production. The release landed on October 20, 2026, under the Apache-2.0 license. The framework is built for DeepSeek models specifically. It powers the lab’s production-scale deployments, including DeepSeek-V4.1-Flash. What DeepInfer actually does DeepInfer bundles a set of optimizations aimed at making inference faster and cheaper to run. The toolkit includes custom kernels, distributed parallelism, speculative decoding and structured output support. Custom kernels are low-level routines written to squeeze more work out of specific chips. Distributed parallelism splits a model’s workload across many processors at once. The framework includes MoE expert parallelism, which suits mixture-of-experts models. Those models route each query to specialized sub-networks instead of firing up the whole system. Speculative decoding lets a model guess several upcoming tokens ahead of time, then verify them. When the guesses land, output arrives faster. Structured output keeps responses in predictable formats, such as clean JSON. The numbers DeepSeek is putting on the table The performance ...