DeepSeek open-sources DeepInfer: the production framework behind its own services becomes open source
The Chinese AI startup DeepSeek open-sourced DeepInfer on October 20, 2026 — the inference framework that powers the company's own production services. The release, made under the Apache 2.0 license, offers a rare first-hand look at how a frontier lab actually serves its models at scale. But the most economically interesting details come with an important caveat: all published benchmarks were run on Nvidia hardware, and every figure comes from DeepSeek itself. Huawei support is official — performance on Ascend is unverified.
What DeepInfer is
DeepInfer is the framework behind DeepSeek's production services, according to SiliconANGLE. The framework was originally built primarily on vLLM, with ideas drawn from SGLang and TensorRT-LLM, according to MarkTechPost, which in turn cites DeepSeek's GitHub documentation.
Alongside DeepInfer, the company launched the deepseek-recipe tool, which handles token encoding and decoding and is designed to match the logic of DeepSeek's official API, Crypto Briefing reports. For developers who want to replicate DeepSeek's serving setup, that means they can expect tokenization identical to what the API actually does.
Which hardware is supported — and what isn't proven
According to DeepSeek's deployment documentation, DeepInfer supports Nvidia's SM100 Blackwell GPUs and Huawei's Ascend 950 neural processors. Hopper and other older Nvidia architectures are not on the list. That means the framework targets the newest chip generations on both sides of the Pacific — while excluding much of the installed base.
This is where the caveat is central: all published performance tests were run on Nvidia chips, and two features that keep output identical from run to run (determinism) work only on Nvidia hardware, according to SiliconANGLE. So the fact that the Ascend 950 is officially supported does not mean that performance on Huawei chips is documented. What is verified is the support in the code — not the speed.
The figures DeepSeek itself reports
DeepSeek's deployment documentation contains a series of economically interesting claims, all of which should be read as the company's own numbers:
- 155 trillion tokens per day per cluster, according to the document, cited by SiliconANGLE.
- $38 per GPU per hour in revenue at off-peak pricing, according to the same document.
- More than 25,000 output tokens per second per GPU in decode throughput at 256 requests per rank, according to MarkTechPost.
- The smallest efficient deployment described uses 64 GPUs for prefill and 64 for decode.
None of these figures has been independently verified. They nonetheless offer a rare glimpse into how a frontier lab calculates its operating economics — and they suggest why the company emphasizes low cost per token as a competitive advantage.
DSpark: speculative decoding that cuts verification
One concrete technical element in the release is DSpark, DeepSeek's system for speculative decoding. The method lets a smaller model propose tokens that the main model verifies in parallel. DeepSeek states that the system cuts the number of draft tokens checked per step by roughly 30 percent, and that the company estimates a throughput gain of 4–7 percent at its own production load. The same caveat applies here as elsewhere: the estimates are DeepSeek's own, not independently measured.
What is being held back
The release is not complete. DeepSeek has not released the code for key-value cache management, P/D disaggregation (separating prefill and decode across different machines), or load balancing, according to SiliconANGLE and MarkTechPost. The company's justification is that these components are tightly coupled to its internal infrastructure. A blog post the company has published on Zhihu describes how they work, but the code is not public.
That means the practical value of the release for outside parties lies mostly in the core of the inference engine — not in the parts that largely govern the costs and utilization of a cluster in production.
Why the Huawei support matters
The release lands in a sensitive period. DeepInfer follows a September 30 release in which DeepSeek, with Huawei's help, brought the TileLang kernel programming language to Huawei's chips, according to SiliconANGLE. A frontier lab with production experience officially listing the Ascend 950 in its production framework is a concrete data point in Huawei's effort to present itself as a credible alternative to Nvidia as U.S. restrictions tighten.
The analysis from Jack Gold, president of J. Gold Associates, given to TechTarget, points in the same direction: DeepSeek running the engine on hardware from the Chinese chipmaker is good for Huawei, which is trying to build a reputation as a reliable alternative to Nvidia. But the picture has two sides. The fact that DeepSeek still runs all published benchmarks on Nvidia chips, and that the determinism features work only there, suggests Huawei is still lagging in at least some areas — a point TechTarget's coverage indicates.
For enterprises, the value of such releases is tied to cost and speed. "There is a lot of discussion around token optimization," Lian Jye Su, an analyst at Omdia, told TechTarget. "Token consumption has exploded, and the idea is that if you can perform a task quickly and at low cost, it helps … optimize your investment in AI infrastructure."
Timing: one day after the DHS order
The release came one day after the U.S. Department of Homeland Security (DHS) ordered federal civilian agencies to remove DeepSeek products from their systems, including third-party services built on the company's technology, according to SiliconANGLE. The source does not link the release directly to the DHS decision, and there is reason not to speculate about the motivation behind the timing. But the context is clear: DeepSeek is open-sourcing its code and documenting its operations at the same time U.S. authorities are tightening access to the company's products in the public sector.
What remains open
Several questions are unanswered by the available source material:
- Ascend performance is unverified. No benchmarks have been published on Huawei's chips, and there are as yet no independent measurements.
- A second-hand estimate of 160,000 Ascend 950DT accelerators planned for a data center in Inner Mongolia, reported by Bloomberg and cited in SiliconANGLE, could not be confirmed and should be treated as unverified.
- The relationship between the September 30 release (TileLang and Ascend tools) and the October 20 release (DeepInfer) is not fully reconciled across the sources. They are treated here as two separate releases, with the September release as background for the Huawei collaboration.
What the DeepInfer release actually delivers is two things at once: rare, concrete documentation of how a frontier lab serves models in production — and a reminder that in geopolitically charged AI inference, there is still a difference between what is officially supported and what is proven.