DeepInfer: DeepSeek inference framework
DeepInfer is DeepSeek's open-source offline inference framework for DeepSeek-V4.1-Flash. It runs on NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs.
Key features
- Runs offline inference for DeepSeek-V4.1-Flash
- Supports NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs
- Constrains generation to a regex, JSON schema, or EBNF grammar
- Overlaps CPU batch scheduling with device execution by default
- Builds on Linux with Python 3.12, CUDA 13, or CANN 9.2
- A KV cache system is outside this release
- Released under the Apache License 2.0
- Category
- Trending Repos
- Who it's for
- Teams serving DeepSeek-V4.1-Flash on their own GPUs or NPUs.
- Tags
- githubdeepseekopen-modelslocal-inferencegpupython
- Website
- github.com
- Listed
- 2026-10-21
Frequently asked questions
Is DeepInfer free?
Yes. DeepInfer is open source and free to use.
What is DeepInfer used for?
Open-source offline inference for DeepSeek-V4.1-Flash on NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs. Typical uses: runs offline inference for DeepSeek-V4.1-Flash; supports NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs; constrains generation to a regex, JSON schema, or EBNF grammar.
Related in Trending Repos
See all →- addyosmani/agent-skills Engineering skills for AI coding agents, from Addy Osmani.
- anthropics/knowledge-work-plugins Open source plugins for knowledge workers to use in Claude Cowork.
- aquasecurity/trivy Scanner for vulnerabilities, misconfigurations, and secrets.
- AtomicBot-ai/atomic-agent Local-first AI agent that runs open-weight models via llama.cpp.
- axios/axios Promise-based HTTP client for the browser and Node.js.
- bevyengine/bevy Data-driven game engine written in Rust.