DeepSeek has just open-sourced their high-performance inference software https://github.com/deepseek-ai/DeepInfer
inference infra is becoming a commodity
GitHub - deepseek-ai/DeepInfer: DeepInfer: High-performance, large-scale inference framework for DeepSeek models
link
GitHub - deepseek-ai/DeepInfer: DeepInfer: High-performance, large-scale inference framework for DeepSeek models
github.com/deepseek-ai/DeepInfer
DeepInfer: High-performance, large-scale inference framework for DeepSeek models - deepseek-ai/DeepInfer
- 2commits
- 2people
DeepSeek open-sourced its production inference engine. Bring your own KV cache, load balancer and 64 Blackwell GPUs. https://github.com/deepseek-ai/DeepInfer
That's every commit about this link.