DeepSeek open-sourced its production inference engine. Bring your own KV cache, load balancer and 64 Blackwell GPUs. https://github.com/deepseek-ai/DeepInfer
GitHubGitHub - deepseek-ai/DeepInfer: DeepInfer: High-performance, large-scale inference framework for DeepSeek modelsDeepInfer: High-performance, large-scale inference framework for DeepSeek models - deepseek-ai/DeepInfer
commit d1ed871