DeepSeek has just open-sourced their high-performance inference software https://github.com/deepseek-ai/DeepInfer
inference infra is becoming a commodity
GitHubGitHub - deepseek-ai/DeepInfer: DeepInfer: High-performance, large-scale inference framework for DeepSeek modelsDeepInfer: High-performance, large-scale inference framework for DeepSeek models - deepseek-ai/DeepInfer
commit ab4cd33