Deploying DeepSeek R1 671B on Kubernetes: vLLM FP8 Cluster Architecture Blueprint

DeepSeek R1 671B is the first open-weights reasoning model to match proprietary closed models on mathematical reasoning and code synthesis. However, running a 671B parameter Mixture of Experts model requires careful GPU memory budgeting and optimized tensor parallelism across high-speed NVLink fabrics.

1. VRAM Mathematics & FP8 Quantization

At BF16 precision, 671B parameters require ~1.34 TB of pure weight storage. By utilizing native FP8 block-wise quantization, the model weights consume ~680 GB. On a standard 8x NVIDIA H100 SXM5 node (8 x 80 GB = 640 GB), weights plus KV cache exceed a single node. Consequently, production deployments utilize either a dual-node 16x H100 setup or 8x H200 (141 GB HBM3e) nodes.

# vLLM DeepSeek R1 Deployment Args\n- "--model=deepseek-ai/DeepSeek-R1"\n- "--tensor-parallel-size=8"\n- "--max-model-len=32768"\n- "--enable-chunked-prefill=true"\n- "--trust-remote-code"\n- "--dtype=fp8"\n- "--kv-cache-dtype=fp8"

2. Multi-Head Latent Attention (MLA) Decoding

Unlike standard multi-head attention where KV cache explodes linearly with context length, DeepSeek R1 employs MLA. KV cache is projected into a low-dimensional latent space, reducing memory consumption to just 512 bytes per token. In vLLM 0.7.0, specialized Triton kernels bypass decompression during decode, allowing up to 128 concurrent active streams per cluster.