DeepSeek · DeepSeek · HF: deepseek-ai/DeepSeek-V3
DeepSeek-V3 671B — VRAM, KV Cache & Hardware Matrix
Serverless API Rate
$0.29 in / $1.14 out per 1M
Parameters
671B (37B act.)
Attention
MLA (128 KV / 128 Q)
Layers × Hidden
61 × 7168
Max Native Context
160k tokens
MoE Experts
8 / 256
Quantization Memory Footprint
| Quantization | BPW | Weights Only | Total @ 8k Ctx | Total @ 32k Ctx | Quality |
|---|---|---|---|---|---|
| FP16 | 16.00 | 1274.83 GiB | 1276.90 GiB | 1278.51 GiB | Lossless Reference |
| Q8_0 | 8.50 | 677.25 GiB | 679.32 GiB | 680.93 GiB | Near-Lossless |
| Q6_K | 6.57 | 523.48 GiB | 525.55 GiB | 527.16 GiB | High Precision |
| Q5_K_M | 5.69 | 453.36 GiB | 455.43 GiB | 457.04 GiB | Balanced High |
| Q4_K_M | 4.85 | 386.43 GiB | 388.50 GiB | 390.11 GiB | Recommended 4-bit |
| MLX_4BIT | 4.50 | 358.55 GiB | 360.62 GiB | 362.23 GiB | Apple Silicon Optimized |
| IQ4_XS | 4.25 | 338.63 GiB | 340.70 GiB | 342.31 GiB | Compact 4-bit |
| Q3_K_M | 3.91 | 311.54 GiB | 313.61 GiB | 315.22 GiB | Aggressive Compression |
Hardware Compatibility & Speed Leaderboard (Q4_K_M @ 32k Context)
Interactive DeepSeek-V3 671B Simulator
671B (37B active) · MLA
819 GB/s · 58 TFLOPS
3. Weight Quantization
4.85 bpw
4. KV Cache
32,768 tok
Compute Roofline
TIGHT FIT Memory-Bandwidth Bound
Max Safe Ctx: 160k
Memory Allocation Footprint 390.11 / 399.36 GiB usable
Weights: 386.43 GiB
KV Cache: 2.14 GiB
Overhead: 1.53 GiB
Headroom: 9.25 GiB
Decode Speed
22.3
tok/s (batch=1)
Prefill Speed
409
prompt tok/s
TTFT Latency
10.04s
for 4,096 tok
Active Params
37.0B
of 671B total
llama.cpp Server Command
llama-server -hf deepseek-ai/DeepSeek-V3:Q4_K_M -ngl 99 -fa on -c 32768 --port 8080| Context | KV Cache | Total VRAM | Decode | Status |
|---|---|---|---|---|
| 4k | 0.27 GiB | 387.74 GiB | 24.3 tok/s | TIGHT |
| 8k | 0.54 GiB | 388.50 GiB | 24.0 tok/s | TIGHT |
| 16k | 1.07 GiB | 389.04 GiB | 23.4 tok/s | TIGHT |
| 32k | 2.14 GiB | 390.11 GiB | 22.3 tok/s | TIGHT |
| 64k | 4.29 GiB | 392.26 GiB | 20.4 tok/s | TIGHT |
| 128k | 8.58 GiB | 396.55 GiB | 17.5 tok/s | TIGHT |
Buy-vs-Cloud-vs-API Monthly TCO Break-Even
USE SERVERLESS APIOwn Hardware/mo
$182.51
220.6h active + power
Cloud GPU/mo
$885.42
@$3.49/hr spot/pod
Serverless API/mo
$31.60
Payback: 299.7 mo
Recommended Hardware & Cloud Deployment
Contextual to fit statusEngineering FAQ — DeepSeek-V3 671B
How much VRAM does DeepSeek-V3 671B require in Q4_K_M?
DeepSeek-V3 671B weights occupy 386.43 GiB in Q4_K_M. Including 32k FP16 KV cache and runtime buffers, total memory is 390.11 GiB.
What is the cheapest local hardware that runs DeepSeek-V3 671B?
Apple Mac Studio M3 Ultra (512 GB) (512 GB) runs DeepSeek-V3 671B Q4_K_M at ~22.3 tok/s decode and 10.04s TTFT.
How does DeepSeek-V3 671B scale KV cache at long context?
DeepSeek-V3 671B uses Multi-Head Latent Attention (MLA, lora rank 512), compressing KV cache dramatically compared to standard GQA.