Nemotron · NVIDIA-Open · HF: nvidia/Llama-3_3-Nemotron-Super-49B-v1
Llama-3.3 Nemotron Super 49B — VRAM, KV Cache & Hardware Matrix
Serverless API Rate
$0.15 in / $0.40 out per 1M
Parameters
49.9B Dense
Attention
GQA (8 KV / 64 Q)
Layers × Hidden
49 × 8192
Max Native Context
128k tokens
MoE Experts
N/A (Dense)
Quantization Memory Footprint
| Quantization | BPW | Weights Only | Total @ 8k Ctx | Total @ 32k Ctx | Quality |
|---|---|---|---|---|---|
| FP16 | 16.00 | 94.80 GiB | 98.21 GiB | 102.80 GiB | Lossless Reference |
| Q8_0 | 8.50 | 50.37 GiB | 53.77 GiB | 58.37 GiB | Near-Lossless |
| Q6_K | 6.57 | 38.93 GiB | 42.34 GiB | 46.93 GiB | High Precision |
| Q5_K_M | 5.69 | 33.71 GiB | 37.12 GiB | 41.71 GiB | Balanced High |
| Q4_K_M | 4.85 | 28.74 GiB | 32.14 GiB | 36.74 GiB | Recommended 4-bit |
| MLX_4BIT | 4.50 | 26.66 GiB | 30.07 GiB | 34.66 GiB | Apple Silicon Optimized |
| IQ4_XS | 4.25 | 25.18 GiB | 28.59 GiB | 33.18 GiB | Compact 4-bit |
| Q3_K_M | 3.91 | 23.17 GiB | 26.57 GiB | 31.17 GiB | Aggressive Compression |
Hardware Compatibility & Speed Leaderboard (Q4_K_M @ 32k Context)
| Hardware | Status | Total / Limit | Decode | TTFT (4k) | Max Safe Ctx | Inspect |
|---|---|---|---|---|---|---|
| Cloud NVIDIA H200 SXM (141 GB) | COMFORTABLE | 36.7 / 140.4 GiB | 107.7 tok/s | 0.63s | 128k | Simulate → |
| Cloud NVIDIA H100 SXM (80 GB) | COMFORTABLE | 36.7 / 79.4 GiB | 75.2 tok/s | 0.63s | 128k | Simulate → |
| Cloud NVIDIA A100 SXM (80 GB) | COMFORTABLE | 36.7 / 79.4 GiB | 44.7 tok/s | 2.02s | 128k | Simulate → |
| NVIDIA RTX PRO 6000 Blackwell (96 GB) | COMFORTABLE | 36.7 / 95.0 GiB | 40.2 tok/s | 2.52s | 128k | Simulate → |
| Cloud NVIDIA L40S (48 GB) | COMFORTABLE | 36.7 / 47.4 GiB | 18.9 tok/s | 1.80s | 88k | Simulate → |
| Apple Mac Studio M3 Ultra (256 GB) | COMFORTABLE | 36.5 / 199.7 GiB | 17.1 tok/s | 12.59s | 128k | Simulate → |
| Apple Mac Studio M3 Ultra (512 GB) | COMFORTABLE | 36.5 / 399.4 GiB | 17.1 tok/s | 12.59s | 128k | Simulate → |
| 2x NVIDIA RTX 3090 Rig (48 GB) | COMFORTABLE | 36.7 / 46.6 GiB | 16.8 tok/s | 11.84s | 84k | Simulate → |
| Apple Mac Studio M2 Ultra (192 GB) | COMFORTABLE | 36.5 / 149.8 GiB | 16.7 tok/s | 13.90s | 128k | Simulate → |
| Apple Mac Studio M4 Max (64 GB) | COMFORTABLE | 36.5 / 49.9 GiB | 11.2 tok/s | 19.16s | 128k | Simulate → |
| Apple Mac Studio M4 Max (128 GB) | COMFORTABLE | 36.5 / 99.8 GiB | 11.2 tok/s | 19.16s | 128k | Simulate → |
| Apple MacBook Pro M3 Max (64 GB) | COMFORTABLE | 36.5 / 49.9 GiB | 8.1 tok/s | 25.67s | 128k | Simulate → |
| Apple MacBook Pro M3 Max (128 GB) | COMFORTABLE | 36.5 / 99.8 GiB | 8.1 tok/s | 25.67s | 128k | Simulate → |
| Apple Mac Mini M4 Pro (64 GB) | COMFORTABLE | 36.5 / 49.9 GiB | 5.4 tok/s | 39.60s | 128k | Simulate → |
| Ryzen AI Max+ 395 Strix Halo (128 GB) | COMFORTABLE | 36.5 / 96.0 GiB | 5.1 tok/s | 14.25s | 128k | Simulate → |
| Ryzen AI Max+ 395 Strix Halo (64 GB) | COMFORTABLE | 36.5 / 48.0 GiB | 5.1 tok/s | 14.25s | 123k | Simulate → |
| Ryzen AI 9 HX 370 Mini-PC (64 GB) | COMFORTABLE | 36.5 / 48.0 GiB | 2.2 tok/s | 40.83s | 123k | Simulate → |
| Apple Mac Mini M4 Pro (48 GB) | TIGHT | 36.5 / 37.4 GiB | 5.4 tok/s | 39.60s | 62k | Simulate → |
| NVIDIA RTX 5090 (32 GB) | OOM | 36.7 / 31.0 GiB | OOM | — | 5k | Simulate → |
| NVIDIA RTX 4090 (24 GB) | OOM | 36.7 / 23.0 GiB | OOM | — | OOM | Simulate → |
| NVIDIA RTX 5080 (16 GB) | OOM | 36.7 / 15.0 GiB | OOM | — | OOM | Simulate → |
| NVIDIA RTX 3090 (24 GB Used) | OOM | 36.7 / 23.0 GiB | OOM | — | OOM | Simulate → |
| NVIDIA RTX 5070 Ti (16 GB) | OOM | 36.7 / 15.0 GiB | OOM | — | OOM | Simulate → |
| AMD Radeon RX 7900 XTX (24 GB) | OOM | 36.7 / 23.0 GiB | OOM | — | OOM | Simulate → |
| NVIDIA RTX 5060 Ti (16 GB) | OOM | 36.7 / 15.0 GiB | OOM | — | OOM | Simulate → |
| NVIDIA RTX 3060 (12 GB) | OOM | 36.7 / 11.0 GiB | OOM | — | OOM | Simulate → |
| Apple Mac Mini M4 (16 GB) | OOM | 36.5 / 10.0 GiB | OOM | — | OOM | Simulate → |
| Apple Mac Mini M4 (24 GB) | OOM | 36.5 / 18.0 GiB | OOM | — | OOM | Simulate → |
Interactive Llama-3.3 Nemotron Super 49B Simulator
49.9B Dense · GQA
1792 GB/s · 250 TFLOPS
3. Weight Quantization
4.85 bpw
4. KV Cache
32,768 tok
Compute Roofline
COMFORTABLE FIT Memory-Bandwidth Bound
Max Safe Ctx: 128k
Memory Allocation Footprint 36.74 / 95.00 GiB usable
Weights: 28.74 GiB
KV Cache: 6.13 GiB
Overhead: 1.88 GiB
Headroom: 58.26 GiB
Decode Speed
40.2
tok/s (batch=1)
Prefill Speed
1649
prompt tok/s
TTFT Latency
2.52s
for 4,096 tok
Active Params
49.9B
of 49.9B total
llama.cpp Server Command
llama-server -hf nvidia/Llama-3_3-Nemotron-Super-49B-v1:Q4_K_M -ngl 99 -fa on -c 32768 --port 8080| Context | KV Cache | Total VRAM | Decode | Status |
|---|---|---|---|---|
| 4k | 0.77 GiB | 30.82 GiB | 47.5 tok/s | COMFORTABLE |
| 8k | 1.53 GiB | 32.14 GiB | 46.3 tok/s | COMFORTABLE |
| 16k | 3.06 GiB | 33.68 GiB | 44.1 tok/s | COMFORTABLE |
| 32k | 6.13 GiB | 36.74 GiB | 40.2 tok/s | COMFORTABLE |
| 64k | 12.25 GiB | 42.86 GiB | 34.2 tok/s | COMFORTABLE |
| 128k | 24.50 GiB | 55.11 GiB | 26.3 tok/s | COMFORTABLE |
Buy-vs-Cloud-vs-API Monthly TCO Break-Even
USE SERVERLESS APIOwn Hardware/mo
$170.56
112.0h active + power
Cloud GPU/mo
$238.38
@$1.85/hr spot/pod
Serverless API/mo
$13.50
API cheaper
Recommended Hardware & Cloud Deployment
Contextual to fit statusEngineering FAQ — Llama-3.3 Nemotron Super 49B
How much VRAM does Llama-3.3 Nemotron Super 49B require in Q4_K_M?
Llama-3.3 Nemotron Super 49B weights occupy 28.74 GiB in Q4_K_M. Including 32k FP16 KV cache and runtime buffers, total memory is 36.74 GiB.
What is the cheapest local hardware that runs Llama-3.3 Nemotron Super 49B?
NVIDIA RTX PRO 6000 Blackwell (96 GB) (96 GB) runs Llama-3.3 Nemotron Super 49B Q4_K_M at ~40.2 tok/s decode and 2.52s TTFT.
How does Llama-3.3 Nemotron Super 49B scale KV cache at long context?
Llama-3.3 Nemotron Super 49B uses GQA with 8 KV heads across 49 layers (128 head_dim).