Gemma 3 · Gemma · HF: google/gemma-3-4b-it
Gemma 3 4B IT — VRAM, KV Cache & Hardware Matrix
Serverless API Rate
$0.05 in / $0.10 out per 1M
Parameters
4.3B Dense
Attention
GQA (4 KV / 8 Q)
Layers × Hidden
34 × 2560
Max Native Context
128k tokens
MoE Experts
N/A (Dense)
Quantization Memory Footprint
| Quantization | BPW | Weights Only | Total @ 8k Ctx | Total @ 32k Ctx | Quality |
|---|---|---|---|---|---|
| FP16 | 16.00 | 8.17 GiB | 9.57 GiB | 10.13 GiB | Lossless Reference |
| Q8_0 | 8.50 | 4.34 GiB | 5.74 GiB | 6.30 GiB | Near-Lossless |
| Q6_K | 6.57 | 3.35 GiB | 4.75 GiB | 5.32 GiB | High Precision |
| Q5_K_M | 5.69 | 2.91 GiB | 4.30 GiB | 4.87 GiB | Balanced High |
| Q4_K_M | 4.85 | 2.48 GiB | 3.87 GiB | 4.44 GiB | Recommended 4-bit |
| MLX_4BIT | 4.50 | 2.30 GiB | 3.70 GiB | 4.26 GiB | Apple Silicon Optimized |
| IQ4_XS | 4.25 | 2.17 GiB | 3.57 GiB | 4.13 GiB | Compact 4-bit |
| Q3_K_M | 3.91 | 2.00 GiB | 3.39 GiB | 3.96 GiB | Aggressive Compression |
Hardware Compatibility & Speed Leaderboard (Q4_K_M @ 32k Context)
| Hardware | Status | Total / Limit | Decode | TTFT (4k) | Max Safe Ctx | Inspect |
|---|---|---|---|---|---|---|
| Cloud NVIDIA H200 SXM (141 GB) | COMFORTABLE | 4.4 / 140.4 GiB | 1125.7 tok/s | 0.09s | 128k | Simulate → |
| Cloud NVIDIA H100 SXM (80 GB) | COMFORTABLE | 4.4 / 79.4 GiB | 785.6 tok/s | 0.09s | 128k | Simulate → |
| Cloud NVIDIA A100 SXM (80 GB) | COMFORTABLE | 4.4 / 79.4 GiB | 466.8 tok/s | 0.21s | 128k | Simulate → |
| NVIDIA RTX PRO 6000 Blackwell (96 GB) | COMFORTABLE | 4.4 / 95.0 GiB | 420.3 tok/s | 0.26s | 128k | Simulate → |
| NVIDIA RTX 5090 (32 GB) | COMFORTABLE | 4.4 / 31.0 GiB | 410.3 tok/s | 0.31s | 128k | Simulate → |
| NVIDIA RTX 4090 (24 GB) | COMFORTABLE | 4.4 / 23.0 GiB | 230.8 tok/s | 0.39s | 128k | Simulate → |
| NVIDIA RTX 5080 (16 GB) | COMFORTABLE | 4.4 / 15.0 GiB | 219.8 tok/s | 0.56s | 128k | Simulate → |
| NVIDIA RTX 3090 (24 GB Used) | COMFORTABLE | 4.4 / 23.0 GiB | 209.1 tok/s | 0.89s | 128k | Simulate → |
| NVIDIA RTX 5070 Ti (16 GB) | COMFORTABLE | 4.4 / 15.0 GiB | 205.1 tok/s | 0.70s | 128k | Simulate → |
| AMD Radeon RX 7900 XTX (24 GB) | COMFORTABLE | 4.4 / 23.0 GiB | 203.7 tok/s | 0.62s | 128k | Simulate → |
| Cloud NVIDIA L40S (48 GB) | COMFORTABLE | 4.4 / 47.4 GiB | 197.8 tok/s | 0.19s | 128k | Simulate → |
| Apple Mac Studio M3 Ultra (256 GB) | COMFORTABLE | 4.2 / 199.7 GiB | 178.3 tok/s | 1.15s | 128k | Simulate → |
| Apple Mac Studio M3 Ultra (512 GB) | COMFORTABLE | 4.2 / 399.4 GiB | 178.3 tok/s | 1.15s | 128k | Simulate → |
| 2x NVIDIA RTX 3090 Rig (48 GB) | COMFORTABLE | 4.4 / 46.6 GiB | 175.6 tok/s | 1.08s | 128k | Simulate → |
| Apple Mac Studio M2 Ultra (192 GB) | COMFORTABLE | 4.2 / 149.8 GiB | 174.2 tok/s | 1.27s | 128k | Simulate → |
| Apple Mac Studio M4 Max (64 GB) | COMFORTABLE | 4.2 / 49.9 GiB | 117.4 tok/s | 1.74s | 128k | Simulate → |
| Apple Mac Studio M4 Max (128 GB) | COMFORTABLE | 4.2 / 99.8 GiB | 117.4 tok/s | 1.74s | 128k | Simulate → |
| NVIDIA RTX 5060 Ti (16 GB) | COMFORTABLE | 4.4 / 15.0 GiB | 100.1 tok/s | 1.31s | 128k | Simulate → |
| Apple MacBook Pro M3 Max (64 GB) | COMFORTABLE | 4.2 / 49.9 GiB | 84.9 tok/s | 2.31s | 128k | Simulate → |
| Apple MacBook Pro M3 Max (128 GB) | COMFORTABLE | 4.2 / 99.8 GiB | 84.9 tok/s | 2.31s | 128k | Simulate → |
| NVIDIA RTX 3060 (12 GB) | COMFORTABLE | 4.4 / 11.0 GiB | 78.4 tok/s | 2.49s | 128k | Simulate → |
| Apple Mac Mini M4 Pro (48 GB) | COMFORTABLE | 4.2 / 37.4 GiB | 56.4 tok/s | 3.56s | 128k | Simulate → |
| Apple Mac Mini M4 Pro (64 GB) | COMFORTABLE | 4.2 / 49.9 GiB | 56.4 tok/s | 3.56s | 128k | Simulate → |
| Ryzen AI Max+ 395 Strix Halo (128 GB) | COMFORTABLE | 4.2 / 96.0 GiB | 52.9 tok/s | 1.30s | 128k | Simulate → |
| Ryzen AI Max+ 395 Strix Halo (64 GB) | COMFORTABLE | 4.2 / 48.0 GiB | 52.9 tok/s | 1.30s | 128k | Simulate → |
| Apple Mac Mini M4 (16 GB) | COMFORTABLE | 4.2 / 10.0 GiB | 24.1 tok/s | 7.83s | 128k | Simulate → |
| Apple Mac Mini M4 (24 GB) | COMFORTABLE | 4.2 / 18.0 GiB | 24.1 tok/s | 7.83s | 128k | Simulate → |
| Ryzen AI 9 HX 370 Mini-PC (64 GB) | COMFORTABLE | 4.2 / 48.0 GiB | 23.4 tok/s | 3.66s | 128k | Simulate → |
Interactive Gemma 3 4B IT Simulator
4.3B Dense · GQA
1792 GB/s · 250 TFLOPS
3. Weight Quantization
4.85 bpw
4. KV Cache
32,768 tok
Compute Roofline
COMFORTABLE FIT Memory-Bandwidth Bound
Max Safe Ctx: 128k
Memory Allocation Footprint 4.44 / 95.00 GiB usable
Weights: 2.48 GiB
KV Cache: 0.86 GiB
Overhead: 1.10 GiB
Headroom: 90.56 GiB
Decode Speed
420.3
tok/s (batch=1)
Prefill Speed
18538
prompt tok/s
TTFT Latency
0.26s
for 4,096 tok
Active Params
4.3B
of 4.3B total
llama.cpp Server Command
llama-server -hf google/gemma-3-4b-it:Q4_K_M -ngl 99 -fa on -c 32768 --port 8080| Context | KV Cache | Total VRAM | Decode | Status |
|---|---|---|---|---|
| 4k | 0.20 GiB | 3.61 GiB | 523.2 tok/s | COMFORTABLE |
| 8k | 0.30 GiB | 3.87 GiB | 505.5 tok/s | COMFORTABLE |
| 16k | 0.48 GiB | 4.06 GiB | 473.5 tok/s | COMFORTABLE |
| 32k | 0.86 GiB | 4.44 GiB | 420.3 tok/s | COMFORTABLE |
| 64k | 1.61 GiB | 5.19 GiB | 343.1 tok/s | COMFORTABLE |
| 128k | 3.11 GiB | 6.69 GiB | 251.0 tok/s | COMFORTABLE |
Buy-vs-Cloud-vs-API Monthly TCO Break-Even
USE SERVERLESS APIOwn Hardware/mo
$160.43
10.7h active + power
Cloud GPU/mo
$22.69
@$1.85/hr spot/pod
Serverless API/mo
$4.00
API cheaper
Recommended Hardware & Cloud Deployment
Contextual to fit statusEngineering FAQ — Gemma 3 4B IT
How much VRAM does Gemma 3 4B IT require in Q4_K_M?
Gemma 3 4B IT weights occupy 2.48 GiB in Q4_K_M. Including 32k FP16 KV cache and runtime buffers, total memory is 4.44 GiB.
What is the cheapest local hardware that runs Gemma 3 4B IT?
NVIDIA RTX PRO 6000 Blackwell (96 GB) (96 GB) runs Gemma 3 4B IT Q4_K_M at ~420.3 tok/s decode and 0.26s TTFT.
How does Gemma 3 4B IT scale KV cache at long context?
Gemma 3 4B IT uses hybrid sliding-window attention (1024 tokens) across most layers, keeping 32k-128k KV memory compact.