Qwen3 · Apache-2.0 · HF: Qwen/Qwen3-235B-A22B
Qwen3 235B-A22B — VRAM, KV Cache & Hardware Matrix
Serverless API Rate
$0.14 in / $0.80 out per 1M
Parameters
235.1B (22B act.)
Attention
GQA (4 KV / 64 Q)
Layers × Hidden
94 × 4096
Max Native Context
40k tokens
MoE Experts
8 / 128
Quantization Memory Footprint
| Quantization | BPW | Weights Only | Total @ 8k Ctx | Total @ 32k Ctx | Quality |
|---|---|---|---|---|---|
| FP16 | 16.00 | 446.67 GiB | 449.25 GiB | 453.65 GiB | Lossless Reference |
| Q8_0 | 8.50 | 237.29 GiB | 239.87 GiB | 244.28 GiB | Near-Lossless |
| Q6_K | 6.57 | 183.41 GiB | 185.99 GiB | 190.40 GiB | High Precision |
| Q5_K_M | 5.69 | 158.85 GiB | 161.43 GiB | 165.83 GiB | Balanced High |
| Q4_K_M | 4.85 | 135.40 GiB | 137.98 GiB | 142.38 GiB | Recommended 4-bit |
| MLX_4BIT | 4.50 | 125.62 GiB | 128.21 GiB | 132.61 GiB | Apple Silicon Optimized |
| IQ4_XS | 4.25 | 118.65 GiB | 121.23 GiB | 125.63 GiB | Compact 4-bit |
| Q3_K_M | 3.91 | 109.15 GiB | 111.74 GiB | 116.14 GiB | Aggressive Compression |
Hardware Compatibility & Speed Leaderboard (Q4_K_M @ 32k Context)
Interactive Qwen3 235B-A22B Simulator
235.1B (22B active) · GQA
819 GB/s · 58 TFLOPS
3. Weight Quantization
4.85 bpw
4. KV Cache
32,768 tok
Compute Roofline
COMFORTABLE FIT Memory-Bandwidth Bound
Max Safe Ctx: 40k
Memory Allocation Footprint 142.38 / 199.68 GiB usable
Weights: 135.40 GiB
KV Cache: 5.88 GiB
Overhead: 1.11 GiB
Headroom: 57.30 GiB
Decode Speed
28.2
tok/s (batch=1)
Prefill Speed
669
prompt tok/s
TTFT Latency
6.16s
for 4,096 tok
Active Params
22.0B
of 235.1B total
llama.cpp Server Command
llama-server -hf Qwen/Qwen3-235B-A22B:Q4_K_M -ngl 99 -fa on -c 32768 --port 8080| Context | KV Cache | Total VRAM | Decode | Status |
|---|---|---|---|---|
| 4k | 0.73 GiB | 136.96 GiB | 39.1 tok/s | COMFORTABLE |
| 8k | 1.47 GiB | 137.98 GiB | 37.0 tok/s | COMFORTABLE |
| 16k | 2.94 GiB | 139.45 GiB | 33.5 tok/s | COMFORTABLE |
| 32k | 5.88 GiB | 142.38 GiB | 28.2 tok/s | COMFORTABLE |
Buy-vs-Cloud-vs-API Monthly TCO Break-Even
USE SERVERLESS APIOwn Hardware/mo
$109.58
168.4h active + power
Cloud GPU/mo
$675.74
@$3.49/hr spot/pod
Serverless API/mo
$19.00
Payback: 346.2 mo
Recommended Hardware & Cloud Deployment
Contextual to fit statusEngineering FAQ — Qwen3 235B-A22B
How much VRAM does Qwen3 235B-A22B require in Q4_K_M?
Qwen3 235B-A22B weights occupy 135.40 GiB in Q4_K_M. Including 32k FP16 KV cache and runtime buffers, total memory is 142.38 GiB.
What is the cheapest local hardware that runs Qwen3 235B-A22B?
Apple Mac Studio M3 Ultra (256 GB) (256 GB) runs Qwen3 235B-A22B Q4_K_M at ~28.2 tok/s decode and 6.16s TTFT.
How does Qwen3 235B-A22B scale KV cache at long context?
Qwen3 235B-A22B uses GQA with 4 KV heads across 94 layers (128 head_dim).