LLM Roofline
AMD · PCIE · Backend: llama.cpp

AMD Radeon RX 7900 XTX (24 GB) — LLM Sizing & Speed Matrix

960 GB/s 24 GB card with ROCm/Vulkan support; slower prefill kernels than CUDA Ada/Blackwell.

Reference Pricing
$949 Street
Usable Memory
23 GiB
Memory Bandwidth
960 GB/s (76% eff.)
FP16 Compute
122.8 TFLOPS
Roofline Ridge
87.5 FLOP/B
Power (Load / Idle)
355W / 35W

Buy or Rent AMD Radeon RX 7900 XTX (24 GB)

Vendor & Cloud Links
Radeon RX 7900 XTX 24 GB on Amazon
Amazon · Direct Hardware
↗
Radeon RX 7900 XTX on Newegg
Newegg · Direct Hardware
↗

All 32 Open-Weight Models on AMD Radeon RX 7900 XTX (24 GB) (Q4_K_M @ 32k)

Model Status Weights Total @ 32k Decode TTFT (4k) Max Safe Ctx
Gemma 3 27B IT COMFORTABLE 15.8 GiB 20.2 GiB 36.4 tok/s 3.68s 68k
Mistral Small 3.1 24B COMFORTABLE 13.8 GiB 20.3 GiB 36.1 tok/s 3.20s 49k
Qwen3 14B COMFORTABLE 8.5 GiB 15.0 GiB 50.3 tok/s 2.04s 40k
Qwen2.5 14B Instruct COMFORTABLE 8.5 GiB 16.0 GiB 46.8 tok/s 2.06s 32k
DeepSeek-R1 Distill Qwen 14B COMFORTABLE 8.5 GiB 16.0 GiB 46.8 tok/s 2.06s 69k
Phi-4 14B COMFORTABLE 8.5 GiB 13.0 GiB 58.6 tok/s 2.03s 16k
Gemma 3 12B IT COMFORTABLE 7.0 GiB 10.6 GiB 72.8 tok/s 1.70s 128k
Qwen3 8B COMFORTABLE 4.7 GiB 10.5 GiB 73.7 tok/s 1.16s 40k
Llama 3.1 8B Instruct COMFORTABLE 4.6 GiB 9.9 GiB 78.8 tok/s 1.13s 128k
Qwen2.5 7B Instruct COMFORTABLE 4.4 GiB 7.4 GiB 110.9 tok/s 1.06s 32k
Gemma 3 4B IT COMFORTABLE 2.5 GiB 4.4 GiB 203.7 tok/s 0.62s 128k
GLM-4 32B 0414 TIGHT 18.6 GiB 22.1 GiB 33.1 tok/s 4.38s 32k
Qwen3 30B-A3B TIGHT 17.6 GiB 21.6 GiB 122.0 tok/s 0.56s 40k
Codestral 22B TIGHT 12.8 GiB 21.4 GiB 34.3 tok/s 3.06s 32k
DeepSeek-R1 671B OOM 386.4 GiB 390.3 GiB OOM — OOM
DeepSeek-V3 671B OOM 386.4 GiB 390.3 GiB OOM — OOM
Llama 3.1 405B Instruct OOM 233.8 GiB 252.5 GiB OOM — OOM
Llama 4 Maverick 400B-A17B OOM 231.5 GiB 239.0 GiB OOM — OOM
Qwen3 235B-A22B OOM 135.4 GiB 142.6 GiB OOM — OOM
Mixtral 8x22B Instruct OOM 81.2 GiB 89.8 GiB OOM — OOM
Command A 111B OOM 63.9 GiB 69.1 GiB OOM — OOM
Llama 4 Scout 109B-A17B OOM 62.8 GiB 70.2 GiB OOM — OOM
Qwen2.5 72B Instruct OOM 41.9 GiB 53.7 GiB OOM — OOM
DeepSeek-R1 Distill Llama 70B OOM 40.7 GiB 52.5 GiB OOM — OOM
Llama 3.3 70B Instruct OOM 40.7 GiB 52.5 GiB OOM — OOM
Llama 3.1 70B Instruct OOM 40.7 GiB 52.5 GiB OOM — OOM
Llama-3.3 Nemotron Super 49B OOM 28.7 GiB 36.7 GiB OOM — OOM
Mixtral 8x7B Instruct OOM 26.9 GiB 32.2 GiB OOM — OOM
Qwen3 32B OOM 18.9 GiB 28.3 GiB OOM — 11k
Qwen2.5 Coder 32B OOM 18.9 GiB 28.3 GiB OOM — 11k
DeepSeek-R1 Distill Qwen 32B OOM 18.9 GiB 28.3 GiB OOM — 11k
QwQ 32B Reasoning OOM 18.9 GiB 28.3 GiB OOM — 11k

Interactive AMD Radeon RX 7900 XTX (24 GB) Simulator

27.4B Dense · GQA
960 GB/s · 122.8 TFLOPS
3. Weight Quantization 4.85 bpw
4. KV Cache
32,768 tok
Compute Roofline
COMFORTABLE FIT Memory-Bandwidth Bound
Max Safe Ctx: 68k
Memory Allocation Footprint 20.17 / 23.00 GiB usable
Weights: 15.78 GiB KV Cache: 2.91 GiB Overhead: 1.49 GiB Headroom: 2.83 GiB
Decode Speed
36.4
tok/s (batch=1)
Prefill Speed
1123
prompt tok/s
TTFT Latency
3.68s
for 4,096 tok
Active Params
27.4B
of 27.4B total
llama.cpp Server Command 
llama-server -hf google/gemma-3-27b-it:Q4_K_M -ngl 99 -fa on -c 32768 --port 8080
Context KV Cache Total VRAM Decode Status
4k 0.72 GiB 17.62 GiB 41.2 tok/s COMFORTABLE
8k 1.03 GiB 18.30 GiB 40.4 tok/s COMFORTABLE
16k 1.66 GiB 18.92 GiB 39.0 tok/s COMFORTABLE
32k 2.91 GiB 20.17 GiB 36.4 tok/s COMFORTABLE
64k 5.41 GiB 22.67 GiB 32.1 tok/s TIGHT
128k 10.41 GiB 27.67 GiB 25.9 tok/s OOM

Buy-vs-Cloud-vs-API Monthly TCO Break-Even

USE SERVERLESS API
Own Hardware/mo
$29.05
127.0h active + power
Cloud GPU/mo
$129.95
@$0.89/hr spot/pod
Serverless API/mo
$9.25
API cheaper

Recommended Hardware & Cloud Deployment

Contextual to fit status

Hardware FAQ — AMD Radeon RX 7900 XTX (24 GB)

How much usable VRAM / GPU memory does AMD Radeon RX 7900 XTX (24 GB) actually provide for LLMs?

AMD Radeon RX 7900 XTX (24 GB) provides 23 GiB usable VRAM after subtracting 1.0 GiB driver/display overhead.

What is the largest Q4_K_M model AMD Radeon RX 7900 XTX (24 GB) can run at 32k context?

With Q4_K_M weights and FP16 KV cache at 32k context, the largest fitting model in our verified catalog on AMD Radeon RX 7900 XTX (24 GB) is Gemma 3 27B IT (27.4B total).