LLM Roofline
NVIDIA · PCIE · Backend: vllm

2x NVIDIA RTX 3090 Rig (48 GB) — LLM Sizing & Speed Matrix

Classic 48 GB homelab tensor-parallel rig for 70B Q4_K_M and 32B Q8_0 at high speed.

Reference Pricing
$1,950 Street
Usable Memory
46.65 GiB
Memory Bandwidth
936 GB/s (80% eff.)
FP16 Compute
71 TFLOPS
Roofline Ridge
56.9 FLOP/B
Power (Load / Idle)
680W / 65W

Buy or Rent 2x NVIDIA RTX 3090 Rig (48 GB)

Vendor & Cloud Links
Deploy on Vast.ai Spot GPU
Vast.ai · On-Demand / Spot GPU
↗
RTX 3090 24 GB Listings on Amazon
Amazon · Direct Hardware
↗

All 32 Open-Weight Models on 2x NVIDIA RTX 3090 Rig (48 GB) (Q4_K_M @ 32k)

Model Status Weights Total @ 32k Decode TTFT (4k) Max Safe Ctx
Llama-3.3 Nemotron Super 49B COMFORTABLE 28.7 GiB 36.7 GiB 16.8 tok/s 11.84s 84k
Mixtral 8x7B Instruct COMFORTABLE 26.9 GiB 32.2 GiB 45.1 tok/s 3.11s 32k
Qwen3 32B COMFORTABLE 18.9 GiB 28.3 GiB 21.8 tok/s 8.04s 40k
Qwen2.5 Coder 32B COMFORTABLE 18.9 GiB 28.3 GiB 21.8 tok/s 7.85s 32k
DeepSeek-R1 Distill Qwen 32B COMFORTABLE 18.9 GiB 28.3 GiB 21.8 tok/s 7.85s 105k
QwQ 32B Reasoning COMFORTABLE 18.9 GiB 28.3 GiB 21.8 tok/s 7.85s 40k
GLM-4 32B 0414 COMFORTABLE 18.6 GiB 22.1 GiB 28.6 tok/s 7.78s 32k
Qwen3 30B-A3B COMFORTABLE 17.6 GiB 21.6 GiB 105.2 tok/s 0.97s 40k
Gemma 3 27B IT COMFORTABLE 15.8 GiB 20.2 GiB 31.4 tok/s 6.55s 128k
Mistral Small 3.1 24B COMFORTABLE 13.8 GiB 20.3 GiB 31.1 tok/s 5.68s 128k
Codestral 22B COMFORTABLE 12.8 GiB 21.4 GiB 29.6 tok/s 5.44s 32k
Qwen3 14B COMFORTABLE 8.5 GiB 15.0 GiB 43.3 tok/s 3.62s 40k
Qwen2.5 14B Instruct COMFORTABLE 8.5 GiB 16.0 GiB 40.3 tok/s 3.65s 32k
DeepSeek-R1 Distill Qwen 14B COMFORTABLE 8.5 GiB 16.0 GiB 40.3 tok/s 3.65s 128k
Phi-4 14B COMFORTABLE 8.5 GiB 13.0 GiB 50.5 tok/s 3.59s 16k
Gemma 3 12B IT COMFORTABLE 7.0 GiB 10.6 GiB 62.7 tok/s 3.01s 128k
Qwen3 8B COMFORTABLE 4.7 GiB 10.5 GiB 63.5 tok/s 2.05s 40k
Llama 3.1 8B Instruct COMFORTABLE 4.6 GiB 9.9 GiB 67.9 tok/s 2.00s 128k
Qwen2.5 7B Instruct COMFORTABLE 4.4 GiB 7.4 GiB 95.6 tok/s 1.87s 32k
Gemma 3 4B IT COMFORTABLE 2.5 GiB 4.4 GiB 175.6 tok/s 1.08s 128k
DeepSeek-R1 671B OOM 386.4 GiB 390.3 GiB OOM — OOM
DeepSeek-V3 671B OOM 386.4 GiB 390.3 GiB OOM — OOM
Llama 3.1 405B Instruct OOM 233.8 GiB 252.5 GiB OOM — OOM
Llama 4 Maverick 400B-A17B OOM 231.5 GiB 239.0 GiB OOM — OOM
Qwen3 235B-A22B OOM 135.4 GiB 142.6 GiB OOM — OOM
Mixtral 8x22B Instruct OOM 81.2 GiB 89.8 GiB OOM — OOM
Command A 111B OOM 63.9 GiB 69.1 GiB OOM — OOM
Llama 4 Scout 109B-A17B OOM 62.8 GiB 70.2 GiB OOM — OOM
Qwen2.5 72B Instruct OOM 41.9 GiB 53.7 GiB OOM — 9k
DeepSeek-R1 Distill Llama 70B OOM 40.7 GiB 52.5 GiB OOM — 13k
Llama 3.3 70B Instruct OOM 40.7 GiB 52.5 GiB OOM — 13k
Llama 3.1 70B Instruct OOM 40.7 GiB 52.5 GiB OOM — 13k

Interactive 2x NVIDIA RTX 3090 Rig (48 GB) Simulator

49.9B Dense · GQA
936 GB/s · 71 TFLOPS
3. Weight Quantization 4.85 bpw
4. KV Cache
32,768 tok
Compute Roofline
COMFORTABLE FIT Memory-Bandwidth Bound
Max Safe Ctx: 84k
Memory Allocation Footprint 36.74 / 46.65 GiB usable
Weights: 28.74 GiB KV Cache: 6.13 GiB Overhead: 1.88 GiB Headroom: 9.91 GiB
Decode Speed
16.8
tok/s (batch=1)
Prefill Speed
347
prompt tok/s
TTFT Latency
11.84s
for 4,096 tok
Active Params
49.9B
of 49.9B total
llama.cpp Server Command 
llama-server -hf nvidia/Llama-3_3-Nemotron-Super-49B-v1:Q4_K_M -ngl 99 -fa on -c 32768 --split-mode layer --port 8080
Context KV Cache Total VRAM Decode Status
4k 0.77 GiB 30.82 GiB 19.9 tok/s COMFORTABLE
8k 1.53 GiB 32.14 GiB 19.4 tok/s COMFORTABLE
16k 3.06 GiB 33.68 GiB 18.4 tok/s COMFORTABLE
32k 6.13 GiB 36.74 GiB 16.8 tok/s COMFORTABLE
64k 12.25 GiB 42.86 GiB 14.3 tok/s TIGHT
128k 24.50 GiB 55.11 GiB 11.0 tok/s OOM

Buy-vs-Cloud-vs-API Monthly TCO Break-Even

USE SERVERLESS API
Own Hardware/mo
$75.63
288.0h active + power
Cloud GPU/mo
$149.05
@$0.45/hr spot/pod
Serverless API/mo
$13.50
API cheaper

Recommended Hardware & Cloud Deployment

Contextual to fit status

Hardware FAQ — 2x NVIDIA RTX 3090 Rig (48 GB)

How much usable VRAM / GPU memory does 2x NVIDIA RTX 3090 Rig (48 GB) actually provide for LLMs?

2x NVIDIA RTX 3090 Rig (48 GB) provides 46.65 GiB usable VRAM after subtracting 1.0 GiB driver/display overhead.

What is the largest Q4_K_M model 2x NVIDIA RTX 3090 Rig (48 GB) can run at 32k context?

With Q4_K_M weights and FP16 KV cache at 32k context, the largest fitting model in our verified catalog on 2x NVIDIA RTX 3090 Rig (48 GB) is Llama-3.3 Nemotron Super 49B (49.9B total).