Cohere · CC-BY-NC-4.0 · HF: CohereLabs/c4ai-command-a-03-2025
Command A 111B — VRAM, KV Cache & Hardware Matrix
Serverless API Rate
$2.50 in / $10.00 out per 1M
Parameters
111B Dense
Attention
GQA (8 KV / 96 Q)
Layers × Hidden
64 × 12288
Max Native Context
256k tokens
MoE Experts
N/A (Dense)
Quantization Memory Footprint
| Quantization | BPW | Weights Only | Total @ 8k Ctx | Total @ 32k Ctx | Quality |
|---|---|---|---|---|---|
| FP16 | 16.00 | 210.89 GiB | 214.58 GiB | 216.08 GiB | Lossless Reference |
| Q8_0 | 8.50 | 112.03 GiB | 115.72 GiB | 117.22 GiB | Near-Lossless |
| Q6_K | 6.57 | 86.60 GiB | 90.28 GiB | 91.78 GiB | High Precision |
| Q5_K_M | 5.69 | 75.00 GiB | 78.68 GiB | 80.18 GiB | Balanced High |
| Q4_K_M | 4.85 | 63.93 GiB | 67.61 GiB | 69.11 GiB | Recommended 4-bit |
| MLX_4BIT | 4.50 | 59.31 GiB | 63.00 GiB | 64.50 GiB | Apple Silicon Optimized |
| IQ4_XS | 4.25 | 56.02 GiB | 59.70 GiB | 61.20 GiB | Compact 4-bit |
| Q3_K_M | 3.91 | 51.54 GiB | 55.22 GiB | 56.72 GiB | Aggressive Compression |
Hardware Compatibility & Speed Leaderboard (Q4_K_M @ 32k Context)
| Hardware | Status | Total / Limit | Decode | TTFT (4k) | Max Safe Ctx | Inspect |
|---|---|---|---|---|---|---|
| Cloud NVIDIA H200 SXM (141 GB) | COMFORTABLE | 69.1 / 140.4 GiB | 56.3 tok/s | 1.35s | 256k | Simulate → |
| Cloud NVIDIA H100 SXM (80 GB) | COMFORTABLE | 69.1 / 79.4 GiB | 39.3 tok/s | 1.35s | 197k | Simulate → |
| Cloud NVIDIA A100 SXM (80 GB) | COMFORTABLE | 69.1 / 79.4 GiB | 23.4 tok/s | 4.45s | 197k | Simulate → |
| NVIDIA RTX PRO 6000 Blackwell (96 GB) | COMFORTABLE | 69.1 / 95.0 GiB | 21.0 tok/s | 5.54s | 256k | Simulate → |
| Apple Mac Studio M3 Ultra (256 GB) | COMFORTABLE | 68.9 / 199.7 GiB | 8.9 tok/s | 27.85s | 256k | Simulate → |
| Apple Mac Studio M3 Ultra (512 GB) | COMFORTABLE | 68.9 / 399.4 GiB | 8.9 tok/s | 27.85s | 256k | Simulate → |
| Apple Mac Studio M2 Ultra (192 GB) | COMFORTABLE | 68.9 / 149.8 GiB | 8.7 tok/s | 30.75s | 256k | Simulate → |
| Apple Mac Studio M4 Max (128 GB) | COMFORTABLE | 68.9 / 99.8 GiB | 5.9 tok/s | 42.41s | 256k | Simulate → |
| Apple MacBook Pro M3 Max (128 GB) | COMFORTABLE | 68.9 / 99.8 GiB | 4.3 tok/s | 56.84s | 256k | Simulate → |
| Ryzen AI Max+ 395 Strix Halo (128 GB) | COMFORTABLE | 68.9 / 96.0 GiB | 2.6 tok/s | 31.54s | 256k | Simulate → |
| NVIDIA RTX 5090 (32 GB) | OOM | 69.1 / 31.0 GiB | OOM | — | OOM | Simulate → |
| NVIDIA RTX 4090 (24 GB) | OOM | 69.1 / 23.0 GiB | OOM | — | OOM | Simulate → |
| NVIDIA RTX 5080 (16 GB) | OOM | 69.1 / 15.0 GiB | OOM | — | OOM | Simulate → |
| NVIDIA RTX 3090 (24 GB Used) | OOM | 69.1 / 23.0 GiB | OOM | — | OOM | Simulate → |
| NVIDIA RTX 5070 Ti (16 GB) | OOM | 69.1 / 15.0 GiB | OOM | — | OOM | Simulate → |
| AMD Radeon RX 7900 XTX (24 GB) | OOM | 69.1 / 23.0 GiB | OOM | — | OOM | Simulate → |
| Cloud NVIDIA L40S (48 GB) | OOM | 69.1 / 47.4 GiB | OOM | — | OOM | Simulate → |
| 2x NVIDIA RTX 3090 Rig (48 GB) | OOM | 69.1 / 46.6 GiB | OOM | — | OOM | Simulate → |
| Apple Mac Studio M4 Max (64 GB) | OOM | 68.9 / 49.9 GiB | OOM | — | OOM | Simulate → |
| NVIDIA RTX 5060 Ti (16 GB) | OOM | 69.1 / 15.0 GiB | OOM | — | OOM | Simulate → |
| Apple MacBook Pro M3 Max (64 GB) | OOM | 68.9 / 49.9 GiB | OOM | — | OOM | Simulate → |
| NVIDIA RTX 3060 (12 GB) | OOM | 69.1 / 11.0 GiB | OOM | — | OOM | Simulate → |
| Apple Mac Mini M4 Pro (48 GB) | OOM | 68.9 / 37.4 GiB | OOM | — | OOM | Simulate → |
| Apple Mac Mini M4 Pro (64 GB) | OOM | 68.9 / 49.9 GiB | OOM | — | OOM | Simulate → |
| Ryzen AI Max+ 395 Strix Halo (64 GB) | OOM | 68.9 / 48.0 GiB | OOM | — | OOM | Simulate → |
| Apple Mac Mini M4 (16 GB) | OOM | 68.9 / 10.0 GiB | OOM | — | OOM | Simulate → |
| Apple Mac Mini M4 (24 GB) | OOM | 68.9 / 18.0 GiB | OOM | — | OOM | Simulate → |
| Ryzen AI 9 HX 370 Mini-PC (64 GB) | OOM | 68.9 / 48.0 GiB | OOM | — | OOM | Simulate → |
Interactive Command A 111B Simulator
111B Dense · GQA
1792 GB/s · 250 TFLOPS
3. Weight Quantization
4.85 bpw
4. KV Cache
32,768 tok
Compute Roofline
COMFORTABLE FIT Memory-Bandwidth Bound
Max Safe Ctx: 256k
Memory Allocation Footprint 69.11 / 95.00 GiB usable
Weights: 63.93 GiB
KV Cache: 2.75 GiB
Overhead: 2.44 GiB
Headroom: 25.89 GiB
Decode Speed
21.0
tok/s (batch=1)
Prefill Speed
744
prompt tok/s
TTFT Latency
5.54s
for 4,096 tok
Active Params
111.0B
of 111B total
llama.cpp Server Command
llama-server -hf CohereLabs/c4ai-command-a-03-2025:Q4_K_M -ngl 99 -fa on -c 32768 --port 8080| Context | KV Cache | Total VRAM | Decode | Status |
|---|---|---|---|---|
| 4k | 1.00 GiB | 66.52 GiB | 21.6 tok/s | COMFORTABLE |
| 8k | 1.25 GiB | 67.61 GiB | 21.5 tok/s | COMFORTABLE |
| 16k | 1.75 GiB | 68.11 GiB | 21.4 tok/s | COMFORTABLE |
| 32k | 2.75 GiB | 69.11 GiB | 21.0 tok/s | COMFORTABLE |
| 64k | 4.75 GiB | 71.11 GiB | 20.4 tok/s | COMFORTABLE |
| 128k | 8.75 GiB | 75.11 GiB | 19.3 tok/s | COMFORTABLE |
Buy-vs-Cloud-vs-API Monthly TCO Break-Even
BUY LOCALOwn Hardware/mo
$181.02
216.8h active + power
Cloud GPU/mo
$461.23
@$1.85/hr spot/pod
Serverless API/mo
$275.00
Payback: 22.3 mo
Recommended Hardware & Cloud Deployment
Contextual to fit statusEngineering FAQ — Command A 111B
How much VRAM does Command A 111B require in Q4_K_M?
Command A 111B weights occupy 63.93 GiB in Q4_K_M. Including 32k FP16 KV cache and runtime buffers, total memory is 69.11 GiB.
What is the cheapest local hardware that runs Command A 111B?
NVIDIA RTX PRO 6000 Blackwell (96 GB) (96 GB) runs Command A 111B Q4_K_M at ~21.0 tok/s decode and 5.54s TTFT.
How does Command A 111B scale KV cache at long context?
Command A 111B uses hybrid sliding-window attention (4096 tokens) across most layers, keeping 32k-128k KV memory compact.