What GPU do you need to run GLM-4.7-Flash?
A new Radeon RX 7900 XTX ($1,350). It is the cheapest priced graphics card that holds GLM-4.7-Flash (about 19.8 GB at Q4) with room for about 84K tokens of context (159K with 8-bit KV cache), at about 209 tok/s estimate; that is the card price only, and it requires a separate compatible PC. 58 of the 157 GPUs and systems we track fit it.
31.22B parameters, 3B active (mixture-of-experts) (Hugging Face, accessed 2026-10-16). Q4_K_M GGUF: 18.31 GB (source). Architecture config (Hugging Face config.json): 47 attention layers caching one 576-wide compressed latent (MLA), so about 53 KB of FP16 KV cache per token of context (how we estimate context).
Model card: "a 30B-A3B MoE model".
Other places to get it
Checked Nov 14, 2026. We list only copies we opened and checked. Check file hashes for copies from other uploaders.
- ModelScope: Lab's own account
- Ollama: Runs on your machine
ollama pull glm-4.7-flash:q4_K_M
Single GPUs that fit, cheapest first
| Single GPU that fits | VRAM | Speed estimate | Max context estimate | Price |
|---|---|---|---|---|
| AMD Radeon RX 7900 XTX | 24 GB | ~209 tok/s | 84K · 159K (8-bit) | low $1,350 |
| NVIDIA GeForce RTX 3090 | 24 GB | ~261 tok/s | 84K · 159K (8-bit) | low $1,499 |
| Apple Mac Studio M5 Max (32-core GPU, 36 GB) | 36 GB (~24.1 usable) | ~106 tok/s | 86K · 162K (8-bit) | low $2,499 |
| NVIDIA GeForce RTX 4090 | 24 GB | ~271 tok/s | 84K · 159K (8-bit) | low $2,910 |
| NVIDIA DGX Spark (64GB) | 64 GB | ~111 tok/s | 198K (full) | low $4,999 |
| Apple Mac Studio M5 Ultra (64-core GPU, 96 GB) | 96 GB (~72 usable) | ~148 tok/s | 198K (full) | low $5,499 |
| NVIDIA GeForce RTX 5090 | 32 GB | ~355 tok/s | 198K (full) | low $6,500 |
| Framework Desktop, AMD Ryzen AI Max+ PRO 495 (Strix Halo), 192 GB | 192 GB | ~93 tok/s | 198K (full) | low $6,799 |
| NVIDIA DGX Spark (128GB) | 128 GB | ~111 tok/s | 198K (full) | low $6,949.99 |
| NVIDIA RTX A6000 | 48 GB | ~232 tok/s | 198K (full) | low $8,363 |
| AMD Radeon RX 7900 XT | 20 GB | ~190 tok/s | 12K · 23K (8-bit) | No price yet |
| NVIDIA RTX 4000 Ada Generation | 20 GB | ~138 tok/s | 12K · 23K (8-bit) | No price yet |
| NVIDIA RTX 4000 SFF Ada Generation | 20 GB | ~113 tok/s | 12K · 23K (8-bit) | No price yet |
| NVIDIA RTX A4500 | 20 GB | ~207 tok/s | 12K · 23K (8-bit) | No price yet |
| Intel Arc Pro B60 | 24 GB | ~112 tok/s | 84K · 159K (8-bit) | No price yet |
| NVIDIA A10 | 24 GB | ~199 tok/s | 84K · 159K (8-bit) | No price yet |
| NVIDIA GeForce RTX 3090 Ti | 24 GB | ~271 tok/s | 84K · 159K (8-bit) | No price yet |
| NVIDIA L4 | 24 GB | ~120 tok/s | 84K · 159K (8-bit) | No price yet |
| NVIDIA Quadro RTX 6000 | 24 GB | ~214 tok/s | 84K · 159K (8-bit) | No price yet |
| NVIDIA RTX 4500 Ada Generation | 24 GB | ~158 tok/s | 84K · 159K (8-bit) | No price yet |
| NVIDIA RTX A5000 | 24 GB | ~232 tok/s | 84K · 159K (8-bit) | No price yet |
| NVIDIA RTX A5500 | 24 GB | ~232 tok/s | 84K · 159K (8-bit) | No price yet |
| NVIDIA RTX PRO 4000 Blackwell | 24 GB | ~214 tok/s | 84K · 159K (8-bit) | No price yet |
| NVIDIA RTX PRO 4000 SFF Blackwell | 24 GB | ~158 tok/s | 84K · 159K (8-bit) | No price yet |
| NVIDIA Tesla P40 | 24 GB | ~105 tok/s | 84K · 159K (8-bit) | No price yet |
| NVIDIA TITAN RTX | 24 GB | ~214 tok/s | 84K · 159K (8-bit) | No price yet |
| AMD Radeon PRO V710 | 28 GB | ~133 tok/s | 156K · 198K (8-bit) | No price yet |
| AMD Radeon AI PRO R9600 | 32 GB | ~167 tok/s | 198K (full) | No price yet |
| AMD Radeon AI PRO R9600D | 32 GB | ~167 tok/s | 198K (full) | No price yet |
| AMD Radeon AI PRO R9700 | 32 GB | ~167 tok/s | 198K (full) | No price yet |
| AMD Radeon AI PRO R9700S | 32 GB | ~167 tok/s | 198K (full) | No price yet |
| AMD Radeon PRO V620 | 32 GB | ~146 tok/s | 198K (full) | No price yet |
| AMD Radeon PRO W6800 | 32 GB | ~146 tok/s | 198K (full) | No price yet |
| AMD Radeon PRO W7800 32GB | 32 GB | ~157 tok/s | 198K (full) | No price yet |
| Intel Arc Pro B70 | 32 GB | ~137 tok/s | 198K (full) | No price yet |
| NVIDIA Quadro GV100 | 32 GB | ~250 tok/s | 198K (full) | No price yet |
| NVIDIA RTX 5000 Ada Generation | 32 GB | ~193 tok/s | 198K (full) | No price yet |
| NVIDIA RTX PRO 4500 Blackwell | 32 GB | ~254 tok/s | 198K (full) | No price yet |
| NVIDIA V100 PCIe 32GB | 32 GB | ~255 tok/s | 198K (full) | No price yet |
| NVIDIA V100S PCIe 32GB | 32 GB | ~289 tok/s | 198K (full) | No price yet |
| Apple M5 Pro 48GB | 48 GB (~36 usable) | ~87 tok/s | 198K (full) | No price yet |
| NVIDIA A100 40GB PCIe | 40 GB | ~335 tok/s | 198K (full) | No price yet |
| NVIDIA A800 40GB Active | 40 GB | ~335 tok/s | 198K (full) | No price yet |
| AMD Radeon PRO W7800 48GB | 48 GB | ~198 tok/s | 198K (full) | No price yet |
| AMD Radeon PRO W7900 | 48 GB | ~198 tok/s | 198K (full) | No price yet |
| AMD Radeon PRO W7900 Dual Slot | 48 GB | ~198 tok/s | 198K (full) | No price yet |
| Apple Mac Studio M5 Max (40-core GPU, 64 GB) | 64 GB (~48 usable) | ~120 tok/s | 198K (full) | No price yet |
| NVIDIA A40 | 48 GB | ~219 tok/s | 198K (full) | No price yet |
| NVIDIA L40 | 48 GB | ~249 tok/s | 198K (full) | No price yet |
| NVIDIA L40S | 48 GB | ~249 tok/s | 198K (full) | No price yet |
| NVIDIA Quadro RTX 8000 | 48 GB | ~214 tok/s | 198K (full) | No price yet |
| NVIDIA RTX 6000 Ada Generation | 48 GB | ~264 tok/s | 198K (full) | No price yet |
| NVIDIA RTX PRO 5000 Blackwell 48GB | 48 GB | ~314 tok/s | 198K (full) | No price yet |
| NVIDIA RTX PRO 5000 Blackwell 72GB | 72 GB | ~314 tok/s | 198K (full) | No price yet |
| NVIDIA A100 80GB PCIe | 80 GB | ~366 tok/s | 198K (full) | No price yet |
| NVIDIA RTX PRO 6000 Blackwell Max-Q | 96 GB | ~355 tok/s | 198K (full) | No price yet |
| NVIDIA RTX PRO 6000 Blackwell Server Edition | 96 GB | ~340 tok/s | 198K (full) | No price yet |
| NVIDIA RTX PRO 6000 Blackwell Workstation Edition | 96 GB | ~355 tok/s | 198K (full) | No price yet |
Every device that runs it
Graphics cards, Macs and mini PCs from the wider hardware list behind My rig, smallest usable memory first. Not sure about yours? Check your exact rig.
Show all 108 devices that run it
| Device | Type | Memory | Speed estimate | Max context estimate | Price |
|---|---|---|---|---|---|
| AMD Radeon RX 7900 XT | GPU | 20 GB | ~190 tok/s | 12K · 23K (8-bit) | No price yet |
| NVIDIA RTX 4000 Ada Generation | Workstation GPU | 20 GB | ~138 tok/s | 12K · 23K (8-bit) | No price yet |
| NVIDIA RTX 4000 SFF Ada Generation | Workstation GPU | 20 GB | ~113 tok/s | 12K · 23K (8-bit) | No price yet |
| NVIDIA RTX A4500 | Workstation GPU | 20 GB | ~207 tok/s | 12K · 23K (8-bit) | No price yet |
| Apple M1 Max 32GB | Mac | 32 GB (~21.4 usable) | ~100 tok/s | 37K · 70K (8-bit) | No price yet |
| Apple M1 Pro 32GB | Mac | 32 GB (~21.4 usable) | ~67 tok/s | 37K · 70K (8-bit) | No price yet |
| Apple M2 Max (30-core GPU) 32GB | Mac | 32 GB (~21.4 usable) | ~100 tok/s | 37K · 70K (8-bit) | No price yet |
| Apple M2 Max (38-core GPU) 32GB | Mac | 32 GB (~21.4 usable) | ~100 tok/s | 37K · 70K (8-bit) | No price yet |
| Apple M2 Pro 32GB | Mac | 32 GB (~21.4 usable) | ~67 tok/s | 37K · 70K (8-bit) | No price yet |
| Apple M4 32GB | Mac | 32 GB (~21.4 usable) | ~46 tok/s | 37K · 70K (8-bit) | No price yet |
| Apple M5 32GB | Mac | 32 GB (~21.4 usable) | ~55 tok/s | 37K · 70K (8-bit) | No price yet |
| Apple M6 32GB | Mac | 32 GB (~21.4 usable) | ~60 tok/s | 37K · 70K (8-bit) | No price yet |
| AMD Radeon RX 7900 XTX | GPU | 24 GB | ~209 tok/s | 84K · 159K (8-bit) | low $1,350 |
| Intel Arc Pro B60 | Workstation GPU | 24 GB | ~112 tok/s | 84K · 159K (8-bit) | No price yet |
| NVIDIA A10 | Data center GPU | 24 GB | ~199 tok/s | 84K · 159K (8-bit) | No price yet |
| NVIDIA GeForce RTX 3090 | GPU | 24 GB | ~261 tok/s | 84K · 159K (8-bit) | low $1,499 |
| NVIDIA GeForce RTX 3090 Ti | GPU | 24 GB | ~271 tok/s | 84K · 159K (8-bit) | No price yet |
| NVIDIA GeForce RTX 4090 | GPU | 24 GB | ~271 tok/s | 84K · 159K (8-bit) | low $2,910 |
| NVIDIA L4 | Data center GPU | 24 GB | ~120 tok/s | 84K · 159K (8-bit) | No price yet |
| NVIDIA Quadro RTX 6000 | Workstation GPU | 24 GB | ~214 tok/s | 84K · 159K (8-bit) | No price yet |
| NVIDIA RTX 4500 Ada Generation | Workstation GPU | 24 GB | ~158 tok/s | 84K · 159K (8-bit) | No price yet |
| NVIDIA RTX A5000 | Workstation GPU | 24 GB | ~232 tok/s | 84K · 159K (8-bit) | No price yet |
| NVIDIA RTX A5500 | Workstation GPU | 24 GB | ~232 tok/s | 84K · 159K (8-bit) | No price yet |
| NVIDIA RTX PRO 4000 Blackwell | Workstation GPU | 24 GB | ~214 tok/s | 84K · 159K (8-bit) | No price yet |
| NVIDIA RTX PRO 4000 SFF Blackwell | Workstation GPU | 24 GB | ~158 tok/s | 84K · 159K (8-bit) | No price yet |
| NVIDIA RTX Spark N1X laptop (6144-core GPU, 24GB) | Laptop | 24 GB | ~120 tok/s | 84K · 159K (8-bit) | No price yet |
| NVIDIA Tesla P40 | Data center GPU | 24 GB | ~105 tok/s | 84K · 159K (8-bit) | No price yet |
| NVIDIA TITAN RTX | GPU | 24 GB | ~214 tok/s | 84K · 159K (8-bit) | No price yet |
| Apple M3 Max (30-core GPU) 36GB | Mac | 36 GB (~24.1 usable) | ~86 tok/s | 86K · 162K (8-bit) | No price yet |
| Apple M3 Pro 36GB | Mac | 36 GB (~24.1 usable) | ~55 tok/s | 86K · 162K (8-bit) | No price yet |
| Apple M4 Max (32-core GPU) 36GB | Mac | 36 GB (~24.1 usable) | ~101 tok/s | 86K · 162K (8-bit) | No price yet |
| Apple Mac Studio M5 Max (32-core GPU, 36 GB) | Mac | 36 GB (~24.1 usable) | ~106 tok/s | 86K · 162K (8-bit) | low $2,499 |
| AMD Radeon PRO V710 | Data center GPU | 28 GB | ~133 tok/s | 156K · 198K (8-bit) | No price yet |
| AMD Radeon AI PRO R9600 | Workstation GPU | 32 GB | ~167 tok/s | 198K (full) | No price yet |
| AMD Radeon AI PRO R9600D | Workstation GPU | 32 GB | ~167 tok/s | 198K (full) | No price yet |
| AMD Radeon AI PRO R9700 | Workstation GPU | 32 GB | ~167 tok/s | 198K (full) | No price yet |
| AMD Radeon AI PRO R9700S | Workstation GPU | 32 GB | ~167 tok/s | 198K (full) | No price yet |
| AMD Radeon PRO V620 | Data center GPU | 32 GB | ~146 tok/s | 198K (full) | No price yet |
| AMD Radeon PRO W6800 | Workstation GPU | 32 GB | ~146 tok/s | 198K (full) | No price yet |
| AMD Radeon PRO W7800 32GB | Workstation GPU | 32 GB | ~157 tok/s | 198K (full) | No price yet |
| AMD Ryzen AI Max 385 (32GB) | Mini PC | 32 GB | ~88 tok/s | 198K (full) | No price yet |
| AMD Ryzen AI Max 390 (32GB) | Mini PC | 32 GB | ~88 tok/s | 198K (full) | No price yet |
| AMD Ryzen AI Max+ 395 (32GB) | Mini PC | 32 GB | ~88 tok/s | 198K (full) | No price yet |
| Intel Arc Pro B70 | Workstation GPU | 32 GB | ~137 tok/s | 198K (full) | No price yet |
| NVIDIA GeForce RTX 5090 | GPU | 32 GB | ~355 tok/s | 198K (full) | low $6,500 |
| NVIDIA Quadro GV100 | Workstation GPU | 32 GB | ~250 tok/s | 198K (full) | No price yet |
| NVIDIA RTX 5000 Ada Generation | Workstation GPU | 32 GB | ~193 tok/s | 198K (full) | No price yet |
| NVIDIA RTX PRO 4500 Blackwell | Workstation GPU | 32 GB | ~254 tok/s | 198K (full) | No price yet |
| NVIDIA RTX Spark N1X laptop (6144-core GPU, 32GB) | Laptop | 32 GB | ~120 tok/s | 198K (full) | No price yet |
| NVIDIA V100 PCIe 32GB | Data center GPU | 32 GB | ~255 tok/s | 198K (full) | No price yet |
| NVIDIA V100S PCIe 32GB | Data center GPU | 32 GB | ~289 tok/s | 198K (full) | No price yet |
| Apple M3 Max (40-core GPU) 48GB | Mac | 48 GB (~36 usable) | ~100 tok/s | 198K (full) | No price yet |
| Apple M4 Max (40-core GPU) 48GB | Mac | 48 GB (~36 usable) | ~115 tok/s | 198K (full) | No price yet |
| Apple M4 Pro 48GB | Mac | 48 GB (~36 usable) | ~81 tok/s | 198K (full) | No price yet |
| Apple M5 Max (40-core GPU) 48GB | Mac | 48 GB (~36 usable) | ~120 tok/s | 198K (full) | No price yet |
| Apple M5 Pro 48GB | Mac | 48 GB (~36 usable) | ~87 tok/s | 198K (full) | No price yet |
| NVIDIA A100 40GB PCIe | Data center GPU | 40 GB | ~335 tok/s | 198K (full) | No price yet |
| NVIDIA A800 40GB Active | Workstation GPU | 40 GB | ~335 tok/s | 198K (full) | No price yet |
| AMD Radeon PRO W7800 48GB | Workstation GPU | 48 GB | ~198 tok/s | 198K (full) | No price yet |
| AMD Radeon PRO W7900 | Workstation GPU | 48 GB | ~198 tok/s | 198K (full) | No price yet |
| AMD Radeon PRO W7900 Dual Slot | Workstation GPU | 48 GB | ~198 tok/s | 198K (full) | No price yet |
| Apple M1 Max 64GB | Mac | 64 GB (~48 usable) | ~100 tok/s | 198K (full) | No price yet |
| Apple M1 Ultra 64GB | Mac | 64 GB (~48 usable) | ~132 tok/s | 198K (full) | No price yet |
| Apple M2 Max (30-core GPU) 64GB | Mac | 64 GB (~48 usable) | ~100 tok/s | 198K (full) | No price yet |
| Apple M2 Max (38-core GPU) 64GB | Mac | 64 GB (~48 usable) | ~100 tok/s | 198K (full) | No price yet |
| Apple M2 Ultra 64GB | Mac | 64 GB (~48 usable) | ~132 tok/s | 198K (full) | No price yet |
| Apple M3 Max (40-core GPU) 64GB | Mac | 64 GB (~48 usable) | ~100 tok/s | 198K (full) | No price yet |
| Apple M4 Max (40-core GPU) 64GB | Mac | 64 GB (~48 usable) | ~115 tok/s | 198K (full) | No price yet |
| Apple M4 Pro 64GB | Mac | 64 GB (~48 usable) | ~81 tok/s | 198K (full) | No price yet |
| Apple M5 Pro 64GB | Mac | 64 GB (~48 usable) | ~87 tok/s | 198K (full) | No price yet |
| Apple Mac Studio M5 Max (40-core GPU, 64 GB) | Mac | 64 GB (~48 usable) | ~120 tok/s | 198K (full) | No price yet |
| NVIDIA A40 | Data center GPU | 48 GB | ~219 tok/s | 198K (full) | No price yet |
| NVIDIA L40 | Data center GPU | 48 GB | ~249 tok/s | 198K (full) | No price yet |
| NVIDIA L40S | Data center GPU | 48 GB | ~249 tok/s | 198K (full) | No price yet |
| NVIDIA Quadro RTX 8000 | Workstation GPU | 48 GB | ~214 tok/s | 198K (full) | No price yet |
| NVIDIA RTX 6000 Ada Generation | Workstation GPU | 48 GB | ~264 tok/s | 198K (full) | No price yet |
| NVIDIA RTX A6000 | Workstation GPU | 48 GB | ~232 tok/s | 198K (full) | low $8,363 |
| NVIDIA RTX PRO 5000 Blackwell 48GB | Workstation GPU | 48 GB | ~314 tok/s | 198K (full) | No price yet |
| NVIDIA RTX Spark N1X laptop (6144-core GPU, 48GB) | Laptop | 48 GB | ~120 tok/s | 198K (full) | No price yet |
| AMD Ryzen AI Max 385 (64GB) | Mini PC | 64 GB | ~88 tok/s | 198K (full) | No price yet |
| AMD Ryzen AI Max 390 (64GB) | Mini PC | 64 GB | ~88 tok/s | 198K (full) | No price yet |
| AMD Ryzen AI Max+ 395 (64GB) | Mini PC | 64 GB | ~88 tok/s | 198K (full) | low $2,199.99 |
| NVIDIA DGX Spark (64GB) | Mini PC | 64 GB | ~111 tok/s | 198K (full) | low $4,999 |
| NVIDIA RTX Spark N1X laptop (6144-core GPU, 64GB) | Laptop | 64 GB | ~120 tok/s | 198K (full) | No price yet |
| Apple M2 Max (38-core GPU) 96GB | Mac | 96 GB (~72 usable) | ~100 tok/s | 198K (full) | No price yet |
| Apple M3 Max (30-core GPU) 96GB | Mac | 96 GB (~72 usable) | ~86 tok/s | 198K (full) | No price yet |
| Apple M3 Ultra 96GB | Mac | 96 GB (~72 usable) | ~133 tok/s | 198K (full) | No price yet |
| Apple Mac Studio M5 Ultra (64-core GPU, 96 GB) | Mac | 96 GB (~72 usable) | ~148 tok/s | 198K (full) | low $5,499 |
| NVIDIA RTX PRO 5000 Blackwell 72GB | Workstation GPU | 72 GB | ~314 tok/s | 198K (full) | No price yet |
| NVIDIA A100 80GB PCIe | Data center GPU | 80 GB | ~366 tok/s | 198K (full) | No price yet |
| Apple M1 Ultra 128GB | Mac | 128 GB (~96 usable) | ~132 tok/s | 198K (full) | No price yet |
| Apple M2 Ultra 128GB | Mac | 128 GB (~96 usable) | ~132 tok/s | 198K (full) | No price yet |
| Apple M3 Max (40-core GPU) 128GB | Mac | 128 GB (~96 usable) | ~100 tok/s | 198K (full) | No price yet |
| Apple M4 Max (40-core GPU) 128GB | Mac | 128 GB (~96 usable) | ~115 tok/s | 198K (full) | No price yet |
| Apple M5 Max (40-core GPU) 128GB | Mac | 128 GB (~96 usable) | ~120 tok/s | 198K (full) | No price yet |
| NVIDIA RTX PRO 6000 Blackwell Max-Q | Workstation GPU | 96 GB | ~355 tok/s | 198K (full) | No price yet |
| NVIDIA RTX PRO 6000 Blackwell Server Edition | Data center GPU | 96 GB | ~340 tok/s | 198K (full) | No price yet |
| NVIDIA RTX PRO 6000 Blackwell Workstation Edition | Workstation GPU | 96 GB | ~355 tok/s | 198K (full) | No price yet |
| AMD Ryzen AI Max 385 (128GB) | Mini PC | 128 GB | ~88 tok/s | 198K (full) | No price yet |
| AMD Ryzen AI Max 390 (128GB) | Mini PC | 128 GB | ~88 tok/s | 198K (full) | No price yet |
| AMD Ryzen AI Max+ 395 (128GB) | Mini PC | 128 GB | ~88 tok/s | 198K (full) | low $3,499.99 |
| NVIDIA DGX Spark (128GB) | Mini PC | 128 GB | ~111 tok/s | 198K (full) | low $6,949.99 |
| NVIDIA RTX Spark N1X laptop (6144-core GPU, 128GB) | Laptop | 128 GB | ~120 tok/s | 198K (full) | No price yet |
| Apple M2 Ultra 192GB | Mac | 192 GB (~144 usable) | ~132 tok/s | 198K (full) | No price yet |
| Apple M3 Ultra 256GB | Mac | 256 GB (~192 usable) | ~133 tok/s | 198K (full) | No price yet |
| Apple M5 Ultra 256GB | Mac | 256 GB (~192 usable) | ~148 tok/s | 198K (full) | No price yet |
| Framework Desktop, AMD Ryzen AI Max+ PRO 495 (Strix Halo), 192 GB | Mini PC | 192 GB | ~93 tok/s | 198K (full) | low $6,799 |
| Apple M5 Ultra 512GB | Mac | 512 GB (~384 usable) | ~148 tok/s | 198K (full) | No price yet |
Multi-GPU: cards that need two or more
| Cards | Total VRAM | Speed estimate | Max context estimate |
|---|---|---|---|
| 2 × Intel Arc B580 | 24 GB | ~112 tok/s | 84K · 159K (8-bit) |
| 2 × NVIDIA GeForce RTX 3060 12 GB | 24 GB | ~138 tok/s | 84K · 159K (8-bit) |
| 2 × NVIDIA GeForce RTX 4070 SUPER | 24 GB | ~176 tok/s | 84K · 159K (8-bit) |
| 3 × NVIDIA GeForce RTX 5060 | 24 GB | ~162 tok/s | 84K · 159K (8-bit) |
| 3 × NVIDIA GeForce RTX 5060 Ti 8 GB | 24 GB | ~162 tok/s | 84K · 159K (8-bit) |
| 2 × NVIDIA GeForce RTX 5070 | 24 GB | ~214 tok/s | 84K · 159K (8-bit) |
| 2 × AMD Radeon RX 7800 XT | 32 GB | ~165 tok/s | 198K (full) |
| 2 × AMD Radeon RX 9060 XT 16 GB | 32 GB | ~105 tok/s | 198K (full) |
| 2 × AMD Radeon RX 9070 | 32 GB | ~167 tok/s | 198K (full) |
| 2 × AMD Radeon RX 9070 XT | 32 GB | ~167 tok/s | 198K (full) |
| 2 × NVIDIA GeForce RTX 5060 Ti 16 GB | 32 GB | ~162 tok/s | 198K (full) |
| 2 × NVIDIA GeForce RTX 5070 Ti | 32 GB | ~254 tok/s | 198K (full) |
| 2 × NVIDIA GeForce RTX 5080 | 32 GB | ~264 tok/s | 198K (full) |
Speed calculator
estimateEstimate = bandwidth ÷ GB read per token (the model's listed GGUF file: Q4_K_M for most models, otherwise the quant named in the list; scaled by active/total parameters for MoE), scaled by a per-platform efficiency and fixed per-token costs calibrated to 45 measured llama.cpp benchmarks (2025–26). Fit = file size + 1.5 GB headroom ≤ total VRAM (on Macs, the part of unified memory macOS lets the GPU use by default), and room for at least 4K tokens of context with an 8-bit KV cache. For a model with no KV cache data, only the file and headroom are checked, so a fit says nothing about context. Multi-GPU: VRAM adds, speed stays one card's (a layer split runs the cards in turn); max context uses the total. Speed is for short context. Max context = what is left after the file and a 1 GB runtime reserve ÷ KV cache bytes per token, default FP16 and 8-bit (q8_0) cache. How speed works · how context works