What GPU do you need to run MiniCPM-V 4.7 1B?
A used GeForce RTX 3060 12 GB ($305). It is the cheapest priced graphics card that holds MiniCPM-V 4.7 1B (about 2.2 GB at Q4) with room for its full 256K context, at about 175 tok/s estimate; that is the card price only, and it requires a separate compatible PC. 157 of the 157 GPUs and systems we track fit it.
1.55B parameters (Hugging Face, accessed 2026-10-19). Q4_K_M GGUF: 0.67 GB (source). Architecture config (Hugging Face config.json): 6 attention layers (the rest are recurrent) × 2 KV heads × 256 head dim, so about 12 KB of FP16 KV cache per token of context (how we estimate context).
OpenBMB on-device vision-language model (dense 1B language backbone on Qwen3.5 text, LLaVA-UHD v4 early-compression vision). Card positions it as the phone/tablet sibling of MiniCPM-V 4.7 35B-A3B. Fit uses the language-model GGUF only; the vision projector is extra when you send images. Release date is the Hugging Face repo creation date.
Other places to get it
Checked Nov 14, 2026. We list only copies we opened and checked. Check file hashes for copies from other uploaders.
- ModelScope: Lab's own account
Single GPUs that fit, cheapest first
| Single GPU that fits | VRAM | Speed estimate | Max context estimate | Price |
|---|---|---|---|---|
| NVIDIA GeForce RTX 3060 12 GB | 12 GB | ~175 tok/s | 256K (full) | low $305 |
| Intel Arc B580 | 12 GB | ~204 tok/s | 256K (full) | low $330 |
| NVIDIA GeForce RTX 5060 | 8 GB | ~189 tok/s | 256K (full) | low $410 |
| NVIDIA GeForce RTX 5060 Ti 8 GB | 8 GB | ~189 tok/s | 256K (full) | low $450 |
| AMD Radeon RX 9070 | 16 GB | ~266 tok/s | 256K (full) | low $570 |
| AMD Radeon RX 7800 XT | 16 GB | ~264 tok/s | 256K (full) | low $599 |
| NVIDIA GeForce RTX 4070 SUPER | 12 GB | ~196 tok/s | 256K (full) | low $620 |
| AMD Radeon RX 9060 XT 16 GB | 16 GB | ~195 tok/s | 256K (full) | low $620 |
| NVIDIA GeForce RTX 5060 Ti 16 GB | 16 GB | ~189 tok/s | 256K (full) | low $780 |
| NVIDIA GeForce RTX 5070 | 12 GB | ~211 tok/s | 256K (full) | low $800 |
| AMD Radeon RX 9070 XT | 16 GB | ~266 tok/s | 256K (full) | low $810 |
| NVIDIA GeForce RTX 5070 Ti | 16 GB | ~225 tok/s | 256K (full) | low $1,120 |
| AMD Radeon RX 7900 XTX | 24 GB | ~302 tok/s | 256K (full) | low $1,350 |
| NVIDIA GeForce RTX 3090 | 24 GB | ~227 tok/s | 256K (full) | low $1,499 |
| NVIDIA GeForce RTX 5080 | 16 GB | ~228 tok/s | 256K (full) | low $1,600 |
| Apple Mac Studio M5 Max (32-core GPU, 36 GB) | 36 GB (~24.1 usable) | ~168 tok/s | 256K (full) | low $2,499 |
| NVIDIA GeForce RTX 4090 | 24 GB | ~230 tok/s | 256K (full) | low $2,910 |
| NVIDIA DGX Spark (64GB) | 64 GB | ~157 tok/s | 256K (full) | low $4,999 |
| Apple Mac Studio M5 Ultra (64-core GPU, 96 GB) | 96 GB (~72 usable) | ~203 tok/s | 256K (full) | low $5,499 |
| NVIDIA GeForce RTX 5090 | 32 GB | ~248 tok/s | 256K (full) | low $6,500 |
| Framework Desktop, AMD Ryzen AI Max+ PRO 495 (Strix Halo), 192 GB | 192 GB | ~179 tok/s | 256K (full) | low $6,799 |
| NVIDIA DGX Spark (128GB) | 128 GB | ~157 tok/s | 256K (full) | low $6,949.99 |
| NVIDIA RTX A6000 | 48 GB | ~218 tok/s | 256K (full) | low $8,363 |
| AMD Radeon PRO W6400 | 4 GB | ~109 tok/s | 185K · 256K (8-bit) | No price yet |
| AMD Radeon RX 6400 | 4 GB | ~109 tok/s | 185K · 256K (8-bit) | No price yet |
| AMD Radeon RX 6500 XT 4GB | 4 GB | ~118 tok/s | 185K · 256K (8-bit) | No price yet |
| AMD Radeon RX 9050 4GB | 4 GB | ~118 tok/s | 185K · 256K (8-bit) | No price yet |
| Intel Arc A310 | 4 GB | ~57 tok/s | 185K · 256K (8-bit) | No price yet |
| NVIDIA GeForce GTX 1050 Ti | 4 GB | ~78 tok/s | 185K · 256K (8-bit) | No price yet |
| NVIDIA GeForce GTX 1650 (GDDR5) | 4 GB | ~105 tok/s | 185K · 256K (8-bit) | No price yet |
| NVIDIA GeForce GTX 1650 (GDDR6) | 4 GB | ~132 tok/s | 185K · 256K (8-bit) | No price yet |
| NVIDIA RTX A400 | 4 GB | ~87 tok/s | 185K · 256K (8-bit) | No price yet |
| Intel Arc A380 | 6 GB | ~81 tok/s | 256K (full) | No price yet |
| Intel Arc Pro A40 | 6 GB | ~83 tok/s | 256K (full) | No price yet |
| Intel Arc Pro A50 | 6 GB | ~83 tok/s | 256K (full) | No price yet |
| NVIDIA GeForce GTX 1060 6GB | 6 GB | ~112 tok/s | 256K (full) | No price yet |
| NVIDIA GeForce GTX 1660 | 6 GB | ~132 tok/s | 256K (full) | No price yet |
| NVIDIA GeForce GTX 1660 SUPER | 6 GB | ~170 tok/s | 256K (full) | No price yet |
| NVIDIA GeForce GTX 1660 Ti | 6 GB | ~160 tok/s | 256K (full) | No price yet |
| NVIDIA GeForce RTX 2060 6GB | 6 GB | ~170 tok/s | 256K (full) | No price yet |
| NVIDIA RTX A2000 6GB | 6 GB | ~160 tok/s | 256K (full) | No price yet |
| AMD Radeon PRO W6600 | 8 GB | ~159 tok/s | 256K (full) | No price yet |
| AMD Radeon PRO W7400 | 8 GB | ~134 tok/s | 256K (full) | No price yet |
| AMD Radeon PRO W7500 | 8 GB | ~172 tok/s | 256K (full) | No price yet |
| AMD Radeon PRO W7600 | 8 GB | ~184 tok/s | 256K (full) | No price yet |
| AMD Radeon RX 6500 XT 8GB | 8 GB | ~118 tok/s | 256K (full) | No price yet |
| AMD Radeon RX 6600 | 8 GB | ~159 tok/s | 256K (full) | No price yet |
| AMD Radeon RX 6600 XT | 8 GB | ~172 tok/s | 256K (full) | No price yet |
| AMD Radeon RX 6650 XT | 8 GB | ~182 tok/s | 256K (full) | No price yet |
| AMD Radeon RX 7400 | 8 GB | ~135 tok/s | 256K (full) | No price yet |
| AMD Radeon RX 7600 | 8 GB | ~184 tok/s | 256K (full) | No price yet |
| AMD Radeon RX 9050 | 8 GB | ~184 tok/s | 256K (full) | No price yet |
| AMD Radeon RX 9060 | 8 GB | ~184 tok/s | 256K (full) | No price yet |
| AMD Radeon RX 9060 XT 8GB | 8 GB | ~195 tok/s | 256K (full) | No price yet |
| Intel Arc A580 | 8 GB | ~166 tok/s | 256K (full) | No price yet |
| Intel Arc A750 | 8 GB | ~166 tok/s | 256K (full) | No price yet |
| Intel Arc A770 8GB | 8 GB | ~166 tok/s | 256K (full) | No price yet |
| NVIDIA GeForce GTX 1070 | 8 GB | ~131 tok/s | 256K (full) | No price yet |
| NVIDIA GeForce GTX 1070 Ti | 8 GB | ~131 tok/s | 256K (full) | No price yet |
| NVIDIA GeForce GTX 1080 | 8 GB | ~147 tok/s | 256K (full) | No price yet |
| NVIDIA GeForce RTX 2060 SUPER | 8 GB | ~189 tok/s | 256K (full) | No price yet |
| NVIDIA GeForce RTX 2070 | 8 GB | ~189 tok/s | 256K (full) | No price yet |
| NVIDIA GeForce RTX 2070 SUPER | 8 GB | ~189 tok/s | 256K (full) | No price yet |
| NVIDIA GeForce RTX 2080 | 8 GB | ~189 tok/s | 256K (full) | No price yet |
| NVIDIA GeForce RTX 2080 SUPER | 8 GB | ~195 tok/s | 256K (full) | No price yet |
| NVIDIA GeForce RTX 3070 | 8 GB | ~189 tok/s | 256K (full) | No price yet |
| NVIDIA GeForce RTX 3070 Ti | 8 GB | ~206 tok/s | 256K (full) | No price yet |
| NVIDIA GeForce RTX 4060 | 8 GB | ~156 tok/s | 256K (full) | No price yet |
| NVIDIA GeForce RTX 4060 Ti 8GB | 8 GB | ~160 tok/s | 256K (full) | No price yet |
| NVIDIA GeForce RTX 5050 | 8 GB | ~167 tok/s | 256K (full) | No price yet |
| NVIDIA RTX A1000 | 8 GB | ~132 tok/s | 256K (full) | No price yet |
| NVIDIA T1000 8GB | 8 GB | ~120 tok/s | 256K (full) | No price yet |
| AMD Radeon RX 6700 | 10 GB | ~195 tok/s | 256K (full) | No price yet |
| Intel Arc B570 | 10 GB | ~186 tok/s | 256K (full) | No price yet |
| NVIDIA GeForce RTX 3080 | 10 GB | ~217 tok/s | 256K (full) | No price yet |
| NVIDIA GeForce GTX 1080 Ti | 11 GB | ~175 tok/s | 256K (full) | No price yet |
| NVIDIA GeForce RTX 2080 Ti | 11 GB | ~207 tok/s | 256K (full) | No price yet |
| AMD Radeon RX 6700 XT | 12 GB | ~214 tok/s | 256K (full) | No price yet |
| AMD Radeon RX 6750 XT | 12 GB | ~227 tok/s | 256K (full) | No price yet |
| AMD Radeon RX 7700 XT | 12 GB | ~227 tok/s | 256K (full) | No price yet |
| AMD Radeon RX 9070 GRE | 12 GB | ~227 tok/s | 256K (full) | No price yet |
| Intel Arc Pro A60 | 12 GB | ~138 tok/s | 256K (full) | No price yet |
| NVIDIA GeForce RTX 3080 Ti | 12 GB | ~225 tok/s | 256K (full) | No price yet |
| NVIDIA GeForce RTX 4070 | 12 GB | ~196 tok/s | 256K (full) | No price yet |
| NVIDIA GeForce RTX 4070 Ti | 12 GB | ~196 tok/s | 256K (full) | No price yet |
| NVIDIA RTX A2000 12GB | 12 GB | ~160 tok/s | 256K (full) | No price yet |
| NVIDIA Tesla P100 PCIe 12GB | 12 GB | ~183 tok/s | 256K (full) | No price yet |
| NVIDIA TITAN Xp | 12 GB | ~182 tok/s | 256K (full) | No price yet |
| AMD Radeon PRO W7700 | 16 GB | ~256 tok/s | 256K (full) | No price yet |
| AMD Radeon RX 6800 | 16 GB | ~244 tok/s | 256K (full) | No price yet |
| AMD Radeon RX 6800 XT | 16 GB | ~244 tok/s | 256K (full) | No price yet |
| AMD Radeon RX 6900 XT | 16 GB | ~244 tok/s | 256K (full) | No price yet |
| AMD Radeon RX 6950 XT | 16 GB | ~256 tok/s | 256K (full) | No price yet |
| AMD Radeon RX 7600 XT | 16 GB | ~184 tok/s | 256K (full) | No price yet |
| AMD Radeon RX 7700 | 16 GB | ~264 tok/s | 256K (full) | No price yet |
| AMD Radeon RX 7900 GRE | 16 GB | ~256 tok/s | 256K (full) | No price yet |
| AMD Radeon RX 9060 XT LP | 16 GB | ~195 tok/s | 256K (full) | No price yet |
| Intel Arc A770 16GB | 16 GB | ~175 tok/s | 256K (full) | No price yet |
| Intel Arc Pro B50 | 16 GB | ~134 tok/s | 256K (full) | No price yet |
| NVIDIA GeForce RTX 4060 Ti 16 GB | 16 GB | ~160 tok/s | 256K (full) | No price yet |
| NVIDIA GeForce RTX 4070 Ti SUPER | 16 GB | ~211 tok/s | 256K (full) | No price yet |
| NVIDIA GeForce RTX 4080 | 16 GB | ~214 tok/s | 256K (full) | No price yet |
| NVIDIA GeForce RTX 4080 SUPER | 16 GB | ~216 tok/s | 256K (full) | No price yet |
| NVIDIA RTX 2000 Ada Generation | 16 GB | ~143 tok/s | 256K (full) | No price yet |
| NVIDIA RTX A4000 | 16 GB | ~189 tok/s | 256K (full) | No price yet |
| NVIDIA RTX PRO 2000 Blackwell | 16 GB | ~160 tok/s | 256K (full) | No price yet |
| NVIDIA T4 | 16 GB | ~163 tok/s | 256K (full) | No price yet |
| NVIDIA Tesla P100 PCIe 16GB | 16 GB | ~200 tok/s | 256K (full) | No price yet |
| NVIDIA V100 PCIe 16GB | 16 GB | ~225 tok/s | 256K (full) | No price yet |
| AMD Radeon RX 7900 XT | 20 GB | ~287 tok/s | 256K (full) | No price yet |
| NVIDIA RTX 4000 Ada Generation | 20 GB | ~175 tok/s | 256K (full) | No price yet |
| NVIDIA RTX 4000 SFF Ada Generation | 20 GB | ~158 tok/s | 256K (full) | No price yet |
| NVIDIA RTX A4500 | 20 GB | ~209 tok/s | 256K (full) | No price yet |
| Intel Arc Pro B60 | 24 GB | ~204 tok/s | 256K (full) | No price yet |
| NVIDIA A10 | 24 GB | ~205 tok/s | 256K (full) | No price yet |
| NVIDIA GeForce RTX 3090 Ti | 24 GB | ~230 tok/s | 256K (full) | No price yet |
| NVIDIA L4 | 24 GB | ~163 tok/s | 256K (full) | No price yet |
| NVIDIA Quadro RTX 6000 | 24 GB | ~211 tok/s | 256K (full) | No price yet |
| NVIDIA RTX 4500 Ada Generation | 24 GB | ~186 tok/s | 256K (full) | No price yet |
| NVIDIA RTX A5000 | 24 GB | ~218 tok/s | 256K (full) | No price yet |
| NVIDIA RTX A5500 | 24 GB | ~218 tok/s | 256K (full) | No price yet |
| NVIDIA RTX PRO 4000 Blackwell | 24 GB | ~211 tok/s | 256K (full) | No price yet |
| NVIDIA RTX PRO 4000 SFF Blackwell | 24 GB | ~186 tok/s | 256K (full) | No price yet |
| NVIDIA Tesla P40 | 24 GB | ~152 tok/s | 256K (full) | No price yet |
| NVIDIA TITAN RTX | 24 GB | ~211 tok/s | 256K (full) | No price yet |
| AMD Radeon PRO V710 | 28 GB | ~230 tok/s | 256K (full) | No price yet |
| AMD Radeon AI PRO R9600 | 32 GB | ~266 tok/s | 256K (full) | No price yet |
| AMD Radeon AI PRO R9600D | 32 GB | ~266 tok/s | 256K (full) | No price yet |
| AMD Radeon AI PRO R9700 | 32 GB | ~266 tok/s | 256K (full) | No price yet |
| AMD Radeon AI PRO R9700S | 32 GB | ~266 tok/s | 256K (full) | No price yet |
| AMD Radeon PRO V620 | 32 GB | ~244 tok/s | 256K (full) | No price yet |
| AMD Radeon PRO W6800 | 32 GB | ~244 tok/s | 256K (full) | No price yet |
| AMD Radeon PRO W7800 32GB | 32 GB | ~256 tok/s | 256K (full) | No price yet |
| Intel Arc Pro B70 | 32 GB | ~234 tok/s | 256K (full) | No price yet |
| NVIDIA Quadro GV100 | 32 GB | ~223 tok/s | 256K (full) | No price yet |
| NVIDIA RTX 5000 Ada Generation | 32 GB | ~203 tok/s | 256K (full) | No price yet |
| NVIDIA RTX PRO 4500 Blackwell | 32 GB | ~225 tok/s | 256K (full) | No price yet |
| NVIDIA V100 PCIe 32GB | 32 GB | ~225 tok/s | 256K (full) | No price yet |
| NVIDIA V100S PCIe 32GB | 32 GB | ~234 tok/s | 256K (full) | No price yet |
| Apple M5 Pro 48GB | 48 GB (~36 usable) | ~148 tok/s | 256K (full) | No price yet |
| NVIDIA A100 40GB PCIe | 40 GB | ~245 tok/s | 256K (full) | No price yet |
| NVIDIA A800 40GB Active | 40 GB | ~245 tok/s | 256K (full) | No price yet |
| AMD Radeon PRO W7800 48GB | 48 GB | ~294 tok/s | 256K (full) | No price yet |
| AMD Radeon PRO W7900 | 48 GB | ~294 tok/s | 256K (full) | No price yet |
| AMD Radeon PRO W7900 Dual Slot | 48 GB | ~294 tok/s | 256K (full) | No price yet |
| Apple Mac Studio M5 Max (40-core GPU, 64 GB) | 64 GB (~48 usable) | ~181 tok/s | 256K (full) | No price yet |
| NVIDIA A40 | 48 GB | ~213 tok/s | 256K (full) | No price yet |
| NVIDIA L40 | 48 GB | ~223 tok/s | 256K (full) | No price yet |
| NVIDIA L40S | 48 GB | ~223 tok/s | 256K (full) | No price yet |
| NVIDIA Quadro RTX 8000 | 48 GB | ~211 tok/s | 256K (full) | No price yet |
| NVIDIA RTX 6000 Ada Generation | 48 GB | ~228 tok/s | 256K (full) | No price yet |
| NVIDIA RTX PRO 5000 Blackwell 48GB | 48 GB | ~240 tok/s | 256K (full) | No price yet |
| NVIDIA RTX PRO 5000 Blackwell 72GB | 72 GB | ~240 tok/s | 256K (full) | No price yet |
| NVIDIA A100 80GB PCIe | 80 GB | ~250 tok/s | 256K (full) | No price yet |
| NVIDIA RTX PRO 6000 Blackwell Max-Q | 96 GB | ~248 tok/s | 256K (full) | No price yet |
| NVIDIA RTX PRO 6000 Blackwell Server Edition | 96 GB | ~246 tok/s | 256K (full) | No price yet |
| NVIDIA RTX PRO 6000 Blackwell Workstation Edition | 96 GB | ~248 tok/s | 256K (full) | No price yet |
Every device that runs it
Graphics cards, Macs and mini PCs from the wider hardware list behind My rig, smallest usable memory first. Not sure about yours? Check your exact rig.
Show all 224 devices that run it
| Device | Type | Memory | Speed estimate | Max context estimate | Price |
|---|---|---|---|---|---|
| AMD Radeon PRO W6400 | Workstation GPU | 4 GB | ~109 tok/s | 185K · 256K (8-bit) | No price yet |
| AMD Radeon RX 6400 | GPU | 4 GB | ~109 tok/s | 185K · 256K (8-bit) | No price yet |
| AMD Radeon RX 6500 XT 4GB | GPU | 4 GB | ~118 tok/s | 185K · 256K (8-bit) | No price yet |
| AMD Radeon RX 9050 4GB | GPU | 4 GB | ~118 tok/s | 185K · 256K (8-bit) | No price yet |
| Intel Arc A310 | GPU | 4 GB | ~57 tok/s | 185K · 256K (8-bit) | No price yet |
| NVIDIA GeForce GTX 1050 Ti | GPU | 4 GB | ~78 tok/s | 185K · 256K (8-bit) | No price yet |
| NVIDIA GeForce GTX 1650 (GDDR5) | GPU | 4 GB | ~105 tok/s | 185K · 256K (8-bit) | No price yet |
| NVIDIA GeForce GTX 1650 (GDDR6) | GPU | 4 GB | ~132 tok/s | 185K · 256K (8-bit) | No price yet |
| NVIDIA RTX A400 | Workstation GPU | 4 GB | ~87 tok/s | 185K · 256K (8-bit) | No price yet |
| Apple M2 8GB | Mac | 8 GB (~5.4 usable) | ~85 tok/s | 256K (full) | No price yet |
| Apple M3 8GB | Mac | 8 GB (~5.4 usable) | ~85 tok/s | 256K (full) | No price yet |
| Intel Arc A380 | GPU | 6 GB | ~81 tok/s | 256K (full) | No price yet |
| Intel Arc Pro A40 | Workstation GPU | 6 GB | ~83 tok/s | 256K (full) | No price yet |
| Intel Arc Pro A50 | Workstation GPU | 6 GB | ~83 tok/s | 256K (full) | No price yet |
| NVIDIA GeForce GTX 1060 6GB | GPU | 6 GB | ~112 tok/s | 256K (full) | No price yet |
| NVIDIA GeForce GTX 1660 | GPU | 6 GB | ~132 tok/s | 256K (full) | No price yet |
| NVIDIA GeForce GTX 1660 SUPER | GPU | 6 GB | ~170 tok/s | 256K (full) | No price yet |
| NVIDIA GeForce GTX 1660 Ti | GPU | 6 GB | ~160 tok/s | 256K (full) | No price yet |
| NVIDIA GeForce RTX 2060 6GB | GPU | 6 GB | ~170 tok/s | 256K (full) | No price yet |
| NVIDIA RTX A2000 6GB | Workstation GPU | 6 GB | ~160 tok/s | 256K (full) | No price yet |
| AMD Radeon PRO W6600 | Workstation GPU | 8 GB | ~159 tok/s | 256K (full) | No price yet |
| AMD Radeon PRO W7400 | Workstation GPU | 8 GB | ~134 tok/s | 256K (full) | No price yet |
| AMD Radeon PRO W7500 | Workstation GPU | 8 GB | ~172 tok/s | 256K (full) | No price yet |
| AMD Radeon PRO W7600 | Workstation GPU | 8 GB | ~184 tok/s | 256K (full) | No price yet |
| AMD Radeon RX 6500 XT 8GB | GPU | 8 GB | ~118 tok/s | 256K (full) | No price yet |
| AMD Radeon RX 6600 | GPU | 8 GB | ~159 tok/s | 256K (full) | No price yet |
| AMD Radeon RX 6600 XT | GPU | 8 GB | ~172 tok/s | 256K (full) | No price yet |
| AMD Radeon RX 6650 XT | GPU | 8 GB | ~182 tok/s | 256K (full) | No price yet |
| AMD Radeon RX 7400 | GPU | 8 GB | ~135 tok/s | 256K (full) | No price yet |
| AMD Radeon RX 7600 | GPU | 8 GB | ~184 tok/s | 256K (full) | No price yet |
| AMD Radeon RX 9050 | GPU | 8 GB | ~184 tok/s | 256K (full) | No price yet |
| AMD Radeon RX 9060 | GPU | 8 GB | ~184 tok/s | 256K (full) | No price yet |
| AMD Radeon RX 9060 XT 8GB | GPU | 8 GB | ~195 tok/s | 256K (full) | No price yet |
| Intel Arc A580 | GPU | 8 GB | ~166 tok/s | 256K (full) | No price yet |
| Intel Arc A750 | GPU | 8 GB | ~166 tok/s | 256K (full) | No price yet |
| Intel Arc A770 8GB | GPU | 8 GB | ~166 tok/s | 256K (full) | No price yet |
| NVIDIA GeForce GTX 1070 | GPU | 8 GB | ~131 tok/s | 256K (full) | No price yet |
| NVIDIA GeForce GTX 1070 Ti | GPU | 8 GB | ~131 tok/s | 256K (full) | No price yet |
| NVIDIA GeForce GTX 1080 | GPU | 8 GB | ~147 tok/s | 256K (full) | No price yet |
| NVIDIA GeForce RTX 2060 SUPER | GPU | 8 GB | ~189 tok/s | 256K (full) | No price yet |
| NVIDIA GeForce RTX 2070 | GPU | 8 GB | ~189 tok/s | 256K (full) | No price yet |
| NVIDIA GeForce RTX 2070 SUPER | GPU | 8 GB | ~189 tok/s | 256K (full) | No price yet |
| NVIDIA GeForce RTX 2080 | GPU | 8 GB | ~189 tok/s | 256K (full) | No price yet |
| NVIDIA GeForce RTX 2080 SUPER | GPU | 8 GB | ~195 tok/s | 256K (full) | No price yet |
| NVIDIA GeForce RTX 3070 | GPU | 8 GB | ~189 tok/s | 256K (full) | No price yet |
| NVIDIA GeForce RTX 3070 Ti | GPU | 8 GB | ~206 tok/s | 256K (full) | No price yet |
| NVIDIA GeForce RTX 4060 | GPU | 8 GB | ~156 tok/s | 256K (full) | No price yet |
| NVIDIA GeForce RTX 4060 Ti 8GB | GPU | 8 GB | ~160 tok/s | 256K (full) | No price yet |
| NVIDIA GeForce RTX 5050 | GPU | 8 GB | ~167 tok/s | 256K (full) | No price yet |
| NVIDIA GeForce RTX 5060 | GPU | 8 GB | ~189 tok/s | 256K (full) | low $410 |
| NVIDIA GeForce RTX 5060 Ti 8 GB | GPU | 8 GB | ~189 tok/s | 256K (full) | low $450 |
| NVIDIA RTX A1000 | Workstation GPU | 8 GB | ~132 tok/s | 256K (full) | No price yet |
| NVIDIA T1000 8GB | Workstation GPU | 8 GB | ~120 tok/s | 256K (full) | No price yet |
| AMD Radeon RX 6700 | GPU | 10 GB | ~195 tok/s | 256K (full) | No price yet |
| Intel Arc B570 | GPU | 10 GB | ~186 tok/s | 256K (full) | No price yet |
| NVIDIA GeForce RTX 3080 | GPU | 10 GB | ~217 tok/s | 256K (full) | No price yet |
| Apple M1 Pro 16GB | Mac | 16 GB (~10.7 usable) | ~124 tok/s | 256K (full) | No price yet |
| Apple M2 16GB | Mac | 16 GB (~10.7 usable) | ~85 tok/s | 256K (full) | No price yet |
| Apple M2 Pro 16GB | Mac | 16 GB (~10.7 usable) | ~124 tok/s | 256K (full) | No price yet |
| Apple M3 16GB | Mac | 16 GB (~10.7 usable) | ~85 tok/s | 256K (full) | No price yet |
| Apple M4 16GB | Mac | 16 GB (~10.7 usable) | ~95 tok/s | 256K (full) | No price yet |
| Apple M5 16GB | Mac | 16 GB (~10.7 usable) | ~108 tok/s | 256K (full) | No price yet |
| Apple M6 16GB | Mac | 16 GB (~10.7 usable) | ~108 tok/s | 256K (full) | No price yet |
| NVIDIA GeForce GTX 1080 Ti | GPU | 11 GB | ~175 tok/s | 256K (full) | No price yet |
| NVIDIA GeForce RTX 2080 Ti | GPU | 11 GB | ~207 tok/s | 256K (full) | No price yet |
| AMD Radeon RX 6700 XT | GPU | 12 GB | ~214 tok/s | 256K (full) | No price yet |
| AMD Radeon RX 6750 XT | GPU | 12 GB | ~227 tok/s | 256K (full) | No price yet |
| AMD Radeon RX 7700 XT | GPU | 12 GB | ~227 tok/s | 256K (full) | No price yet |
| AMD Radeon RX 9070 GRE | GPU | 12 GB | ~227 tok/s | 256K (full) | No price yet |
| Intel Arc B580 | GPU | 12 GB | ~204 tok/s | 256K (full) | low $330 |
| Intel Arc Pro A60 | Workstation GPU | 12 GB | ~138 tok/s | 256K (full) | No price yet |
| NVIDIA GeForce RTX 3060 12 GB | GPU | 12 GB | ~175 tok/s | 256K (full) | low $305 |
| NVIDIA GeForce RTX 3080 Ti | GPU | 12 GB | ~225 tok/s | 256K (full) | No price yet |
| NVIDIA GeForce RTX 4070 | GPU | 12 GB | ~196 tok/s | 256K (full) | No price yet |
| NVIDIA GeForce RTX 4070 SUPER | GPU | 12 GB | ~196 tok/s | 256K (full) | low $620 |
| NVIDIA GeForce RTX 4070 Ti | GPU | 12 GB | ~196 tok/s | 256K (full) | No price yet |
| NVIDIA GeForce RTX 5070 | GPU | 12 GB | ~211 tok/s | 256K (full) | low $800 |
| NVIDIA RTX A2000 12GB | Workstation GPU | 12 GB | ~160 tok/s | 256K (full) | No price yet |
| NVIDIA Tesla P100 PCIe 12GB | Data center GPU | 12 GB | ~183 tok/s | 256K (full) | No price yet |
| NVIDIA TITAN Xp | GPU | 12 GB | ~182 tok/s | 256K (full) | No price yet |
| Apple M3 Pro 18GB | Mac | 18 GB (~12.1 usable) | ~107 tok/s | 256K (full) | No price yet |
| AMD Radeon PRO W7700 | Workstation GPU | 16 GB | ~256 tok/s | 256K (full) | No price yet |
| AMD Radeon RX 6800 | GPU | 16 GB | ~244 tok/s | 256K (full) | No price yet |
| AMD Radeon RX 6800 XT | GPU | 16 GB | ~244 tok/s | 256K (full) | No price yet |
| AMD Radeon RX 6900 XT | GPU | 16 GB | ~244 tok/s | 256K (full) | No price yet |
| AMD Radeon RX 6950 XT | GPU | 16 GB | ~256 tok/s | 256K (full) | No price yet |
| AMD Radeon RX 7600 XT | GPU | 16 GB | ~184 tok/s | 256K (full) | No price yet |
| AMD Radeon RX 7700 | GPU | 16 GB | ~264 tok/s | 256K (full) | No price yet |
| AMD Radeon RX 7800 XT | GPU | 16 GB | ~264 tok/s | 256K (full) | low $599 |
| AMD Radeon RX 7900 GRE | GPU | 16 GB | ~256 tok/s | 256K (full) | No price yet |
| AMD Radeon RX 9060 XT 16 GB | GPU | 16 GB | ~195 tok/s | 256K (full) | low $620 |
| AMD Radeon RX 9060 XT LP | GPU | 16 GB | ~195 tok/s | 256K (full) | No price yet |
| AMD Radeon RX 9070 | GPU | 16 GB | ~266 tok/s | 256K (full) | low $570 |
| AMD Radeon RX 9070 XT | GPU | 16 GB | ~266 tok/s | 256K (full) | low $810 |
| Intel Arc A770 16GB | GPU | 16 GB | ~175 tok/s | 256K (full) | No price yet |
| Intel Arc Pro B50 | Workstation GPU | 16 GB | ~134 tok/s | 256K (full) | No price yet |
| NVIDIA GeForce RTX 4060 Ti 16 GB | GPU | 16 GB | ~160 tok/s | 256K (full) | No price yet |
| NVIDIA GeForce RTX 4070 Ti SUPER | GPU | 16 GB | ~211 tok/s | 256K (full) | No price yet |
| NVIDIA GeForce RTX 4080 | GPU | 16 GB | ~214 tok/s | 256K (full) | No price yet |
| NVIDIA GeForce RTX 4080 SUPER | GPU | 16 GB | ~216 tok/s | 256K (full) | No price yet |
| NVIDIA GeForce RTX 5060 Ti 16 GB | GPU | 16 GB | ~189 tok/s | 256K (full) | low $780 |
| NVIDIA GeForce RTX 5070 Ti | GPU | 16 GB | ~225 tok/s | 256K (full) | low $1,120 |
| NVIDIA GeForce RTX 5080 | GPU | 16 GB | ~228 tok/s | 256K (full) | low $1,600 |
| NVIDIA RTX 2000 Ada Generation | Workstation GPU | 16 GB | ~143 tok/s | 256K (full) | No price yet |
| NVIDIA RTX A4000 | Workstation GPU | 16 GB | ~189 tok/s | 256K (full) | No price yet |
| NVIDIA RTX PRO 2000 Blackwell | Workstation GPU | 16 GB | ~160 tok/s | 256K (full) | No price yet |
| NVIDIA T4 | Data center GPU | 16 GB | ~163 tok/s | 256K (full) | No price yet |
| NVIDIA Tesla P100 PCIe 16GB | Data center GPU | 16 GB | ~200 tok/s | 256K (full) | No price yet |
| NVIDIA V100 PCIe 16GB | Data center GPU | 16 GB | ~225 tok/s | 256K (full) | No price yet |
| Apple M2 24GB | Mac | 24 GB (~16.1 usable) | ~85 tok/s | 256K (full) | No price yet |
| Apple M3 24GB | Mac | 24 GB (~16.1 usable) | ~85 tok/s | 256K (full) | No price yet |
| Apple M4 24GB | Mac | 24 GB (~16.1 usable) | ~95 tok/s | 256K (full) | No price yet |
| Apple M4 Pro 24GB | Mac | 24 GB (~16.1 usable) | ~142 tok/s | 256K (full) | No price yet |
| Apple M5 24GB | Mac | 24 GB (~16.1 usable) | ~108 tok/s | 256K (full) | No price yet |
| Apple M5 Pro 24GB | Mac | 24 GB (~16.1 usable) | ~148 tok/s | 256K (full) | No price yet |
| Apple M6 24GB | Mac | 24 GB (~16.1 usable) | ~115 tok/s | 256K (full) | No price yet |
| AMD Radeon RX 7900 XT | GPU | 20 GB | ~287 tok/s | 256K (full) | No price yet |
| NVIDIA RTX 4000 Ada Generation | Workstation GPU | 20 GB | ~175 tok/s | 256K (full) | No price yet |
| NVIDIA RTX 4000 SFF Ada Generation | Workstation GPU | 20 GB | ~158 tok/s | 256K (full) | No price yet |
| NVIDIA RTX A4500 | Workstation GPU | 20 GB | ~209 tok/s | 256K (full) | No price yet |
| Apple M1 Max 32GB | Mac | 32 GB (~21.4 usable) | ~162 tok/s | 256K (full) | No price yet |
| Apple M1 Pro 32GB | Mac | 32 GB (~21.4 usable) | ~124 tok/s | 256K (full) | No price yet |
| Apple M2 Max (30-core GPU) 32GB | Mac | 32 GB (~21.4 usable) | ~162 tok/s | 256K (full) | No price yet |
| Apple M2 Max (38-core GPU) 32GB | Mac | 32 GB (~21.4 usable) | ~162 tok/s | 256K (full) | No price yet |
| Apple M2 Pro 32GB | Mac | 32 GB (~21.4 usable) | ~124 tok/s | 256K (full) | No price yet |
| Apple M4 32GB | Mac | 32 GB (~21.4 usable) | ~95 tok/s | 256K (full) | No price yet |
| Apple M5 32GB | Mac | 32 GB (~21.4 usable) | ~108 tok/s | 256K (full) | No price yet |
| Apple M6 32GB | Mac | 32 GB (~21.4 usable) | ~115 tok/s | 256K (full) | No price yet |
| AMD Radeon RX 7900 XTX | GPU | 24 GB | ~302 tok/s | 256K (full) | low $1,350 |
| Intel Arc Pro B60 | Workstation GPU | 24 GB | ~204 tok/s | 256K (full) | No price yet |
| NVIDIA A10 | Data center GPU | 24 GB | ~205 tok/s | 256K (full) | No price yet |
| NVIDIA GeForce RTX 3090 | GPU | 24 GB | ~227 tok/s | 256K (full) | low $1,499 |
| NVIDIA GeForce RTX 3090 Ti | GPU | 24 GB | ~230 tok/s | 256K (full) | No price yet |
| NVIDIA GeForce RTX 4090 | GPU | 24 GB | ~230 tok/s | 256K (full) | low $2,910 |
| NVIDIA L4 | Data center GPU | 24 GB | ~163 tok/s | 256K (full) | No price yet |
| NVIDIA Quadro RTX 6000 | Workstation GPU | 24 GB | ~211 tok/s | 256K (full) | No price yet |
| NVIDIA RTX 4500 Ada Generation | Workstation GPU | 24 GB | ~186 tok/s | 256K (full) | No price yet |
| NVIDIA RTX A5000 | Workstation GPU | 24 GB | ~218 tok/s | 256K (full) | No price yet |
| NVIDIA RTX A5500 | Workstation GPU | 24 GB | ~218 tok/s | 256K (full) | No price yet |
| NVIDIA RTX PRO 4000 Blackwell | Workstation GPU | 24 GB | ~211 tok/s | 256K (full) | No price yet |
| NVIDIA RTX PRO 4000 SFF Blackwell | Workstation GPU | 24 GB | ~186 tok/s | 256K (full) | No price yet |
| NVIDIA RTX Spark N1X laptop (6144-core GPU, 24GB) | Laptop | 24 GB | ~163 tok/s | 256K (full) | No price yet |
| NVIDIA Tesla P40 | Data center GPU | 24 GB | ~152 tok/s | 256K (full) | No price yet |
| NVIDIA TITAN RTX | GPU | 24 GB | ~211 tok/s | 256K (full) | No price yet |
| Apple M3 Max (30-core GPU) 36GB | Mac | 36 GB (~24.1 usable) | ~147 tok/s | 256K (full) | No price yet |
| Apple M3 Pro 36GB | Mac | 36 GB (~24.1 usable) | ~107 tok/s | 256K (full) | No price yet |
| Apple M4 Max (32-core GPU) 36GB | Mac | 36 GB (~24.1 usable) | ~163 tok/s | 256K (full) | No price yet |
| Apple Mac Studio M5 Max (32-core GPU, 36 GB) | Mac | 36 GB (~24.1 usable) | ~168 tok/s | 256K (full) | low $2,499 |
| AMD Radeon PRO V710 | Data center GPU | 28 GB | ~230 tok/s | 256K (full) | No price yet |
| AMD Radeon AI PRO R9600 | Workstation GPU | 32 GB | ~266 tok/s | 256K (full) | No price yet |
| AMD Radeon AI PRO R9600D | Workstation GPU | 32 GB | ~266 tok/s | 256K (full) | No price yet |
| AMD Radeon AI PRO R9700 | Workstation GPU | 32 GB | ~266 tok/s | 256K (full) | No price yet |
| AMD Radeon AI PRO R9700S | Workstation GPU | 32 GB | ~266 tok/s | 256K (full) | No price yet |
| AMD Radeon PRO V620 | Data center GPU | 32 GB | ~244 tok/s | 256K (full) | No price yet |
| AMD Radeon PRO W6800 | Workstation GPU | 32 GB | ~244 tok/s | 256K (full) | No price yet |
| AMD Radeon PRO W7800 32GB | Workstation GPU | 32 GB | ~256 tok/s | 256K (full) | No price yet |
| AMD Ryzen AI Max 385 (32GB) | Mini PC | 32 GB | ~172 tok/s | 256K (full) | No price yet |
| AMD Ryzen AI Max 390 (32GB) | Mini PC | 32 GB | ~172 tok/s | 256K (full) | No price yet |
| AMD Ryzen AI Max+ 395 (32GB) | Mini PC | 32 GB | ~172 tok/s | 256K (full) | No price yet |
| Intel Arc Pro B70 | Workstation GPU | 32 GB | ~234 tok/s | 256K (full) | No price yet |
| NVIDIA GeForce RTX 5090 | GPU | 32 GB | ~248 tok/s | 256K (full) | low $6,500 |
| NVIDIA Quadro GV100 | Workstation GPU | 32 GB | ~223 tok/s | 256K (full) | No price yet |
| NVIDIA RTX 5000 Ada Generation | Workstation GPU | 32 GB | ~203 tok/s | 256K (full) | No price yet |
| NVIDIA RTX PRO 4500 Blackwell | Workstation GPU | 32 GB | ~225 tok/s | 256K (full) | No price yet |
| NVIDIA RTX Spark N1X laptop (6144-core GPU, 32GB) | Laptop | 32 GB | ~163 tok/s | 256K (full) | No price yet |
| NVIDIA V100 PCIe 32GB | Data center GPU | 32 GB | ~225 tok/s | 256K (full) | No price yet |
| NVIDIA V100S PCIe 32GB | Data center GPU | 32 GB | ~234 tok/s | 256K (full) | No price yet |
| Apple M3 Max (40-core GPU) 48GB | Mac | 48 GB (~36 usable) | ~162 tok/s | 256K (full) | No price yet |
| Apple M4 Max (40-core GPU) 48GB | Mac | 48 GB (~36 usable) | ~176 tok/s | 256K (full) | No price yet |
| Apple M4 Pro 48GB | Mac | 48 GB (~36 usable) | ~142 tok/s | 256K (full) | No price yet |
| Apple M5 Max (40-core GPU) 48GB | Mac | 48 GB (~36 usable) | ~181 tok/s | 256K (full) | No price yet |
| Apple M5 Pro 48GB | Mac | 48 GB (~36 usable) | ~148 tok/s | 256K (full) | No price yet |
| NVIDIA A100 40GB PCIe | Data center GPU | 40 GB | ~245 tok/s | 256K (full) | No price yet |
| NVIDIA A800 40GB Active | Workstation GPU | 40 GB | ~245 tok/s | 256K (full) | No price yet |
| AMD Radeon PRO W7800 48GB | Workstation GPU | 48 GB | ~294 tok/s | 256K (full) | No price yet |
| AMD Radeon PRO W7900 | Workstation GPU | 48 GB | ~294 tok/s | 256K (full) | No price yet |
| AMD Radeon PRO W7900 Dual Slot | Workstation GPU | 48 GB | ~294 tok/s | 256K (full) | No price yet |
| Apple M1 Max 64GB | Mac | 64 GB (~48 usable) | ~162 tok/s | 256K (full) | No price yet |
| Apple M1 Ultra 64GB | Mac | 64 GB (~48 usable) | ~191 tok/s | 256K (full) | No price yet |
| Apple M2 Max (30-core GPU) 64GB | Mac | 64 GB (~48 usable) | ~162 tok/s | 256K (full) | No price yet |
| Apple M2 Max (38-core GPU) 64GB | Mac | 64 GB (~48 usable) | ~162 tok/s | 256K (full) | No price yet |
| Apple M2 Ultra 64GB | Mac | 64 GB (~48 usable) | ~191 tok/s | 256K (full) | No price yet |
| Apple M3 Max (40-core GPU) 64GB | Mac | 64 GB (~48 usable) | ~162 tok/s | 256K (full) | No price yet |
| Apple M4 Max (40-core GPU) 64GB | Mac | 64 GB (~48 usable) | ~176 tok/s | 256K (full) | No price yet |
| Apple M4 Pro 64GB | Mac | 64 GB (~48 usable) | ~142 tok/s | 256K (full) | No price yet |
| Apple M5 Pro 64GB | Mac | 64 GB (~48 usable) | ~148 tok/s | 256K (full) | No price yet |
| Apple Mac Studio M5 Max (40-core GPU, 64 GB) | Mac | 64 GB (~48 usable) | ~181 tok/s | 256K (full) | No price yet |
| NVIDIA A40 | Data center GPU | 48 GB | ~213 tok/s | 256K (full) | No price yet |
| NVIDIA L40 | Data center GPU | 48 GB | ~223 tok/s | 256K (full) | No price yet |
| NVIDIA L40S | Data center GPU | 48 GB | ~223 tok/s | 256K (full) | No price yet |
| NVIDIA Quadro RTX 8000 | Workstation GPU | 48 GB | ~211 tok/s | 256K (full) | No price yet |
| NVIDIA RTX 6000 Ada Generation | Workstation GPU | 48 GB | ~228 tok/s | 256K (full) | No price yet |
| NVIDIA RTX A6000 | Workstation GPU | 48 GB | ~218 tok/s | 256K (full) | low $8,363 |
| NVIDIA RTX PRO 5000 Blackwell 48GB | Workstation GPU | 48 GB | ~240 tok/s | 256K (full) | No price yet |
| NVIDIA RTX Spark N1X laptop (6144-core GPU, 48GB) | Laptop | 48 GB | ~163 tok/s | 256K (full) | No price yet |
| AMD Ryzen AI Max 385 (64GB) | Mini PC | 64 GB | ~172 tok/s | 256K (full) | No price yet |
| AMD Ryzen AI Max 390 (64GB) | Mini PC | 64 GB | ~172 tok/s | 256K (full) | No price yet |
| AMD Ryzen AI Max+ 395 (64GB) | Mini PC | 64 GB | ~172 tok/s | 256K (full) | low $2,199.99 |
| NVIDIA DGX Spark (64GB) | Mini PC | 64 GB | ~157 tok/s | 256K (full) | low $4,999 |
| NVIDIA RTX Spark N1X laptop (6144-core GPU, 64GB) | Laptop | 64 GB | ~163 tok/s | 256K (full) | No price yet |
| Apple M2 Max (38-core GPU) 96GB | Mac | 96 GB (~72 usable) | ~162 tok/s | 256K (full) | No price yet |
| Apple M3 Max (30-core GPU) 96GB | Mac | 96 GB (~72 usable) | ~147 tok/s | 256K (full) | No price yet |
| Apple M3 Ultra 96GB | Mac | 96 GB (~72 usable) | ~192 tok/s | 256K (full) | No price yet |
| Apple Mac Studio M5 Ultra (64-core GPU, 96 GB) | Mac | 96 GB (~72 usable) | ~203 tok/s | 256K (full) | low $5,499 |
| NVIDIA RTX PRO 5000 Blackwell 72GB | Workstation GPU | 72 GB | ~240 tok/s | 256K (full) | No price yet |
| NVIDIA A100 80GB PCIe | Data center GPU | 80 GB | ~250 tok/s | 256K (full) | No price yet |
| Apple M1 Ultra 128GB | Mac | 128 GB (~96 usable) | ~191 tok/s | 256K (full) | No price yet |
| Apple M2 Ultra 128GB | Mac | 128 GB (~96 usable) | ~191 tok/s | 256K (full) | No price yet |
| Apple M3 Max (40-core GPU) 128GB | Mac | 128 GB (~96 usable) | ~162 tok/s | 256K (full) | No price yet |
| Apple M4 Max (40-core GPU) 128GB | Mac | 128 GB (~96 usable) | ~176 tok/s | 256K (full) | No price yet |
| Apple M5 Max (40-core GPU) 128GB | Mac | 128 GB (~96 usable) | ~181 tok/s | 256K (full) | No price yet |
| NVIDIA RTX PRO 6000 Blackwell Max-Q | Workstation GPU | 96 GB | ~248 tok/s | 256K (full) | No price yet |
| NVIDIA RTX PRO 6000 Blackwell Server Edition | Data center GPU | 96 GB | ~246 tok/s | 256K (full) | No price yet |
| NVIDIA RTX PRO 6000 Blackwell Workstation Edition | Workstation GPU | 96 GB | ~248 tok/s | 256K (full) | No price yet |
| AMD Ryzen AI Max 385 (128GB) | Mini PC | 128 GB | ~172 tok/s | 256K (full) | No price yet |
| AMD Ryzen AI Max 390 (128GB) | Mini PC | 128 GB | ~172 tok/s | 256K (full) | No price yet |
| AMD Ryzen AI Max+ 395 (128GB) | Mini PC | 128 GB | ~172 tok/s | 256K (full) | low $3,499.99 |
| NVIDIA DGX Spark (128GB) | Mini PC | 128 GB | ~157 tok/s | 256K (full) | low $6,949.99 |
| NVIDIA RTX Spark N1X laptop (6144-core GPU, 128GB) | Laptop | 128 GB | ~163 tok/s | 256K (full) | No price yet |
| Apple M2 Ultra 192GB | Mac | 192 GB (~144 usable) | ~191 tok/s | 256K (full) | No price yet |
| Apple M3 Ultra 256GB | Mac | 256 GB (~192 usable) | ~192 tok/s | 256K (full) | No price yet |
| Apple M5 Ultra 256GB | Mac | 256 GB (~192 usable) | ~203 tok/s | 256K (full) | No price yet |
| Framework Desktop, AMD Ryzen AI Max+ PRO 495 (Strix Halo), 192 GB | Mini PC | 192 GB | ~179 tok/s | 256K (full) | low $6,799 |
| Apple M5 Ultra 512GB | Mac | 512 GB (~384 usable) | ~203 tok/s | 256K (full) | No price yet |
Speed calculator
estimateEstimate = bandwidth ÷ GB read per token (the model's listed GGUF file: Q4_K_M for most models, otherwise the quant named in the list; scaled by active/total parameters for MoE), scaled by a per-platform efficiency and fixed per-token costs calibrated to 45 measured llama.cpp benchmarks (2025–26). Fit = file size + 1.5 GB headroom ≤ total VRAM (on Macs, the part of unified memory macOS lets the GPU use by default), and room for at least 4K tokens of context with an 8-bit KV cache. For a model with no KV cache data, only the file and headroom are checked, so a fit says nothing about context. Multi-GPU: VRAM adds, speed stays one card's (a layer split runs the cards in turn); max context uses the total. Speed is for short context. Max context = what is left after the file and a 1 GB runtime reserve ÷ KV cache bytes per token, default FP16 and 8-bit (q8_0) cache. How speed works · how context works