| AMD Radeon RX 7900 XT | GPU | 20 GB | ~33 tok/s | 23K · 43K (8-bit) | No price yet |
| NVIDIA RTX 4000 Ada Generation | Workstation GPU | 20 GB | ~17 tok/s | 23K · 43K (8-bit) | No price yet |
| NVIDIA RTX 4000 SFF Ada Generation | Workstation GPU | 20 GB | ~13 tok/s | 23K · 43K (8-bit) | No price yet |
| NVIDIA RTX A4500 | Workstation GPU | 20 GB | ~29 tok/s | 23K · 43K (8-bit) | No price yet |
| Apple M1 Max 32GB | Mac | 32 GB (~21.4 usable) | ~19 tok/s | 44K · 83K (8-bit) | No price yet |
| Apple M1 Pro 32GB | Mac | 32 GB (~21.4 usable) | ~10 tok/s | 44K · 83K (8-bit) | No price yet |
| Apple M2 Max (30-core GPU) 32GB | Mac | 32 GB (~21.4 usable) | ~19 tok/s | 44K · 83K (8-bit) | No price yet |
| Apple M2 Max (38-core GPU) 32GB | Mac | 32 GB (~21.4 usable) | ~19 tok/s | 44K · 83K (8-bit) | No price yet |
| Apple M2 Pro 32GB | Mac | 32 GB (~21.4 usable) | ~10 tok/s | 44K · 83K (8-bit) | No price yet |
| Apple M4 32GB | Mac | 32 GB (~21.4 usable) | ~6 tok/s | 44K · 83K (8-bit) | No price yet |
| Apple M5 32GB | Mac | 32 GB (~21.4 usable) | ~8 tok/s | 44K · 83K (8-bit) | No price yet |
| Apple M6 32GB | Mac | 32 GB (~21.4 usable) | ~8 tok/s | 44K · 83K (8-bit) | No price yet |
| AMD Radeon RX 7900 XTX | GPU | 24 GB | ~38 tok/s | 82K · 155K (8-bit) | low $1,350avg $1,552 · new · source, 2026-11-02 |
| Intel Arc Pro B60 | Workstation GPU | 24 GB | ~15 tok/s | 82K · 155K (8-bit) | No price yet |
| NVIDIA A10 | Data center GPU | 24 GB | ~27 tok/s | 82K · 155K (8-bit) | No price yet |
| NVIDIA GeForce RTX 3090 | GPU | 24 GB | ~40 tok/s | 82K · 155K (8-bit) | low $1,499avg $1,551 · used · source, 2026-11-02 |
| NVIDIA GeForce RTX 3090 Ti | GPU | 24 GB | ~43 tok/s | 82K · 155K (8-bit) | No price yet |
| NVIDIA GeForce RTX 4090 | GPU | 24 GB | ~43 tok/s | 82K · 155K (8-bit) | low $2,910avg $3,100 · used · source, 2026-11-02 |
| NVIDIA L4 | Data center GPU | 24 GB | ~14 tok/s | 82K · 155K (8-bit) | No price yet |
| NVIDIA Quadro RTX 6000 | Workstation GPU | 24 GB | ~30 tok/s | 82K · 155K (8-bit) | No price yet |
| NVIDIA RTX 4500 Ada Generation | Workstation GPU | 24 GB | ~20 tok/s | 82K · 155K (8-bit) | No price yet |
| NVIDIA RTX A5000 | Workstation GPU | 24 GB | ~34 tok/s | 82K · 155K (8-bit) | No price yet |
| NVIDIA RTX A5500 | Workstation GPU | 24 GB | ~34 tok/s | 82K · 155K (8-bit) | No price yet |
| NVIDIA RTX PRO 4000 Blackwell | Workstation GPU | 24 GB | ~30 tok/s | 82K · 155K (8-bit) | No price yet |
| NVIDIA RTX PRO 4000 SFF Blackwell | Workstation GPU | 24 GB | ~20 tok/s | 82K · 155K (8-bit) | No price yet |
| NVIDIA RTX Spark N1X laptop (6144-core GPU, 24GB) | Laptop | 24 GB | ~14 tok/s | 82K · 155K (8-bit) | No price yet |
| NVIDIA Tesla P40 | Data center GPU | 24 GB | ~12 tok/s | 82K · 155K (8-bit) | No price yet |
| NVIDIA TITAN RTX | GPU | 24 GB | ~30 tok/s | 82K · 155K (8-bit) | No price yet |
| Apple M3 Max (30-core GPU) 36GB | Mac | 36 GB (~24.1 usable) | ~14 tok/sllama.cpp-style estimateMLX (measured, community): 16.0 tok/s at 4K (M3 Max 30c)With speculative decoding (MTP/DFlash): 17.8–21.3 tok/s at 1K–4K (M3 Max 30c) | 84K · 158K (8-bit) | No price yet |
| Apple M3 Pro 36GB | Mac | 36 GB (~24.1 usable) | ~7 tok/s | 84K · 158K (8-bit) | No price yet |
| Apple M4 Max (32-core GPU) 36GB | Mac | 36 GB (~24.1 usable) | ~19 tok/sllama.cpp-style estimateMLX (measured, community): 20.1 tok/s at 4K (M4 Max 32c) | 84K · 158K (8-bit) | No price yet |
| Apple Mac Studio M5 Max (32-core GPU, 36 GB) | Mac | 36 GB (~24.1 usable) | ~21 tok/s | 84K · 158K (8-bit) | low $2,499new · source, 2026-10-16 |
| AMD Radeon PRO V710 | Data center GPU | 28 GB | ~19 tok/s | 142K · 256K (8-bit) | No price yet |
| AMD Radeon AI PRO R9600 | Workstation GPU | 32 GB | ~26 tok/s | 202K · 256K (8-bit) | No price yet |
| AMD Radeon AI PRO R9600D | Workstation GPU | 32 GB | ~26 tok/s | 202K · 256K (8-bit) | No price yet |
| AMD Radeon AI PRO R9700 | Workstation GPU | 32 GB | ~26 tok/s | 202K · 256K (8-bit) | No price yet |
| AMD Radeon AI PRO R9700S | Workstation GPU | 32 GB | ~26 tok/s | 202K · 256K (8-bit) | No price yet |
| AMD Radeon PRO V620 | Data center GPU | 32 GB | ~21 tok/s | 202K · 256K (8-bit) | No price yet |
| AMD Radeon PRO W6800 | Workstation GPU | 32 GB | ~21 tok/s | 202K · 256K (8-bit) | No price yet |
| AMD Radeon PRO W7800 32GB | Workstation GPU | 32 GB | ~24 tok/s | 202K · 256K (8-bit) | No price yet |
| AMD Ryzen AI Max 385 (32GB) | Mini PC | 32 GB | ~11 tok/s | 202K · 256K (8-bit) | No price yet |
| AMD Ryzen AI Max 390 (32GB) | Mini PC | 32 GB | ~11 tok/s | 202K · 256K (8-bit) | No price yet |
| AMD Ryzen AI Max+ 395 (32GB) | Mini PC | 32 GB | ~11 tok/s | 202K · 256K (8-bit) | No price yet |
| Intel Arc Pro B70 | Workstation GPU | 32 GB | ~20 tok/s | 202K · 256K (8-bit) | No price yet |
| NVIDIA GeForce RTX 5090 | GPU | 32 GB | ~68 tok/s | 202K · 256K (8-bit) | low $6,500avg $8,757 · new · source, 2026-11-02 |
| NVIDIA Quadro GV100 | Workstation GPU | 32 GB | ~38 tok/s | 202K · 256K (8-bit) | No price yet |
| NVIDIA RTX 5000 Ada Generation | Workstation GPU | 32 GB | ~26 tok/s | 202K · 256K (8-bit) | No price yet |
| NVIDIA RTX PRO 4500 Blackwell | Workstation GPU | 32 GB | ~39 tok/s | 202K · 256K (8-bit) | No price yet |
| NVIDIA RTX Spark N1X laptop (6144-core GPU, 32GB) | Laptop | 32 GB | ~14 tok/s | 202K · 256K (8-bit) | No price yet |
| NVIDIA V100 PCIe 32GB | Data center GPU | 32 GB | ~39 tok/s | 202K · 256K (8-bit) | No price yet |
| NVIDIA V100S PCIe 32GB | Data center GPU | 32 GB | ~47 tok/s | 202K · 256K (8-bit) | No price yet |
| Apple M3 Max (40-core GPU) 48GB | Mac | 48 GB (~36 usable) | ~19 tok/s | 256K (full) | No price yet |
| Apple M4 Max (40-core GPU) 48GB | Mac | 48 GB (~36 usable) | ~25 tok/s | 256K (full) | No price yet |
| Apple M4 Pro 48GB | Mac | 48 GB (~36 usable) | ~13 tok/sllama.cpp-style estimateMLX (measured, community): 15.6 tok/s at 1K (M4 Pro 16c)With speculative decoding (MTP/DFlash): 22.9–32.2 tok/s at 4K–16K (M4 Pro 16c) | 256K (full) | No price yet |
| Apple M5 Max (40-core GPU) 48GB | Mac | 48 GB (~36 usable) | ~28 tok/s | 256K (full) | No price yet |
| Apple M5 Pro 48GB | Mac | 48 GB (~36 usable) | ~15 tok/sllama.cpp-style estimateMLX (measured, community): 19.0 tok/s at 1K (M5 Pro 20c)With speculative decoding (MTP/DFlash): 18.8–18.9 tok/s at 1K–4K (M5 Pro 20c)GPU cores differ between runs (M5 Pro 16c / M5 Pro 20c); listed here by chip and memory. | 256K (full) | No price yet |
| NVIDIA A100 40GB PCIe | Data center GPU | 40 GB | ~61 tok/s | 256K (full) | No price yet |
| NVIDIA A800 40GB Active | Workstation GPU | 40 GB | ~61 tok/s | 256K (full) | No price yet |
| AMD Radeon PRO W7800 48GB | Workstation GPU | 48 GB | ~35 tok/s | 256K (full) | No price yet |
| AMD Radeon PRO W7900 | Workstation GPU | 48 GB | ~35 tok/s | 256K (full) | No price yet |
| AMD Radeon PRO W7900 Dual Slot | Workstation GPU | 48 GB | ~35 tok/s | 256K (full) | No price yet |
| Apple M1 Max 64GB | Mac | 64 GB (~48 usable) | ~19 tok/s | 256K (full) | No price yet |
| Apple M1 Ultra 64GB | Mac | 64 GB (~48 usable) | ~35 tok/s | 256K (full) | No price yet |
| Apple M2 Max (30-core GPU) 64GB | Mac | 64 GB (~48 usable) | ~19 tok/s | 256K (full) | No price yet |
| Apple M2 Max (38-core GPU) 64GB | Mac | 64 GB (~48 usable) | ~19 tok/s | 256K (full) | No price yet |
| Apple M2 Ultra 64GB | Mac | 64 GB (~48 usable) | ~35 tok/s | 256K (full) | No price yet |
| Apple M3 Max (40-core GPU) 64GB | Mac | 64 GB (~48 usable) | ~19 tok/sllama.cpp-style estimateMLX (measured, community), with speculative decoding (MTP/DFlash): 9.9–13.9 tok/s at 1K–16K (M3 Max 40c) | 256K (full) | No price yet |
| Apple M4 Max (40-core GPU) 64GB | Mac | 64 GB (~48 usable) | ~25 tok/sllama.cpp-style estimateMLX (measured, community): 29.3 tok/s at 1K (M4 Max 40c)With speculative decoding (MTP/DFlash): 63.7–68.7 tok/s (runs 1, 2) at 8K–16K (M4 Max 40c) | 256K (full) | No price yet |
| Apple M4 Pro 64GB | Mac | 64 GB (~48 usable) | ~13 tok/s | 256K (full) | No price yet |
| Apple M5 Pro 64GB | Mac | 64 GB (~48 usable) | ~15 tok/sllama.cpp-style estimateMLX (measured, community): 17.8 tok/s at 1K (M5 Pro 20c)With speculative decoding (MTP/DFlash): 26.8–28.2 tok/s at 64K–128K (M5 Pro 16c)GPU cores differ between runs (M5 Pro 16c / M5 Pro 20c); listed here by chip and memory. | 256K (full) | No price yet |
| Apple Mac Studio M5 Max (40-core GPU, 64 GB) | Mac | 64 GB (~48 usable) | ~28 tok/sllama.cpp-style estimateMLX (measured, community), with speculative decoding (MTP/DFlash): 27.2–119.8 tok/s (runs 1, 2) at 1K–195K (M5 Max 40c) | 256K (full) | No price yet |
| NVIDIA A40 | Data center GPU | 48 GB | ~31 tok/s | 256K (full) | No price yet |
| NVIDIA L40 | Data center GPU | 48 GB | ~38 tok/s | 256K (full) | No price yet |
| NVIDIA L40S | Data center GPU | 48 GB | ~38 tok/s | 256K (full) | No price yet |
| NVIDIA Quadro RTX 8000 | Workstation GPU | 48 GB | ~30 tok/s | 256K (full) | No price yet |
| NVIDIA RTX 6000 Ada Generation | Workstation GPU | 48 GB | ~41 tok/s | 256K (full) | No price yet |
| NVIDIA RTX A6000 | Workstation GPU | 48 GB | ~34 tok/s | 256K (full) | low $8,363new · source, 2026-11-02 |
| NVIDIA RTX PRO 5000 Blackwell 48GB | Workstation GPU | 48 GB | ~55 tok/s | 256K (full) | No price yet |
| NVIDIA RTX Spark N1X laptop (6144-core GPU, 48GB) | Laptop | 48 GB | ~14 tok/s | 256K (full) | No price yet |
| AMD Ryzen AI Max 385 (64GB) | Mini PC | 64 GB | ~11 tok/s | 256K (full) | No price yet |
| AMD Ryzen AI Max 390 (64GB) | Mini PC | 64 GB | ~11 tok/s | 256K (full) | No price yet |
| AMD Ryzen AI Max+ 395 (64GB) | Mini PC | 64 GB | ~11 tok/s | 256K (full) | low $2,199.99new · source, 2026-11-13 |
| NVIDIA DGX Spark (64GB) | Mini PC | 64 GB | ~13 tok/s | 256K (full) | low $4,999new · source, 2026-11-13 |
| NVIDIA RTX Spark N1X laptop (6144-core GPU, 64GB) | Laptop | 64 GB | ~14 tok/s | 256K (full) | No price yet |
| Apple M2 Max (38-core GPU) 96GB | Mac | 96 GB (~72 usable) | ~19 tok/s | 256K (full) | No price yet |
| Apple M3 Max (30-core GPU) 96GB | Mac | 96 GB (~72 usable) | ~14 tok/s | 256K (full) | No price yet |
| Apple M3 Ultra 96GB | Mac | 96 GB (~72 usable) | ~35 tok/s | 256K (full) | No price yet |
| Apple Mac Studio M5 Ultra (64-core GPU, 96 GB) | Mac | 96 GB (~72 usable) | ~48 tok/sllama.cpp-style estimateMLX (measured, community): 55.1 tok/s at 1K (M5 Ultra 80c) | 256K (full) | low $5,499new · source, 2026-10-23 |
| NVIDIA RTX PRO 5000 Blackwell 72GB | Workstation GPU | 72 GB | ~55 tok/s | 256K (full) | No price yet |
| NVIDIA A100 80GB PCIe | Data center GPU | 80 GB | ~72 tok/s | 256K (full) | No price yet |
| Apple M1 Ultra 128GB | Mac | 128 GB (~96 usable) | ~35 tok/s | 256K (full) | No price yet |
| Apple M2 Ultra 128GB | Mac | 128 GB (~96 usable) | ~35 tok/s | 256K (full) | No price yet |
| Apple M3 Max (40-core GPU) 128GB | Mac | 128 GB (~96 usable) | ~19 tok/sllama.cpp-style estimateMLX (measured, community), with speculative decoding (MTP/DFlash): 21.4–34.6 tok/s at 1K–4K (M3 Max 40c) | 256K (full) | No price yet |
| Apple M4 Max (40-core GPU) 128GB | Mac | 128 GB (~96 usable) | ~25 tok/s | 256K (full) | No price yet |
| Apple M5 Max (40-core GPU) 128GB | Mac | 128 GB (~96 usable) | ~28 tok/s | 256K (full) | No price yet |
| NVIDIA RTX PRO 6000 Blackwell Max-Q | Workstation GPU | 96 GB | ~68 tok/s | 256K (full) | No price yet |
| NVIDIA RTX PRO 6000 Blackwell Server Edition | Data center GPU | 96 GB | ~63 tok/s | 256K (full) | No price yet |
| NVIDIA RTX PRO 6000 Blackwell Workstation Edition | Workstation GPU | 96 GB | ~68 tok/s | 256K (full) | No price yet |
| AMD Ryzen AI Max 385 (128GB) | Mini PC | 128 GB | ~11 tok/s | 256K (full) | No price yet |
| AMD Ryzen AI Max 390 (128GB) | Mini PC | 128 GB | ~11 tok/s | 256K (full) | No price yet |
| AMD Ryzen AI Max+ 395 (128GB) | Mini PC | 128 GB | ~11 tok/s | 256K (full) | low $3,499.99new · source, 2026-11-13 |
| NVIDIA DGX Spark (128GB) | Mini PC | 128 GB | ~13 tok/s | 256K (full) | low $6,949.99new · source, 2026-10-23 |
| NVIDIA RTX Spark N1X laptop (6144-core GPU, 128GB) | Laptop | 128 GB | ~14 tok/s | 256K (full) | No price yet |
| Apple M2 Ultra 192GB | Mac | 192 GB (~144 usable) | ~35 tok/sllama.cpp-style estimateMLX (measured, community): 34.6 tok/s at 4K (M2 Ultra 76c) | 256K (full) | No price yet |
| Apple M3 Ultra 256GB | Mac | 256 GB (~192 usable) | ~35 tok/s | 256K (full) | No price yet |
| Apple M5 Ultra 256GB | Mac | 256 GB (~192 usable) | ~48 tok/s | 256K (full) | No price yet |
| Framework Desktop, AMD Ryzen AI Max+ PRO 495 (Strix Halo), 192 GB | Mini PC | 192 GB | ~12 tok/s | 256K (full) | low $6,799new · source, 2026-10-16 |
| Apple M5 Ultra 512GB | Mac | 512 GB (~384 usable) | ~48 tok/s | 256K (full) | No price yet |