What GPU do you need to run Qwen3.6-35B-A3B?
A new Radeon RX 7900 XTX ($1,350). It is the cheapest priced graphics card that holds Qwen3.6-35B-A3B (about 23.8 GB at Q4) with room for about 33K tokens of context (63K with 8-bit KV cache), at about 137 tok/s estimate; that is the card price only, and it requires a separate compatible PC. 54 of the 157 GPUs and systems we track fit it.
Parameters35.95B3B active (mixture-of-experts)
VRAM needed~23.8 GBfile + headroom
Single GPUs that fit54
ReleasedApr 15, 2026
LicenseApache 2.0
Context window256K tokens
35.95B parameters, 3B active (mixture-of-experts) (Hugging Face, accessed 2026-10-16). Q4_K_M GGUF: 22.29 GB (source). Architecture config (Hugging Face config.json): 10 attention layers (the rest are recurrent) × 2 KV heads × 256 head dim, so about 20 KB of FP16 KV cache per token of context (how we estimate context).
Other places to get it
Checked Nov 14, 2026. We list only copies we opened and checked. Check file hashes for copies from other uploaders.
- ModelScope: Lab's own account
- Ollama: Runs on your machine
ollama pull qwen3.6:35b-a3b
Single GPUs that fit, cheapest first
| Single GPU that fits | VRAM | Speed estimate | Max context estimate | Price |
|---|---|---|---|---|
| AMD Radeon RX 7900 XTX | 24 GB | ~137 tok/s | 33K · 63K (8-bit) | low $1,350 |
| NVIDIA GeForce RTX 3090 | 24 GB | ~157 tok/s | 33K · 63K (8-bit) | low $1,499 |
| Apple Mac Studio M5 Max (32-core GPU, 36 GB) | 36 GB (~24.1 usable) | ~83 tok/s | 38K · 72K (8-bit) | low $2,499 |
| NVIDIA GeForce RTX 4090 | 24 GB | ~161 tok/s | 33K · 63K (8-bit) | low $2,910 |
| NVIDIA DGX Spark (64GB) | 64 GB | ~84 tok/s | 256K (full) | low $4,999 |
| Apple Mac Studio M5 Ultra (64-core GPU, 96 GB) | 96 GB (~72 usable) | ~108 tok/sllama.cpp-style estimateMLX (measured, community), with speculative decoding (MTP/DFlash): 74.4–213.5 tok/s (runs 1, 2, 3, 4) at 1K–195K (M5 Ultra 80c) | 256K (full) | low $5,499 |
| NVIDIA GeForce RTX 5090 | 32 GB | ~189 tok/s | 256K (full) | low $6,500 |
| Framework Desktop, AMD Ryzen AI Max+ PRO 495 (Strix Halo), 192 GB | 192 GB | ~73 tok/s | 256K (full) | low $6,799 |
| NVIDIA DGX Spark (128GB) | 128 GB | ~84 tok/s | 256K (full) | low $6,949.99 |
| NVIDIA RTX A6000 | 48 GB | ~146 tok/s | 256K (full) | low $8,363 |
| Intel Arc Pro B60 | 24 GB | ~85 tok/s | 33K · 63K (8-bit) | No price yet |
| NVIDIA A10 | 24 GB | ~131 tok/s | 33K · 63K (8-bit) | No price yet |
| NVIDIA GeForce RTX 3090 Ti | 24 GB | ~161 tok/s | 33K · 63K (8-bit) | No price yet |
| NVIDIA L4 | 24 GB | ~90 tok/s | 33K · 63K (8-bit) | No price yet |
| NVIDIA Quadro RTX 6000 | 24 GB | ~138 tok/s | 33K · 63K (8-bit) | No price yet |
| NVIDIA RTX 4500 Ada Generation | 24 GB | ~111 tok/s | 33K · 63K (8-bit) | No price yet |
| NVIDIA RTX A5000 | 24 GB | ~146 tok/s | 33K · 63K (8-bit) | No price yet |
| NVIDIA RTX A5500 | 24 GB | ~146 tok/s | 33K · 63K (8-bit) | No price yet |
| NVIDIA RTX PRO 4000 Blackwell | 24 GB | ~138 tok/s | 33K · 63K (8-bit) | No price yet |
| NVIDIA RTX PRO 4000 SFF Blackwell | 24 GB | ~111 tok/s | 33K · 63K (8-bit) | No price yet |
| NVIDIA Tesla P40 | 24 GB | ~81 tok/s | 33K · 63K (8-bit) | No price yet |
| NVIDIA TITAN RTX | 24 GB | ~138 tok/s | 33K · 63K (8-bit) | No price yet |
| AMD Radeon PRO V710 | 28 GB | ~98 tok/s | 224K · 256K (8-bit) | No price yet |
| AMD Radeon AI PRO R9600 | 32 GB | ~117 tok/s | 256K (full) | No price yet |
| AMD Radeon AI PRO R9600D | 32 GB | ~117 tok/s | 256K (full) | No price yet |
| AMD Radeon AI PRO R9700 | 32 GB | ~117 tok/s | 256K (full) | No price yet |
| AMD Radeon AI PRO R9700S | 32 GB | ~117 tok/s | 256K (full) | No price yet |
| AMD Radeon PRO V620 | 32 GB | ~105 tok/s | 256K (full) | No price yet |
| AMD Radeon PRO W6800 | 32 GB | ~105 tok/s | 256K (full) | No price yet |
| AMD Radeon PRO W7800 32GB | 32 GB | ~111 tok/s | 256K (full) | No price yet |
| Intel Arc Pro B70 | 32 GB | ~100 tok/s | 256K (full) | No price yet |
| NVIDIA Quadro GV100 | 32 GB | ~153 tok/s | 256K (full) | No price yet |
| NVIDIA RTX 5000 Ada Generation | 32 GB | ~129 tok/s | 256K (full) | No price yet |
| NVIDIA RTX PRO 4500 Blackwell | 32 GB | ~155 tok/s | 256K (full) | No price yet |
| NVIDIA V100 PCIe 32GB | 32 GB | ~155 tok/s | 256K (full) | No price yet |
| NVIDIA V100S PCIe 32GB | 32 GB | ~168 tok/s | 256K (full) | No price yet |
| Apple M5 Pro 48GB | 48 GB (~36 usable) | ~70 tok/sllama.cpp-style estimateMLX (measured, community): 106.2 tok/s at 4K (M5 Pro 20c)With speculative decoding (MTP/DFlash): 55.7 tok/s at 16K (M5 Pro 16c)GPU cores differ between runs (M5 Pro 16c / M5 Pro 20c); listed here by chip and memory. | 256K (full) | No price yet |
| NVIDIA A100 40GB PCIe | 40 GB | ~183 tok/s | 256K (full) | No price yet |
| NVIDIA A800 40GB Active | 40 GB | ~183 tok/s | 256K (full) | No price yet |
| AMD Radeon PRO W7800 48GB | 48 GB | ~132 tok/s | 256K (full) | No price yet |
| AMD Radeon PRO W7900 | 48 GB | ~132 tok/s | 256K (full) | No price yet |
| AMD Radeon PRO W7900 Dual Slot | 48 GB | ~132 tok/s | 256K (full) | No price yet |
| Apple Mac Studio M5 Max (40-core GPU, 64 GB) | 64 GB (~48 usable) | ~92 tok/sllama.cpp-style estimateMLX (measured, community), with speculative decoding (MTP/DFlash): 89.5–125.4 tok/s at 1K–4K (M5 Max 40c) | 256K (full) | No price yet |
| NVIDIA A40 | 48 GB | ~140 tok/s | 256K (full) | No price yet |
| NVIDIA L40 | 48 GB | ~153 tok/s | 256K (full) | No price yet |
| NVIDIA L40S | 48 GB | ~153 tok/s | 256K (full) | No price yet |
| NVIDIA Quadro RTX 8000 | 48 GB | ~138 tok/s | 256K (full) | No price yet |
| NVIDIA RTX 6000 Ada Generation | 48 GB | ~159 tok/s | 256K (full) | No price yet |
| NVIDIA RTX PRO 5000 Blackwell 48GB | 48 GB | ~176 tok/s | 256K (full) | No price yet |
| NVIDIA RTX PRO 5000 Blackwell 72GB | 72 GB | ~176 tok/s | 256K (full) | No price yet |
| NVIDIA A100 80GB PCIe | 80 GB | ~193 tok/s | 256K (full) | No price yet |
| NVIDIA RTX PRO 6000 Blackwell Max-Q | 96 GB | ~189 tok/s | 256K (full) | No price yet |
| NVIDIA RTX PRO 6000 Blackwell Server Edition | 96 GB | ~185 tok/s | 256K (full) | No price yet |
| NVIDIA RTX PRO 6000 Blackwell Workstation Edition | 96 GB | ~189 tok/s | 256K (full) | No price yet |
Every device that runs it
Graphics cards, Macs and mini PCs from the wider hardware list behind My rig, smallest usable memory first. Not sure about yours? Check your exact rig.
Show all 96 devices that run it
| Device | Type | Memory | Speed estimate | Max context estimate | Price |
|---|---|---|---|---|---|
| AMD Radeon RX 7900 XTX | GPU | 24 GB | ~137 tok/s | 33K · 63K (8-bit) | low $1,350 |
| Intel Arc Pro B60 | Workstation GPU | 24 GB | ~85 tok/s | 33K · 63K (8-bit) | No price yet |
| NVIDIA A10 | Data center GPU | 24 GB | ~131 tok/s | 33K · 63K (8-bit) | No price yet |
| NVIDIA GeForce RTX 3090 | GPU | 24 GB | ~157 tok/s | 33K · 63K (8-bit) | low $1,499 |
| NVIDIA GeForce RTX 3090 Ti | GPU | 24 GB | ~161 tok/s | 33K · 63K (8-bit) | No price yet |
| NVIDIA GeForce RTX 4090 | GPU | 24 GB | ~161 tok/s | 33K · 63K (8-bit) | low $2,910 |
| NVIDIA L4 | Data center GPU | 24 GB | ~90 tok/s | 33K · 63K (8-bit) | No price yet |
| NVIDIA Quadro RTX 6000 | Workstation GPU | 24 GB | ~138 tok/s | 33K · 63K (8-bit) | No price yet |
| NVIDIA RTX 4500 Ada Generation | Workstation GPU | 24 GB | ~111 tok/s | 33K · 63K (8-bit) | No price yet |
| NVIDIA RTX A5000 | Workstation GPU | 24 GB | ~146 tok/s | 33K · 63K (8-bit) | No price yet |
| NVIDIA RTX A5500 | Workstation GPU | 24 GB | ~146 tok/s | 33K · 63K (8-bit) | No price yet |
| NVIDIA RTX PRO 4000 Blackwell | Workstation GPU | 24 GB | ~138 tok/s | 33K · 63K (8-bit) | No price yet |
| NVIDIA RTX PRO 4000 SFF Blackwell | Workstation GPU | 24 GB | ~111 tok/s | 33K · 63K (8-bit) | No price yet |
| NVIDIA RTX Spark N1X laptop (6144-core GPU, 24GB) | Laptop | 24 GB | ~90 tok/s | 33K · 63K (8-bit) | No price yet |
| NVIDIA Tesla P40 | Data center GPU | 24 GB | ~81 tok/s | 33K · 63K (8-bit) | No price yet |
| NVIDIA TITAN RTX | GPU | 24 GB | ~138 tok/s | 33K · 63K (8-bit) | No price yet |
| Apple M3 Max (30-core GPU) 36GB | Mac | 36 GB (~24.1 usable) | ~69 tok/s | 38K · 72K (8-bit) | No price yet |
| Apple M3 Pro 36GB | Mac | 36 GB (~24.1 usable) | ~47 tok/s | 38K · 72K (8-bit) | No price yet |
| Apple M4 Max (32-core GPU) 36GB | Mac | 36 GB (~24.1 usable) | ~79 tok/s | 38K · 72K (8-bit) | No price yet |
| Apple Mac Studio M5 Max (32-core GPU, 36 GB) | Mac | 36 GB (~24.1 usable) | ~83 tok/s | 38K · 72K (8-bit) | low $2,499 |
| AMD Radeon PRO V710 | Data center GPU | 28 GB | ~98 tok/s | 224K · 256K (8-bit) | No price yet |
| AMD Radeon AI PRO R9600 | Workstation GPU | 32 GB | ~117 tok/s | 256K (full) | No price yet |
| AMD Radeon AI PRO R9600D | Workstation GPU | 32 GB | ~117 tok/s | 256K (full) | No price yet |
| AMD Radeon AI PRO R9700 | Workstation GPU | 32 GB | ~117 tok/s | 256K (full) | No price yet |
| AMD Radeon AI PRO R9700S | Workstation GPU | 32 GB | ~117 tok/s | 256K (full) | No price yet |
| AMD Radeon PRO V620 | Data center GPU | 32 GB | ~105 tok/s | 256K (full) | No price yet |
| AMD Radeon PRO W6800 | Workstation GPU | 32 GB | ~105 tok/s | 256K (full) | No price yet |
| AMD Radeon PRO W7800 32GB | Workstation GPU | 32 GB | ~111 tok/s | 256K (full) | No price yet |
| AMD Ryzen AI Max 385 (32GB) | Mini PC | 32 GB | ~70 tok/s | 256K (full) | No price yet |
| AMD Ryzen AI Max 390 (32GB) | Mini PC | 32 GB | ~70 tok/s | 256K (full) | No price yet |
| AMD Ryzen AI Max+ 395 (32GB) | Mini PC | 32 GB | ~70 tok/s | 256K (full) | No price yet |
| Intel Arc Pro B70 | Workstation GPU | 32 GB | ~100 tok/s | 256K (full) | No price yet |
| NVIDIA GeForce RTX 5090 | GPU | 32 GB | ~189 tok/s | 256K (full) | low $6,500 |
| NVIDIA Quadro GV100 | Workstation GPU | 32 GB | ~153 tok/s | 256K (full) | No price yet |
| NVIDIA RTX 5000 Ada Generation | Workstation GPU | 32 GB | ~129 tok/s | 256K (full) | No price yet |
| NVIDIA RTX PRO 4500 Blackwell | Workstation GPU | 32 GB | ~155 tok/s | 256K (full) | No price yet |
| NVIDIA RTX Spark N1X laptop (6144-core GPU, 32GB) | Laptop | 32 GB | ~90 tok/s | 256K (full) | No price yet |
| NVIDIA V100 PCIe 32GB | Data center GPU | 32 GB | ~155 tok/s | 256K (full) | No price yet |
| NVIDIA V100S PCIe 32GB | Data center GPU | 32 GB | ~168 tok/s | 256K (full) | No price yet |
| Apple M3 Max (40-core GPU) 48GB | Mac | 48 GB (~36 usable) | ~79 tok/s | 256K (full) | No price yet |
| Apple M4 Max (40-core GPU) 48GB | Mac | 48 GB (~36 usable) | ~88 tok/s | 256K (full) | No price yet |
| Apple M4 Pro 48GB | Mac | 48 GB (~36 usable) | ~66 tok/sllama.cpp-style estimateMLX (measured, community), with speculative decoding (MTP/DFlash): 43.3–112.1 tok/s (runs 1, 2) at 16K–128K (M4 Pro 20c) | 256K (full) | No price yet |
| Apple M5 Max (40-core GPU) 48GB | Mac | 48 GB (~36 usable) | ~92 tok/s | 256K (full) | No price yet |
| Apple M5 Pro 48GB | Mac | 48 GB (~36 usable) | ~70 tok/sllama.cpp-style estimateMLX (measured, community): 106.2 tok/s at 4K (M5 Pro 20c)With speculative decoding (MTP/DFlash): 55.7 tok/s at 16K (M5 Pro 16c)GPU cores differ between runs (M5 Pro 16c / M5 Pro 20c); listed here by chip and memory. | 256K (full) | No price yet |
| NVIDIA A100 40GB PCIe | Data center GPU | 40 GB | ~183 tok/s | 256K (full) | No price yet |
| NVIDIA A800 40GB Active | Workstation GPU | 40 GB | ~183 tok/s | 256K (full) | No price yet |
| AMD Radeon PRO W7800 48GB | Workstation GPU | 48 GB | ~132 tok/s | 256K (full) | No price yet |
| AMD Radeon PRO W7900 | Workstation GPU | 48 GB | ~132 tok/s | 256K (full) | No price yet |
| AMD Radeon PRO W7900 Dual Slot | Workstation GPU | 48 GB | ~132 tok/s | 256K (full) | No price yet |
| Apple M1 Max 64GB | Mac | 64 GB (~48 usable) | ~79 tok/s | 256K (full) | No price yet |
| Apple M1 Ultra 64GB | Mac | 64 GB (~48 usable) | ~99 tok/s | 256K (full) | No price yet |
| Apple M2 Max (30-core GPU) 64GB | Mac | 64 GB (~48 usable) | ~79 tok/s | 256K (full) | No price yet |
| Apple M2 Max (38-core GPU) 64GB | Mac | 64 GB (~48 usable) | ~79 tok/s | 256K (full) | No price yet |
| Apple M2 Ultra 64GB | Mac | 64 GB (~48 usable) | ~99 tok/sllama.cpp-style estimateMLX (measured, community): 54.8 tok/s at 128K (M2 Ultra 60c) | 256K (full) | No price yet |
| Apple M3 Max (40-core GPU) 64GB | Mac | 64 GB (~48 usable) | ~79 tok/s | 256K (full) | No price yet |
| Apple M4 Max (40-core GPU) 64GB | Mac | 64 GB (~48 usable) | ~88 tok/s | 256K (full) | No price yet |
| Apple M4 Pro 64GB | Mac | 64 GB (~48 usable) | ~66 tok/s | 256K (full) | No price yet |
| Apple M5 Pro 64GB | Mac | 64 GB (~48 usable) | ~70 tok/s | 256K (full) | No price yet |
| Apple Mac Studio M5 Max (40-core GPU, 64 GB) | Mac | 64 GB (~48 usable) | ~92 tok/sllama.cpp-style estimateMLX (measured, community), with speculative decoding (MTP/DFlash): 89.5–125.4 tok/s at 1K–4K (M5 Max 40c) | 256K (full) | No price yet |
| NVIDIA A40 | Data center GPU | 48 GB | ~140 tok/s | 256K (full) | No price yet |
| NVIDIA L40 | Data center GPU | 48 GB | ~153 tok/s | 256K (full) | No price yet |
| NVIDIA L40S | Data center GPU | 48 GB | ~153 tok/s | 256K (full) | No price yet |
| NVIDIA Quadro RTX 8000 | Workstation GPU | 48 GB | ~138 tok/s | 256K (full) | No price yet |
| NVIDIA RTX 6000 Ada Generation | Workstation GPU | 48 GB | ~159 tok/s | 256K (full) | No price yet |
| NVIDIA RTX A6000 | Workstation GPU | 48 GB | ~146 tok/s | 256K (full) | low $8,363 |
| NVIDIA RTX PRO 5000 Blackwell 48GB | Workstation GPU | 48 GB | ~176 tok/s | 256K (full) | No price yet |
| NVIDIA RTX Spark N1X laptop (6144-core GPU, 48GB) | Laptop | 48 GB | ~90 tok/s | 256K (full) | No price yet |
| AMD Ryzen AI Max 385 (64GB) | Mini PC | 64 GB | ~70 tok/s | 256K (full) | No price yet |
| AMD Ryzen AI Max 390 (64GB) | Mini PC | 64 GB | ~70 tok/s | 256K (full) | No price yet |
| AMD Ryzen AI Max+ 395 (64GB) | Mini PC | 64 GB | ~70 tok/s | 256K (full) | low $2,199.99 |
| NVIDIA DGX Spark (64GB) | Mini PC | 64 GB | ~84 tok/s | 256K (full) | low $4,999 |
| NVIDIA RTX Spark N1X laptop (6144-core GPU, 64GB) | Laptop | 64 GB | ~90 tok/s | 256K (full) | No price yet |
| Apple M2 Max (38-core GPU) 96GB | Mac | 96 GB (~72 usable) | ~79 tok/s | 256K (full) | No price yet |
| Apple M3 Max (30-core GPU) 96GB | Mac | 96 GB (~72 usable) | ~69 tok/s | 256K (full) | No price yet |
| Apple M3 Ultra 96GB | Mac | 96 GB (~72 usable) | ~99 tok/sllama.cpp-style estimateMLX (measured, community): 100.2 tok/s at 1K (M3 Ultra 60c) | 256K (full) | No price yet |
| Apple Mac Studio M5 Ultra (64-core GPU, 96 GB) | Mac | 96 GB (~72 usable) | ~108 tok/sllama.cpp-style estimateMLX (measured, community), with speculative decoding (MTP/DFlash): 74.4–213.5 tok/s (runs 1, 2, 3, 4) at 1K–195K (M5 Ultra 80c) | 256K (full) | low $5,499 |
| NVIDIA RTX PRO 5000 Blackwell 72GB | Workstation GPU | 72 GB | ~176 tok/s | 256K (full) | No price yet |
| NVIDIA A100 80GB PCIe | Data center GPU | 80 GB | ~193 tok/s | 256K (full) | No price yet |
| Apple M1 Ultra 128GB | Mac | 128 GB (~96 usable) | ~99 tok/s | 256K (full) | No price yet |
| Apple M2 Ultra 128GB | Mac | 128 GB (~96 usable) | ~99 tok/s | 256K (full) | No price yet |
| Apple M3 Max (40-core GPU) 128GB | Mac | 128 GB (~96 usable) | ~79 tok/s | 256K (full) | No price yet |
| Apple M4 Max (40-core GPU) 128GB | Mac | 128 GB (~96 usable) | ~88 tok/sllama.cpp-style estimateMLX (measured, community), with speculative decoding (MTP/DFlash): 54.7–163.9 tok/s at 1K–195K (M4 Max 40c) | 256K (full) | No price yet |
| Apple M5 Max (40-core GPU) 128GB | Mac | 128 GB (~96 usable) | ~92 tok/sllama.cpp-style estimateMLX (measured, community): 82.8–82.9 tok/s (runs 1, 2) at 4K (M5 Max 40c)With speculative decoding (MTP/DFlash): 87.5–119.2 tok/s at 4K–32K (M5 Max 40c) | 256K (full) | No price yet |
| NVIDIA RTX PRO 6000 Blackwell Max-Q | Workstation GPU | 96 GB | ~189 tok/s | 256K (full) | No price yet |
| NVIDIA RTX PRO 6000 Blackwell Server Edition | Data center GPU | 96 GB | ~185 tok/s | 256K (full) | No price yet |
| NVIDIA RTX PRO 6000 Blackwell Workstation Edition | Workstation GPU | 96 GB | ~189 tok/s | 256K (full) | No price yet |
| AMD Ryzen AI Max 385 (128GB) | Mini PC | 128 GB | ~70 tok/s | 256K (full) | No price yet |
| AMD Ryzen AI Max 390 (128GB) | Mini PC | 128 GB | ~70 tok/s | 256K (full) | No price yet |
| AMD Ryzen AI Max+ 395 (128GB) | Mini PC | 128 GB | ~70 tok/s | 256K (full) | low $3,499.99 |
| NVIDIA DGX Spark (128GB) | Mini PC | 128 GB | ~84 tok/s | 256K (full) | low $6,949.99 |
| NVIDIA RTX Spark N1X laptop (6144-core GPU, 128GB) | Laptop | 128 GB | ~90 tok/s | 256K (full) | No price yet |
| Apple M2 Ultra 192GB | Mac | 192 GB (~144 usable) | ~99 tok/s | 256K (full) | No price yet |
| Apple M3 Ultra 256GB | Mac | 256 GB (~192 usable) | ~99 tok/s | 256K (full) | No price yet |
| Apple M5 Ultra 256GB | Mac | 256 GB (~192 usable) | ~108 tok/s | 256K (full) | No price yet |
| Framework Desktop, AMD Ryzen AI Max+ PRO 495 (Strix Halo), 192 GB | Mini PC | 192 GB | ~73 tok/s | 256K (full) | low $6,799 |
| Apple M5 Ultra 512GB | Mac | 512 GB (~384 usable) | ~108 tok/s | 256K (full) | No price yet |
Multi-GPU: cards that need two or more
| Cards | Total VRAM | Speed estimate | Max context estimate |
|---|---|---|---|
| 2 × Intel Arc B580 | 24 GB | ~85 tok/s | 33K · 63K (8-bit) |
| 2 × NVIDIA GeForce RTX 3060 12 GB | 24 GB | ~100 tok/s | 33K · 63K (8-bit) |
| 2 × NVIDIA GeForce RTX 4070 SUPER | 24 GB | ~121 tok/s | 33K · 63K (8-bit) |
| 3 × NVIDIA GeForce RTX 5060 | 24 GB | ~113 tok/s | 33K · 63K (8-bit) |
| 3 × NVIDIA GeForce RTX 5060 Ti 8 GB | 24 GB | ~113 tok/s | 33K · 63K (8-bit) |
| 2 × NVIDIA GeForce RTX 5070 | 24 GB | ~138 tok/s | 33K · 63K (8-bit) |
| 2 × AMD Radeon RX 7800 XT | 32 GB | ~115 tok/s | 256K (full) |
| 2 × AMD Radeon RX 9060 XT 16 GB | 32 GB | ~81 tok/s | 256K (full) |
| 2 × AMD Radeon RX 9070 | 32 GB | ~117 tok/s | 256K (full) |
| 2 × AMD Radeon RX 9070 XT | 32 GB | ~117 tok/s | 256K (full) |
| 2 × NVIDIA GeForce RTX 5060 Ti 16 GB | 32 GB | ~113 tok/s | 256K (full) |
| 2 × NVIDIA GeForce RTX 5070 Ti | 32 GB | ~155 tok/s | 256K (full) |
| 2 × NVIDIA GeForce RTX 5080 | 32 GB | ~159 tok/s | 256K (full) |
Measured on Macs with MLX (community)
MLX (measured, community): decode speeds people ran with MLX through oMLX and submitted to its public leaderboard. They are kept apart from this site's llama.cpp-style estimate, which is for a GGUF file in llama.cpp and is unchanged. The plain figure has no speculative decoding, at the shortest context the run used; runs with speculative decoding (MTP/DFlash) are shown separately, as a range, because their speed depends on how many drafted tokens are accepted, which varies with the prompt.
| Mac | llama.cpp-style estimate | MLX plain decode | With speculative decoding (MTP/DFlash) | Long context, one run |
|---|---|---|---|---|
| Apple M4 Pro 48GB | ~66 tok/s | — | 43.3–112.1 tok/s (runs 1, 2) at 16K–128K (M4 Pro 20c) | 64K 61.4 → 128K 43.3 tok/s; prefill 325–499 tok/s (M4 Pro 20c, speculative run) |
| Apple M5 Pro 48GBGPU cores differ between runs (M5 Pro 16c / M5 Pro 20c); listed here by chip and memory. | ~70 tok/s | 106.2 tok/s at 4K (M5 Pro 20c) | 55.7 tok/s at 16K (M5 Pro 16c) | 4K 106.2 → 16K 94.5 tok/s; prefill 1508–2975 tok/s (M5 Pro 20c) |
| Apple M2 Ultra 64GB | ~99 tok/s | 54.8 tok/s at 128K (M2 Ultra 60c) | — | — |
| Apple M5 Max (40-core GPU) 64GB | ~92 tok/s | — | 89.5–125.4 tok/s at 1K–4K (M5 Max 40c) | 1K 89.5 → 4K 125.4 tok/s; prefill 1139–3270 tok/s (M5 Max 40c, speculative run) |
| Apple M3 Ultra 96GB | ~99 tok/s | 100.2 tok/s at 1K (M3 Ultra 60c) | — | 1K 100.2 → 32K 81.7 tok/s; prefill 1819–2602 tok/s (M3 Ultra 60c) |
| Apple M5 Ultra 96GB | ~108 tok/s | — | 74.4–213.5 tok/s (runs 1, 2, 3, 4) at 1K–195K (M5 Ultra 80c) | 1K 168.2 → 195K 132.8 tok/s; prefill 3301–9201 tok/s (M5 Ultra 80c, speculative run) |
| Apple M4 Max (40-core GPU) 128GB | ~88 tok/s | — | 54.7–163.9 tok/s at 1K–195K (M4 Max 40c) | 1K 100.9 → 195K 54.7 tok/s; prefill 464–1966 tok/s (M4 Max 40c, speculative run) |
| Apple M5 Max (40-core GPU) 128GB | ~92 tok/s | 82.8–82.9 tok/s (runs 1, 2) at 4K (M5 Max 40c) | 87.5–119.2 tok/s at 4K–32K (M5 Max 40c) | 4K 82.8 → 32K 69.5 tok/s; prefill 1842–2908 tok/s (M5 Max 40c) |
Long context: decode at the shortest and longest context of one run, and that run's prompt-processing (prefill) range. Agent jobs re-read long prompts, so prefill (prompt processing) speed matters as much as decode there. Community-submitted to oMLX, accessed 2026-10-26; each figure links its run(s). MLX 4-bit files, not the GGUF the estimate uses. About the MLX data
Speed calculator
estimateEstimate = bandwidth ÷ GB read per token (the model's listed GGUF file: Q4_K_M for most models, otherwise the quant named in the list; scaled by active/total parameters for MoE), scaled by a per-platform efficiency and fixed per-token costs calibrated to 45 measured llama.cpp benchmarks (2025–26). Fit = file size + 1.5 GB headroom ≤ total VRAM (on Macs, the part of unified memory macOS lets the GPU use by default), and room for at least 4K tokens of context with an 8-bit KV cache. For a model with no KV cache data, only the file and headroom are checked, so a fit says nothing about context. Multi-GPU: VRAM adds, speed stays one card's (a layer split runs the cards in turn); max context uses the total. Speed is for short context. Max context = what is left after the file and a 1 GB runtime reserve ÷ KV cache bytes per token, default FP16 and 8-bit (q8_0) cache. How speed works · how context works
Related
- Can it runCan the NVIDIA GeForce RTX 3090 run Qwen3.6-35B-A3B?
- Can it runCan the AMD Radeon RX 7900 XTX run Qwen3.6-35B-A3B?
- Can it runCan the NVIDIA GeForce RTX 5090 run Qwen3.6-35B-A3B?
- Can it runCan the NVIDIA RTX A6000 run Qwen3.6-35B-A3B?
- BuildBuild that fits it: 24 GB: one used RTX 3090
- BuildBuild that fits it: 32 GB: two Radeon RX 9070
- BuildBuild that fits it: 48 GB: dual used RTX 3090