RigPerDollar

What GPU do you need to run Qwen3.8-27B?

A new Radeon RX 7900 XTX ($1,350). It is the cheapest priced graphics card that holds Qwen3.8-27B (about 18.9 GB at Q4) with room for about 82K tokens of context (155K with 8-bit KV cache), at about 38 tok/s estimate; that is the card price only, and it requires a separate compatible PC. 58 of the 157 GPUs and systems we track fit it.

Parameters27.78B
Q4_K_M GGUF17.44 GBsource
VRAM needed~18.9 GBfile + headroom
Single GPUs that fit58
ReleasedAug 5, 2026
LicenseApache 2.0
Context window256K tokens

27.78B parameters (Hugging Face, accessed 2026-10-16). Q4_K_M GGUF: 17.44 GB (source). Architecture config (Hugging Face config.json): 16 attention layers (the rest are recurrent) × 4 KV heads × 256 head dim, so about 64 KB of FP16 KV cache per token of context (how we estimate context).

Other places to get it

Checked Nov 14, 2026. We list only copies we opened and checked. Check file hashes for copies from other uploaders.

Single GPUs that fit, cheapest first

Single GPU that fitsVRAMSpeed estimateMax context estimatePrice
AMD Radeon RX 7900 XTX24 GB~38 tok/s82K · 155K (8-bit)low $1,350avg $1,552 · new · source, 2026-11-02
NVIDIA GeForce RTX 309024 GB~40 tok/s82K · 155K (8-bit)low $1,499avg $1,551 · used · source, 2026-11-02
Apple Mac Studio M5 Max (32-core GPU, 36 GB)36 GB (~24.1 usable)~21 tok/s84K · 158K (8-bit)low $2,499new · source, 2026-10-16
NVIDIA GeForce RTX 409024 GB~43 tok/s82K · 155K (8-bit)low $2,910avg $3,100 · used · source, 2026-11-02
NVIDIA DGX Spark (64GB)64 GB~13 tok/s256K (full)low $4,999new · source, 2026-11-13
Apple Mac Studio M5 Ultra (64-core GPU, 96 GB)96 GB (~72 usable)~48 tok/sllama.cpp-style estimateMLX (measured, community): 55.1 tok/s at 1K (M5 Ultra 80c)256K (full)low $5,499new · source, 2026-10-23
NVIDIA GeForce RTX 509032 GB~68 tok/s202K · 256K (8-bit)low $6,500avg $8,757 · new · source, 2026-11-02
Framework Desktop, AMD Ryzen AI Max+ PRO 495 (Strix Halo), 192 GB192 GB~12 tok/s256K (full)low $6,799new · source, 2026-10-16
NVIDIA DGX Spark (128GB)128 GB~13 tok/s256K (full)low $6,949.99new · source, 2026-10-23
NVIDIA RTX A600048 GB~34 tok/s256K (full)low $8,363new · source, 2026-11-02
AMD Radeon RX 7900 XT20 GB~33 tok/s23K · 43K (8-bit)No price yet
NVIDIA RTX 4000 Ada Generation20 GB~17 tok/s23K · 43K (8-bit)No price yet
NVIDIA RTX 4000 SFF Ada Generation20 GB~13 tok/s23K · 43K (8-bit)No price yet
NVIDIA RTX A450020 GB~29 tok/s23K · 43K (8-bit)No price yet
Intel Arc Pro B6024 GB~15 tok/s82K · 155K (8-bit)No price yet
NVIDIA A1024 GB~27 tok/s82K · 155K (8-bit)No price yet
NVIDIA GeForce RTX 3090 Ti24 GB~43 tok/s82K · 155K (8-bit)No price yet
NVIDIA L424 GB~14 tok/s82K · 155K (8-bit)No price yet
NVIDIA Quadro RTX 600024 GB~30 tok/s82K · 155K (8-bit)No price yet
NVIDIA RTX 4500 Ada Generation24 GB~20 tok/s82K · 155K (8-bit)No price yet
NVIDIA RTX A500024 GB~34 tok/s82K · 155K (8-bit)No price yet
NVIDIA RTX A550024 GB~34 tok/s82K · 155K (8-bit)No price yet
NVIDIA RTX PRO 4000 Blackwell24 GB~30 tok/s82K · 155K (8-bit)No price yet
NVIDIA RTX PRO 4000 SFF Blackwell24 GB~20 tok/s82K · 155K (8-bit)No price yet
NVIDIA Tesla P4024 GB~12 tok/s82K · 155K (8-bit)No price yet
NVIDIA TITAN RTX24 GB~30 tok/s82K · 155K (8-bit)No price yet
AMD Radeon PRO V71028 GB~19 tok/s142K · 256K (8-bit)No price yet
AMD Radeon AI PRO R960032 GB~26 tok/s202K · 256K (8-bit)No price yet
AMD Radeon AI PRO R9600D32 GB~26 tok/s202K · 256K (8-bit)No price yet
AMD Radeon AI PRO R970032 GB~26 tok/s202K · 256K (8-bit)No price yet
AMD Radeon AI PRO R9700S32 GB~26 tok/s202K · 256K (8-bit)No price yet
AMD Radeon PRO V62032 GB~21 tok/s202K · 256K (8-bit)No price yet
AMD Radeon PRO W680032 GB~21 tok/s202K · 256K (8-bit)No price yet
AMD Radeon PRO W7800 32GB32 GB~24 tok/s202K · 256K (8-bit)No price yet
Intel Arc Pro B7032 GB~20 tok/s202K · 256K (8-bit)No price yet
NVIDIA Quadro GV10032 GB~38 tok/s202K · 256K (8-bit)No price yet
NVIDIA RTX 5000 Ada Generation32 GB~26 tok/s202K · 256K (8-bit)No price yet
NVIDIA RTX PRO 4500 Blackwell32 GB~39 tok/s202K · 256K (8-bit)No price yet
NVIDIA V100 PCIe 32GB32 GB~39 tok/s202K · 256K (8-bit)No price yet
NVIDIA V100S PCIe 32GB32 GB~47 tok/s202K · 256K (8-bit)No price yet
Apple M5 Pro 48GB48 GB (~36 usable)~15 tok/sllama.cpp-style estimateMLX (measured, community): 19.0 tok/s at 1K (M5 Pro 20c)With speculative decoding (MTP/DFlash): 18.8–18.9 tok/s at 1K–4K (M5 Pro 20c)GPU cores differ between runs (M5 Pro 16c / M5 Pro 20c); listed here by chip and memory.256K (full)No price yet
NVIDIA A100 40GB PCIe40 GB~61 tok/s256K (full)No price yet
NVIDIA A800 40GB Active40 GB~61 tok/s256K (full)No price yet
AMD Radeon PRO W7800 48GB48 GB~35 tok/s256K (full)No price yet
AMD Radeon PRO W790048 GB~35 tok/s256K (full)No price yet
AMD Radeon PRO W7900 Dual Slot48 GB~35 tok/s256K (full)No price yet
Apple Mac Studio M5 Max (40-core GPU, 64 GB)64 GB (~48 usable)~28 tok/sllama.cpp-style estimateMLX (measured, community), with speculative decoding (MTP/DFlash): 27.2–119.8 tok/s (runs 1, 2) at 1K–195K (M5 Max 40c)256K (full)No price yet
NVIDIA A4048 GB~31 tok/s256K (full)No price yet
NVIDIA L4048 GB~38 tok/s256K (full)No price yet
NVIDIA L40S48 GB~38 tok/s256K (full)No price yet
NVIDIA Quadro RTX 800048 GB~30 tok/s256K (full)No price yet
NVIDIA RTX 6000 Ada Generation48 GB~41 tok/s256K (full)No price yet
NVIDIA RTX PRO 5000 Blackwell 48GB48 GB~55 tok/s256K (full)No price yet
NVIDIA RTX PRO 5000 Blackwell 72GB72 GB~55 tok/s256K (full)No price yet
NVIDIA A100 80GB PCIe80 GB~72 tok/s256K (full)No price yet
NVIDIA RTX PRO 6000 Blackwell Max-Q96 GB~68 tok/s256K (full)No price yet
NVIDIA RTX PRO 6000 Blackwell Server Edition96 GB~63 tok/s256K (full)No price yet
NVIDIA RTX PRO 6000 Blackwell Workstation Edition96 GB~68 tok/s256K (full)No price yet

Every device that runs it

Graphics cards, Macs and mini PCs from the wider hardware list behind My rig, smallest usable memory first. Not sure about yours? Check your exact rig.

Show all 108 devices that run it
DeviceTypeMemorySpeed estimateMax context estimatePrice
AMD Radeon RX 7900 XTGPU20 GB~33 tok/s23K · 43K (8-bit)No price yet
NVIDIA RTX 4000 Ada GenerationWorkstation GPU20 GB~17 tok/s23K · 43K (8-bit)No price yet
NVIDIA RTX 4000 SFF Ada GenerationWorkstation GPU20 GB~13 tok/s23K · 43K (8-bit)No price yet
NVIDIA RTX A4500Workstation GPU20 GB~29 tok/s23K · 43K (8-bit)No price yet
Apple M1 Max 32GBMac32 GB (~21.4 usable)~19 tok/s44K · 83K (8-bit)No price yet
Apple M1 Pro 32GBMac32 GB (~21.4 usable)~10 tok/s44K · 83K (8-bit)No price yet
Apple M2 Max (30-core GPU) 32GBMac32 GB (~21.4 usable)~19 tok/s44K · 83K (8-bit)No price yet
Apple M2 Max (38-core GPU) 32GBMac32 GB (~21.4 usable)~19 tok/s44K · 83K (8-bit)No price yet
Apple M2 Pro 32GBMac32 GB (~21.4 usable)~10 tok/s44K · 83K (8-bit)No price yet
Apple M4 32GBMac32 GB (~21.4 usable)~6 tok/s44K · 83K (8-bit)No price yet
Apple M5 32GBMac32 GB (~21.4 usable)~8 tok/s44K · 83K (8-bit)No price yet
Apple M6 32GBMac32 GB (~21.4 usable)~8 tok/s44K · 83K (8-bit)No price yet
AMD Radeon RX 7900 XTXGPU24 GB~38 tok/s82K · 155K (8-bit)low $1,350avg $1,552 · new · source, 2026-11-02
Intel Arc Pro B60Workstation GPU24 GB~15 tok/s82K · 155K (8-bit)No price yet
NVIDIA A10Data center GPU24 GB~27 tok/s82K · 155K (8-bit)No price yet
NVIDIA GeForce RTX 3090GPU24 GB~40 tok/s82K · 155K (8-bit)low $1,499avg $1,551 · used · source, 2026-11-02
NVIDIA GeForce RTX 3090 TiGPU24 GB~43 tok/s82K · 155K (8-bit)No price yet
NVIDIA GeForce RTX 4090GPU24 GB~43 tok/s82K · 155K (8-bit)low $2,910avg $3,100 · used · source, 2026-11-02
NVIDIA L4Data center GPU24 GB~14 tok/s82K · 155K (8-bit)No price yet
NVIDIA Quadro RTX 6000Workstation GPU24 GB~30 tok/s82K · 155K (8-bit)No price yet
NVIDIA RTX 4500 Ada GenerationWorkstation GPU24 GB~20 tok/s82K · 155K (8-bit)No price yet
NVIDIA RTX A5000Workstation GPU24 GB~34 tok/s82K · 155K (8-bit)No price yet
NVIDIA RTX A5500Workstation GPU24 GB~34 tok/s82K · 155K (8-bit)No price yet
NVIDIA RTX PRO 4000 BlackwellWorkstation GPU24 GB~30 tok/s82K · 155K (8-bit)No price yet
NVIDIA RTX PRO 4000 SFF BlackwellWorkstation GPU24 GB~20 tok/s82K · 155K (8-bit)No price yet
NVIDIA RTX Spark N1X laptop (6144-core GPU, 24GB)Laptop24 GB~14 tok/s82K · 155K (8-bit)No price yet
NVIDIA Tesla P40Data center GPU24 GB~12 tok/s82K · 155K (8-bit)No price yet
NVIDIA TITAN RTXGPU24 GB~30 tok/s82K · 155K (8-bit)No price yet
Apple M3 Max (30-core GPU) 36GBMac36 GB (~24.1 usable)~14 tok/sllama.cpp-style estimateMLX (measured, community): 16.0 tok/s at 4K (M3 Max 30c)With speculative decoding (MTP/DFlash): 17.8–21.3 tok/s at 1K–4K (M3 Max 30c)84K · 158K (8-bit)No price yet
Apple M3 Pro 36GBMac36 GB (~24.1 usable)~7 tok/s84K · 158K (8-bit)No price yet
Apple M4 Max (32-core GPU) 36GBMac36 GB (~24.1 usable)~19 tok/sllama.cpp-style estimateMLX (measured, community): 20.1 tok/s at 4K (M4 Max 32c)84K · 158K (8-bit)No price yet
Apple Mac Studio M5 Max (32-core GPU, 36 GB)Mac36 GB (~24.1 usable)~21 tok/s84K · 158K (8-bit)low $2,499new · source, 2026-10-16
AMD Radeon PRO V710Data center GPU28 GB~19 tok/s142K · 256K (8-bit)No price yet
AMD Radeon AI PRO R9600Workstation GPU32 GB~26 tok/s202K · 256K (8-bit)No price yet
AMD Radeon AI PRO R9600DWorkstation GPU32 GB~26 tok/s202K · 256K (8-bit)No price yet
AMD Radeon AI PRO R9700Workstation GPU32 GB~26 tok/s202K · 256K (8-bit)No price yet
AMD Radeon AI PRO R9700SWorkstation GPU32 GB~26 tok/s202K · 256K (8-bit)No price yet
AMD Radeon PRO V620Data center GPU32 GB~21 tok/s202K · 256K (8-bit)No price yet
AMD Radeon PRO W6800Workstation GPU32 GB~21 tok/s202K · 256K (8-bit)No price yet
AMD Radeon PRO W7800 32GBWorkstation GPU32 GB~24 tok/s202K · 256K (8-bit)No price yet
AMD Ryzen AI Max 385 (32GB)Mini PC32 GB~11 tok/s202K · 256K (8-bit)No price yet
AMD Ryzen AI Max 390 (32GB)Mini PC32 GB~11 tok/s202K · 256K (8-bit)No price yet
AMD Ryzen AI Max+ 395 (32GB)Mini PC32 GB~11 tok/s202K · 256K (8-bit)No price yet
Intel Arc Pro B70Workstation GPU32 GB~20 tok/s202K · 256K (8-bit)No price yet
NVIDIA GeForce RTX 5090GPU32 GB~68 tok/s202K · 256K (8-bit)low $6,500avg $8,757 · new · source, 2026-11-02
NVIDIA Quadro GV100Workstation GPU32 GB~38 tok/s202K · 256K (8-bit)No price yet
NVIDIA RTX 5000 Ada GenerationWorkstation GPU32 GB~26 tok/s202K · 256K (8-bit)No price yet
NVIDIA RTX PRO 4500 BlackwellWorkstation GPU32 GB~39 tok/s202K · 256K (8-bit)No price yet
NVIDIA RTX Spark N1X laptop (6144-core GPU, 32GB)Laptop32 GB~14 tok/s202K · 256K (8-bit)No price yet
NVIDIA V100 PCIe 32GBData center GPU32 GB~39 tok/s202K · 256K (8-bit)No price yet
NVIDIA V100S PCIe 32GBData center GPU32 GB~47 tok/s202K · 256K (8-bit)No price yet
Apple M3 Max (40-core GPU) 48GBMac48 GB (~36 usable)~19 tok/s256K (full)No price yet
Apple M4 Max (40-core GPU) 48GBMac48 GB (~36 usable)~25 tok/s256K (full)No price yet
Apple M4 Pro 48GBMac48 GB (~36 usable)~13 tok/sllama.cpp-style estimateMLX (measured, community): 15.6 tok/s at 1K (M4 Pro 16c)With speculative decoding (MTP/DFlash): 22.9–32.2 tok/s at 4K–16K (M4 Pro 16c)256K (full)No price yet
Apple M5 Max (40-core GPU) 48GBMac48 GB (~36 usable)~28 tok/s256K (full)No price yet
Apple M5 Pro 48GBMac48 GB (~36 usable)~15 tok/sllama.cpp-style estimateMLX (measured, community): 19.0 tok/s at 1K (M5 Pro 20c)With speculative decoding (MTP/DFlash): 18.8–18.9 tok/s at 1K–4K (M5 Pro 20c)GPU cores differ between runs (M5 Pro 16c / M5 Pro 20c); listed here by chip and memory.256K (full)No price yet
NVIDIA A100 40GB PCIeData center GPU40 GB~61 tok/s256K (full)No price yet
NVIDIA A800 40GB ActiveWorkstation GPU40 GB~61 tok/s256K (full)No price yet
AMD Radeon PRO W7800 48GBWorkstation GPU48 GB~35 tok/s256K (full)No price yet
AMD Radeon PRO W7900Workstation GPU48 GB~35 tok/s256K (full)No price yet
AMD Radeon PRO W7900 Dual SlotWorkstation GPU48 GB~35 tok/s256K (full)No price yet
Apple M1 Max 64GBMac64 GB (~48 usable)~19 tok/s256K (full)No price yet
Apple M1 Ultra 64GBMac64 GB (~48 usable)~35 tok/s256K (full)No price yet
Apple M2 Max (30-core GPU) 64GBMac64 GB (~48 usable)~19 tok/s256K (full)No price yet
Apple M2 Max (38-core GPU) 64GBMac64 GB (~48 usable)~19 tok/s256K (full)No price yet
Apple M2 Ultra 64GBMac64 GB (~48 usable)~35 tok/s256K (full)No price yet
Apple M3 Max (40-core GPU) 64GBMac64 GB (~48 usable)~19 tok/sllama.cpp-style estimateMLX (measured, community), with speculative decoding (MTP/DFlash): 9.9–13.9 tok/s at 1K–16K (M3 Max 40c)256K (full)No price yet
Apple M4 Max (40-core GPU) 64GBMac64 GB (~48 usable)~25 tok/sllama.cpp-style estimateMLX (measured, community): 29.3 tok/s at 1K (M4 Max 40c)With speculative decoding (MTP/DFlash): 63.7–68.7 tok/s (runs 1, 2) at 8K–16K (M4 Max 40c)256K (full)No price yet
Apple M4 Pro 64GBMac64 GB (~48 usable)~13 tok/s256K (full)No price yet
Apple M5 Pro 64GBMac64 GB (~48 usable)~15 tok/sllama.cpp-style estimateMLX (measured, community): 17.8 tok/s at 1K (M5 Pro 20c)With speculative decoding (MTP/DFlash): 26.8–28.2 tok/s at 64K–128K (M5 Pro 16c)GPU cores differ between runs (M5 Pro 16c / M5 Pro 20c); listed here by chip and memory.256K (full)No price yet
Apple Mac Studio M5 Max (40-core GPU, 64 GB)Mac64 GB (~48 usable)~28 tok/sllama.cpp-style estimateMLX (measured, community), with speculative decoding (MTP/DFlash): 27.2–119.8 tok/s (runs 1, 2) at 1K–195K (M5 Max 40c)256K (full)No price yet
NVIDIA A40Data center GPU48 GB~31 tok/s256K (full)No price yet
NVIDIA L40Data center GPU48 GB~38 tok/s256K (full)No price yet
NVIDIA L40SData center GPU48 GB~38 tok/s256K (full)No price yet
NVIDIA Quadro RTX 8000Workstation GPU48 GB~30 tok/s256K (full)No price yet
NVIDIA RTX 6000 Ada GenerationWorkstation GPU48 GB~41 tok/s256K (full)No price yet
NVIDIA RTX A6000Workstation GPU48 GB~34 tok/s256K (full)low $8,363new · source, 2026-11-02
NVIDIA RTX PRO 5000 Blackwell 48GBWorkstation GPU48 GB~55 tok/s256K (full)No price yet
NVIDIA RTX Spark N1X laptop (6144-core GPU, 48GB)Laptop48 GB~14 tok/s256K (full)No price yet
AMD Ryzen AI Max 385 (64GB)Mini PC64 GB~11 tok/s256K (full)No price yet
AMD Ryzen AI Max 390 (64GB)Mini PC64 GB~11 tok/s256K (full)No price yet
AMD Ryzen AI Max+ 395 (64GB)Mini PC64 GB~11 tok/s256K (full)low $2,199.99new · source, 2026-11-13
NVIDIA DGX Spark (64GB)Mini PC64 GB~13 tok/s256K (full)low $4,999new · source, 2026-11-13
NVIDIA RTX Spark N1X laptop (6144-core GPU, 64GB)Laptop64 GB~14 tok/s256K (full)No price yet
Apple M2 Max (38-core GPU) 96GBMac96 GB (~72 usable)~19 tok/s256K (full)No price yet
Apple M3 Max (30-core GPU) 96GBMac96 GB (~72 usable)~14 tok/s256K (full)No price yet
Apple M3 Ultra 96GBMac96 GB (~72 usable)~35 tok/s256K (full)No price yet
Apple Mac Studio M5 Ultra (64-core GPU, 96 GB)Mac96 GB (~72 usable)~48 tok/sllama.cpp-style estimateMLX (measured, community): 55.1 tok/s at 1K (M5 Ultra 80c)256K (full)low $5,499new · source, 2026-10-23
NVIDIA RTX PRO 5000 Blackwell 72GBWorkstation GPU72 GB~55 tok/s256K (full)No price yet
NVIDIA A100 80GB PCIeData center GPU80 GB~72 tok/s256K (full)No price yet
Apple M1 Ultra 128GBMac128 GB (~96 usable)~35 tok/s256K (full)No price yet
Apple M2 Ultra 128GBMac128 GB (~96 usable)~35 tok/s256K (full)No price yet
Apple M3 Max (40-core GPU) 128GBMac128 GB (~96 usable)~19 tok/sllama.cpp-style estimateMLX (measured, community), with speculative decoding (MTP/DFlash): 21.4–34.6 tok/s at 1K–4K (M3 Max 40c)256K (full)No price yet
Apple M4 Max (40-core GPU) 128GBMac128 GB (~96 usable)~25 tok/s256K (full)No price yet
Apple M5 Max (40-core GPU) 128GBMac128 GB (~96 usable)~28 tok/s256K (full)No price yet
NVIDIA RTX PRO 6000 Blackwell Max-QWorkstation GPU96 GB~68 tok/s256K (full)No price yet
NVIDIA RTX PRO 6000 Blackwell Server EditionData center GPU96 GB~63 tok/s256K (full)No price yet
NVIDIA RTX PRO 6000 Blackwell Workstation EditionWorkstation GPU96 GB~68 tok/s256K (full)No price yet
AMD Ryzen AI Max 385 (128GB)Mini PC128 GB~11 tok/s256K (full)No price yet
AMD Ryzen AI Max 390 (128GB)Mini PC128 GB~11 tok/s256K (full)No price yet
AMD Ryzen AI Max+ 395 (128GB)Mini PC128 GB~11 tok/s256K (full)low $3,499.99new · source, 2026-11-13
NVIDIA DGX Spark (128GB)Mini PC128 GB~13 tok/s256K (full)low $6,949.99new · source, 2026-10-23
NVIDIA RTX Spark N1X laptop (6144-core GPU, 128GB)Laptop128 GB~14 tok/s256K (full)No price yet
Apple M2 Ultra 192GBMac192 GB (~144 usable)~35 tok/sllama.cpp-style estimateMLX (measured, community): 34.6 tok/s at 4K (M2 Ultra 76c)256K (full)No price yet
Apple M3 Ultra 256GBMac256 GB (~192 usable)~35 tok/s256K (full)No price yet
Apple M5 Ultra 256GBMac256 GB (~192 usable)~48 tok/s256K (full)No price yet
Framework Desktop, AMD Ryzen AI Max+ PRO 495 (Strix Halo), 192 GBMini PC192 GB~12 tok/s256K (full)low $6,799new · source, 2026-10-16
Apple M5 Ultra 512GBMac512 GB (~384 usable)~48 tok/s256K (full)No price yet

Multi-GPU: cards that need two or more

CardsTotal VRAMSpeed estimateMax context estimate
2 × Intel Arc B58024 GB~15 tok/s82K · 155K (8-bit)
2 × NVIDIA GeForce RTX 3060 12 GB24 GB~17 tok/s82K · 155K (8-bit)
2 × NVIDIA GeForce RTX 4070 SUPER24 GB~23 tok/s82K · 155K (8-bit)
3 × NVIDIA GeForce RTX 506024 GB~21 tok/s82K · 155K (8-bit)
3 × NVIDIA GeForce RTX 5060 Ti 8 GB24 GB~21 tok/s82K · 155K (8-bit)
2 × NVIDIA GeForce RTX 507024 GB~30 tok/s82K · 155K (8-bit)
2 × AMD Radeon RX 7800 XT32 GB~26 tok/s202K · 256K (8-bit)
2 × AMD Radeon RX 9060 XT 16 GB32 GB~14 tok/s202K · 256K (8-bit)
2 × AMD Radeon RX 907032 GB~26 tok/s202K · 256K (8-bit)
2 × AMD Radeon RX 9070 XT32 GB~26 tok/s202K · 256K (8-bit)
2 × NVIDIA GeForce RTX 5060 Ti 16 GB32 GB~21 tok/s202K · 256K (8-bit)
2 × NVIDIA GeForce RTX 5070 Ti32 GB~39 tok/s202K · 256K (8-bit)
2 × NVIDIA GeForce RTX 508032 GB~41 tok/s202K · 256K (8-bit)

Measured on Macs with MLX (community)

MLX (measured, community): decode speeds people ran with MLX through oMLX and submitted to its public leaderboard. They are kept apart from this site's llama.cpp-style estimate, which is for a GGUF file in llama.cpp and is unchanged. The plain figure has no speculative decoding, at the shortest context the run used; runs with speculative decoding (MTP/DFlash) are shown separately, as a range, because their speed depends on how many drafted tokens are accepted, which varies with the prompt.

Macllama.cpp-style estimateMLX plain decodeWith speculative decoding (MTP/DFlash)Long context, one run
Apple M3 Max (30-core GPU) 36GB~14 tok/s16.0 tok/s at 4K (M3 Max 30c)17.8–21.3 tok/s at 1K–4K (M3 Max 30c)4K 16.0 → 16K 14.7 tok/s; prefill 142–164 tok/s (M3 Max 30c)
Apple M4 Max (32-core GPU) 36GB~19 tok/s20.1 tok/s at 4K (M4 Max 32c)——
Apple M4 Pro 48GB~13 tok/s15.6 tok/s at 1K (M4 Pro 16c)22.9–32.2 tok/s at 4K–16K (M4 Pro 16c)1K 15.6 → 4K 15.2 tok/s; prefill 106–106 tok/s (M4 Pro 16c)
Apple M5 Pro 48GBGPU cores differ between runs (M5 Pro 16c / M5 Pro 20c); listed here by chip and memory.~15 tok/s19.0 tok/s at 1K (M5 Pro 20c)18.8–18.9 tok/s at 1K–4K (M5 Pro 20c)4K 14.3 → 64K 13.1 tok/s; prefill 396–591 tok/s (M5 Pro 16c)
Apple M3 Max (40-core GPU) 64GB~19 tok/s—9.9–13.9 tok/s at 1K–16K (M3 Max 40c)1K 13.9 → 16K 9.9 tok/s; prefill 104–129 tok/s (M3 Max 40c, speculative run)
Apple M4 Max (40-core GPU) 64GB~25 tok/s29.3 tok/s at 1K (M4 Max 40c)63.7–68.7 tok/s (runs 1, 2) at 8K–16K (M4 Max 40c)1K 29.3 → 4K 28.8 tok/s; prefill 261–273 tok/s (M4 Max 40c)
Apple M5 Max (40-core GPU) 64GB~28 tok/s—27.2–119.8 tok/s (runs 1, 2) at 1K–195K (M5 Max 40c)1K 50.8 → 195K 27.2 tok/s; prefill 337–840 tok/s (M5 Max 40c, speculative run)
Apple M5 Pro 64GBGPU cores differ between runs (M5 Pro 16c / M5 Pro 20c); listed here by chip and memory.~15 tok/s17.8 tok/s at 1K (M5 Pro 20c)26.8–28.2 tok/s at 64K–128K (M5 Pro 16c)1K 17.8 → 4K 17.6 tok/s; prefill 474–509 tok/s (M5 Pro 20c)
Apple M5 Ultra 96GB~48 tok/s55.1 tok/s at 1K (M5 Ultra 80c)——
Apple M3 Max (40-core GPU) 128GB~19 tok/s—21.4–34.6 tok/s at 1K–4K (M3 Max 40c)1K 21.4 → 4K 34.6 tok/s; prefill 218–281 tok/s (M3 Max 40c, speculative run)
Apple M2 Ultra 192GB~35 tok/s34.6 tok/s at 4K (M2 Ultra 76c)—4K 34.6 → 16K 32.5 tok/s; prefill 236–252 tok/s (M2 Ultra 76c)

Long context: decode at the shortest and longest context of one run, and that run's prompt-processing (prefill) range. Agent jobs re-read long prompts, so prefill (prompt processing) speed matters as much as decode there. Community-submitted to oMLX, accessed 2026-10-26; each figure links its run(s). MLX 4-bit files, not the GGUF the estimate uses. About the MLX data

Speed calculator

estimate

Estimate = bandwidth ÷ GB read per token (the model's listed GGUF file: Q4_K_M for most models, otherwise the quant named in the list; scaled by active/total parameters for MoE), scaled by a per-platform efficiency and fixed per-token costs calibrated to 45 measured llama.cpp benchmarks (2025–26). Fit = file size + 1.5 GB headroom ≤ total VRAM (on Macs, the part of unified memory macOS lets the GPU use by default), and room for at least 4K tokens of context with an 8-bit KV cache. For a model with no KV cache data, only the file and headroom are checked, so a fit says nothing about context. Multi-GPU: VRAM adds, speed stays one card's (a layer split runs the cards in turn); max context uses the total. Speed is for short context. Max context = what is left after the file and a 1 GB runtime reserve ÷ KV cache bytes per token, default FP16 and 8-bit (q8_0) cache. How speed works · how context works

Reported by others

These are speed reports posted by individual users or reviewers. We haven't checked or reproduced them. They are not our estimates, they don't feed the estimator or My Rig, and they can't be compared with the tok/s figures elsewhere on this site or with each other, because each used a different model, quant, engine and test.

What this does and doesn't show

For coding agents, how much total output a machine keeps up with several requests at once, and how long each waits for its first token, matter as much as single-request tok/s. These reports show that for specific setups only; they don't show what you'd get with a different engine, quant or workload.

Qwen3.8-27B Q6 on one Arc Pro B70: what each software change did

Reported by Reddit user on r/LocalLLaMA (summarized by @wei_wang on X)
How this run was set up
Model & revisionQwen3.8-27B; exact GGUF file and revision Unknown
QuantQ6 (exact file Unknown)
Runtime & versionSteps 1–3: Ollama on Vulkan (Mesa 25.2, then Mesa 26.2). Steps 4–5: llama.cpp llama-server built with SYCL (Intel oneAPI); llama.cpp version Unknown
Topology1x Intel Arc Pro B70 (32 GB); kernel 6.17 or newer (Ubuntu 7.0 HWE kernel), Resizable BAR on, Intel compute runtime from a PPA
Input / output / contextFigures from the author's 16k-context column; output length Unknown
ConcurrencyUnknown
Speculative decodingOff in steps 1–4. On only in step 5: MTP with up to 4 draft tokens (--spec-type draft-mtp --spec-draft-n-max 4)
How speed was timedUnknown
Date measuredUnknown (summary posted 2026-11-10)
SourceOriginal Reddit post (r/LocalLLaMA) · @wei_wang on X, 2026-11-10
Evidence statusSelf-reported, not independently reproduced

Plain decoding (no drafting), generation at 16k context

WhatReportedNote
1. Ollama + Mesa 25.26.8 tok/s—
2. Mesa 26.219.4 tok/s—
3. Mesa 26.2, f16 KV cache33 tok/s—
4. llama.cpp SYCL, f16 KV cache49 tok/s—

With speculative decoding (MTP, up to 4 drafts)

WhatReportedNote
5. llama.cpp SYCL, q8_0 KV cache, MTP56.5 tok/s—

This page's own figure for Qwen3.8-27B is an estimate for Q4_K_M and is unchanged. This ladder used Q6 and different engines, so it isn't a check of that estimate.

The author found that on Vulkan any quantized KV cache slowed generation a lot, so steps 2–3 keep f16 there. Ollama has no SYCL backend.

Caveats

  • One person's setup on one card.
  • Self-reported, not independently reproduced.
  • Step 5 uses drafting; compare steps 1–4 with each other, not with step 5.

Qwen3.8-27B on one RTX 3090 with HyperQwen (patched vLLM)

Reported by syv-ai, HyperQwen repo
How this run was set up
Model & revisionQwen/Qwen3.8-27B, requantized by the repo to int4 weights (W4A16, AutoRound, the repo's 'fast' variant); revision Unknown
QuantW4A16 AutoRound (int4 weights, 16-bit activations), made by the repo's prepare step
Runtime & versionHyperQwen: a patch series on vLLM. The repo pins vLLM 0.33.0 (commit 3d6c954), but its mode pages say their tables were measured on earlier pins (baseline vLLM 0.27.1); version behind the headline figures: Unknown
Topology1x RTX 3090 (24 GB) at a 250 W power limit, shared with training jobs (README)
Input / output / contextShort real prompts with thinking on (per the repo's single-user page); 64k context in the default single-user mode; output length Unknown
Concurrency1 request (C1)
Speculative decodingBatch mode: none (batch mode ignores speculation). Single-user mode: on, DFlash2 drafter, 7 drafts proposed per pass
How speed was timedSingle-stream decode tok/s from the README summary table; method in the repo's bench/ folder
Date measuredUnknown (README as of 2026-11-10)
SourceHyperQwen README (GitHub)
Evidence statusSelf-reported, not independently reproduced

Plain decoding (no drafting), batch mode

WhatReportedNote
Single stream46 tok/s—

With speculative decoding (DFlash2, 7 drafts), single-user mode

WhatReportedNote
Single stream127 tok/s—

The README's 381 tok/s figure applies only when the answer quotes its own prompt (at 25k context), so it isn't listed as a generation speed here.

Speculation helps most with one or a few users; the README says plain batching takes over from about 8 concurrent users.

Caveats

  • One card, shared with training jobs.
  • The README says a first benchmark run after a start reads 30–50% low.
  • Self-reported in the project README, not independently reproduced.

Related