Best GPU for MoE (mixture-of-experts) LLMs
Memory first, bandwidth second. K2-Horizon-MoVA-36B-A4B needs about 23.9 GB for its whole file, yet runs at about 217 tok/s estimate on the GeForce RTX 3090, against 37 for the dense K2-Horizon-32B. The GeForce RTX 5090 has 1.9× the GeForce RTX 3090's bandwidth but is only 1.1× to 1.4× faster on MoE models.
A mixture-of-experts model has to fit whole, but each token only reads its active experts. That splits the GPU question in two: memory decides what fits, and bandwidth matters less than it does for dense models.
Memory decides what fits; active parameters decide the speed. K2-Horizon-MoVA-36B-A4B is a 22.37 GB file but reads only about 2.39 GB per token (4B of 37.44B parameters active). On the NVIDIA GeForce RTX 3090 that is ~217 tok/s against ~37 for the dense K2-Horizon-32B (21.08 GB) estimate.
So buy memory first. A faster card helps less: the GeForce RTX 5090 has 1.9× the bandwidth of the GeForce RTX 3090, which lifts the dense estimate 1.9× but the MoE estimates only 1.1× to 1.4×, because part of every token is fixed routing and kernel overhead that bandwidth doesn't shorten.
Cheapest way in: LFM2.5-8B-A1B and JetBrains Mellum2.1 12B-A2.5B Thinking: GeForce RTX 3060 12 GB ($305); gpt-oss-20b: Radeon RX 9070 ($570); LFM2.5-24B-A2B and 9 more: Radeon RX 7900 XTX ($1,350); NVIDIA Nemotron 3.5 Lightning 30B-A3B and 4 more: DGX Spark ($4,999); gpt-oss-120b and NVIDIA Nemotron 3.5 Super VL 120B-A12B: Mac Studio M5 Ultra ($5,499); Mistral Small 4 119B and 10 more: Framework Desktop, AMD Ryzen AI Max+ PRO 495 ($6,799). 8 MoE models need more memory than any device we have a price for.
Last updated Oct 16, 2026. Speeds are bandwidth-based estimates, not benchmarks. Prices are dated snapshots and move daily; "No price yet" means we have no verified price.
Every current MoE model: what it needs and the cheapest way to run it
| Model | Active / total | File | Needs | GB read per token | Cheapest priced device that fits | Best $ per tok/s estimate |
|---|---|---|---|---|---|---|
| LFM2.5-8B-A1B | 1.5B / 8.47B (18%) | 5.16 GB | 6.66 GB | 0.91 GB | GeForce RTX 3060 12 GB $305, ~143 tok/s | GeForce RTX 3060 12 GB $2.13 |
| JetBrains Mellum2.1 12B-A2.5B Thinking | 2.5B / 12.15B (21%) | 8.07 GB | 9.57 GB | 1.66 GB | GeForce RTX 3060 12 GB $305, ~144 tok/s | GeForce RTX 3060 12 GB $2.12 |
| gpt-oss-20b | 3.6B / 20.91B (17%) | 11.67 GB | 13.17 GB | 2.01 GB | Radeon RX 9070 $570, ~154 tok/s | Radeon RX 9070 $3.70 |
| LFM2.5-24B-A2B | 2B / 23.97B (8%) | 14.53 GB | 16.03 GB | 1.21 GB | Radeon RX 7900 XTX $1,350, ~155 tok/s | GeForce RTX 3090 $8.37 |
| Gemma 4 26B A4B | 3.8B / 25.81B (15%) | 17.04 GB | 18.54 GB | 2.51 GB | Radeon RX 7900 XTX $1,350, ~173 tok/s | GeForce RTX 3090 $7.10 |
| GLM-4.7-Flash | 3B / 31.22B (10%) | 18.31 GB | 19.81 GB | 1.76 GB | Radeon RX 7900 XTX $1,350, ~209 tok/s | GeForce RTX 3090 $5.74 |
| Raon-1 mini preview | 4.3B / 28.8B (15%) | 19.69 GB | 21.19 GB | 2.94 GB | Radeon RX 7900 XTX $1,350, ~114 tok/s | GeForce RTX 3090 $12 |
| Xing4.0-29B-A4B | 4B / 31.22B (13%) | 20.06 GB | 21.56 GB | 2.57 GB | Radeon RX 7900 XTX $1,350, ~170 tok/s | GeForce RTX 3090 $7.24 |
| MiniCPM-V 4.7 35B-A3B | 3B / 35.21B (9%) | 21.17 GB | 22.67 GB | 1.8 GB | Radeon RX 7900 XTX $1,350, ~138 tok/s | GeForce RTX 3090 $9.43 |
| NVIDIA Nemotron 3.5 Bolt 74B-A9B 24GB (2-bit) | 9B / 74.89B (12%) | 22.02 GB (native 2-bit) | 23.52 GB | 2.65 GB | Radeon RX 7900 XTX $1,350, ~119 tok/s | GeForce RTX 3090 $11 |
| Qwen3.5-35B-A3B | 3B / 35.95B (8%) | 22.29 GB | 23.79 GB | 1.86 GB | Radeon RX 7900 XTX $1,350, ~137 tok/s | GeForce RTX 3090 $9.55 |
| Qwen3.6-35B-A3B | 3B / 35.95B (8%) | 22.29 GB | 23.79 GB | 1.86 GB | Radeon RX 7900 XTX $1,350, ~137 tok/s | GeForce RTX 3090 $9.55 |
| K2-Horizon-MoVA-36B-A4B | 4B / 37.44B (11%) | 22.37 GB | 23.87 GB | 2.39 GB | Radeon RX 7900 XTX $1,350, ~178 tok/s | GeForce RTX 3090 $6.91 |
| NVIDIA Nemotron 3.5 Lightning 30B-A3B | 3B / 31.58B (9%) | 25.48 GB | 26.98 GB | 2.42 GB | DGX Spark $4,999, ~71 tok/s | GeForce RTX 5090 $37 |
| Recursive Flint 40B-A2.5B | 2.5B / 39.8B (6%) | 25.99 GB (Q4_1) | 27.49 GB | 1.63 GB | DGX Spark $4,999, ~118 tok/s | GeForce RTX 5090 $18 |
| Qwen3-Coder-Next | 3B / 79.67B (4%) | 48.53 GB | 50.03 GB | 1.83 GB | DGX Spark $4,999, ~85 tok/s | Mac Studio M5 Ultra $50 |
| Aleph Alpha Kolibri-1 78B-A3.5B | 3.46B / 78.1B (4%) | 49.76 GB | 51.26 GB | 2.2 GB | DGX Spark $4,999, ~92 tok/s | Mac Studio M5 Ultra $39 |
| NVIDIA Nemotron 3.5 Bolt 74B-A9B | 9B / 74.89B (12%) | 50.22 GB (UD-Q4_K_XL) | 51.72 GB | 6.04 GB | DGX Spark $4,999, ~34 tok/s | Mac Studio M5 Ultra $72 |
| gpt-oss-120b | 5.1B / 116.83B (4%) | 62.84 GB | 64.34 GB | 2.74 GB | Mac Studio M5 Ultra $5,499, ~130 tok/s | Mac Studio M5 Ultra $42 |
| NVIDIA Nemotron 3.5 Super VL 120B-A12B | 12B / 124.45B (10%) | 70.1 GB | 71.60 GB | 6.76 GB | Mac Studio M5 Ultra $5,499, ~87 tok/s | Mac Studio M5 Ultra $63 |
| Mistral Small 4 119B | 6.5B / 119.4B (5%) | 72.64 GB | 74.14 GB | 3.95 GB | Framework Desktop, AMD Ryzen AI Max+ PRO 495 $6,799, ~47 tok/s | DGX Spark $126 |
| Qwen3.5-122B-A10B | 10B / 125.09B (8%) | 77.62 GB | 79.12 GB | 6.21 GB | Framework Desktop, AMD Ryzen AI Max+ PRO 495 $6,799, ~29 tok/s | DGX Spark $211 |
| Ornith-1.5-122B | 10B / 125.09B (8%) | 78.92 GB | 80.42 GB | 6.31 GB | Framework Desktop, AMD Ryzen AI Max+ PRO 495 $6,799, ~29 tok/s | DGX Spark $211 |
| Qwen3.8-Flash-Next | 6B / 180B (3%) | 119.6 GB | 121.10 GB | 3.99 GB | Framework Desktop, AMD Ryzen AI Max+ PRO 495 $6,799, ~42 tok/s | DGX Spark $145 |
| Swift 1.5 Qwen3.8-Flash-Next | 6B / 180B (3%) | 119.6 GB | 121.10 GB | 3.99 GB | Framework Desktop, AMD Ryzen AI Max+ PRO 495 $6,799, ~42 tok/s | DGX Spark $145 |
| StepFun Step 3.7 Flash | 11B / 201.37B (5%) | 122.09 GB (UD-Q4_K_M) | 123.59 GB | 6.67 GB | Framework Desktop, AMD Ryzen AI Max+ PRO 495 $6,799, ~29 tok/s | DGX Spark $204 |
| Qwen3-235B-A22B | 22B / 235.09B (9%) | 142.15 GB | 143.65 GB | 13.3 GB | Framework Desktop, AMD Ryzen AI Max+ PRO 495 $6,799, ~15 tok/s | Framework Desktop, AMD Ryzen AI Max+ PRO 495 $453 |
| MiniMax-M3.1-Flash Unsloth 2-bit | 23B / 427.05B (5%) | 143.91 GB (UD-Q2_K_XL) | 145.41 GB | 7.75 GB | Framework Desktop, AMD Ryzen AI Max+ PRO 495 $6,799, ~25 tok/s | Framework Desktop, AMD Ryzen AI Max+ PRO 495 $272 |
| DeepSeek-V4-Flash | 13B / 290.94B (4%) | 155.1 GB (UD-Q4_K_XL) | 156.60 GB | 6.93 GB | Framework Desktop, AMD Ryzen AI Max+ PRO 495 $6,799, ~28 tok/s | Framework Desktop, AMD Ryzen AI Max+ PRO 495 $243 |
| MiniMax-M3.1-Flash Q2_K | 23B / 427.05B (5%) | 155.91 GB (Q2_K) | 157.41 GB | 8.4 GB | Framework Desktop, AMD Ryzen AI Max+ PRO 495 $6,799, ~24 tok/s | Framework Desktop, AMD Ryzen AI Max+ PRO 495 $283 |
| Xiaomi MiMo-V2.6-Flash 309B-A15B | 15B / 310.76B (5%) | 167.37 GB (MXFP4) | 168.87 GB | 8.08 GB | Framework Desktop, AMD Ryzen AI Max+ PRO 495 $6,799, ~24 tok/s | Framework Desktop, AMD Ryzen AI Max+ PRO 495 $283 |
| GLM-5.3-Flash | 18B / 321.32B (6%) | 199.71 GB (UD-Q4_K_XL) | 201.21 GB | 11.19 GB | none we price | — |
| MiniMax-M3.1-Flash | 23B / 427.05B (5%) | 258.01 GB | 259.51 GB | 13.9 GB | none we price | — |
| Reflection Beam 1.0 501B-A23B | 23B / 501.27B (5%) | 302.41 GB | 303.91 GB | 13.88 GB | none we price | — |
| Thinking Machines Inkling 2 Small | 17B / 593.72B (3%) | 354.15 GB (UD-Q4_K_XL) | 355.65 GB | 10.14 GB | none we price | — |
| inclusionAI Ling-3.1-flash 560B-A25B | 25B / 572.12B (4%) | 357.97 GB | 359.47 GB | 15.64 GB | none we price | — |
| DeepSeek-V4.1-Flash (1017) | 16B / 763.21B (2%) | 500.71 GB (UD-Q4_K_XL) | 502.21 GB | 10.5 GB | none we price | — |
| Meta Muse Spark Open 816B-A42B | 42B / 822.04B (5%) | 505.5 GB (UD-Q4_K_XL) | 507.00 GB | 25.83 GB | none we price | — |
| Mistral Large 4 | 52B / 1050B (5%) | 605 GB (UD-Q4_K_XL) | 606.50 GB | 29.96 GB | none we price | — |
Devices include whole systems (Macs, at the share of unified memory macOS gives the GPU by default, and the Framework Desktop). Speeds use bandwidth ÷ GB read per token, scaled by a per-platform efficiency and fixed per-token costs calibrated to 45 measured llama.cpp benchmarks (2025–26) (how).
MoE models on 6 common choices
| Model | GeForce RTX 5060 Ti 16 GB (16 GB) | Radeon RX 9070 (16 GB) | GeForce RTX 3090 (24 GB) | Radeon RX 7900 XTX (24 GB) | GeForce RTX 5090 (32 GB) | Framework Desktop, AMD Ryzen AI Max+ PRO 495 (192 GB) |
|---|---|---|---|---|---|---|
| LFM2.5-8B-A1B | ~156 tok/s | ~150 tok/s | ~192 tok/s | ~166 tok/s | ~214 tok/s | ~109 tok/s |
| JetBrains Mellum2.1 12B-A2.5B Thinking | ~169 tok/s | ~173 tok/s | ~269 tok/s | ~215 tok/s | ~363 tok/s | ~97 tok/s |
| gpt-oss-20b | ~147 tok/s | ~154 tok/s | ~242 tok/s | ~195 tok/s | ~336 tok/s | ~84 tok/s |
| LFM2.5-24B-A2B | doesn't fit | doesn't fit | ~179 tok/s | ~155 tok/s | ~205 tok/s | ~95 tok/s |
| Gemma 4 26B A4B | doesn't fit | doesn't fit | ~211 tok/s | ~173 tok/s | ~304 tok/s | ~70 tok/s |
| GLM-4.7-Flash | doesn't fit | doesn't fit | ~261 tok/s | ~209 tok/s | ~355 tok/s | ~93 tok/s |
| Raon-1 mini preview | doesn't fit | doesn't fit | ~130 tok/s | ~114 tok/s | ~168 tok/s | ~53 tok/s |
| Xing4.0-29B-A4B | doesn't fit | doesn't fit | ~207 tok/s | ~170 tok/s | ~300 tok/s | ~68 tok/s |
| MiniCPM-V 4.7 35B-A3B | doesn't fit | doesn't fit | ~159 tok/s | ~138 tok/s | ~191 tok/s | ~75 tok/s |
| NVIDIA Nemotron 3.5 Bolt 74B-A9B 24GB (2-bit) | doesn't fit | doesn't fit | ~137 tok/s | ~119 tok/s | ~173 tok/s | ~58 tok/s |
| Qwen3.5-35B-A3B | doesn't fit | doesn't fit | ~157 tok/s | ~137 tok/s | ~189 tok/s | ~73 tok/s |
| Qwen3.6-35B-A3B | doesn't fit | doesn't fit | ~157 tok/s | ~137 tok/s | ~189 tok/s | ~73 tok/s |
| K2-Horizon-MoVA-36B-A4B | doesn't fit | doesn't fit | ~217 tok/s | ~178 tok/s | ~311 tok/s | ~73 tok/s |
| NVIDIA Nemotron 3.5 Lightning 30B-A3B | doesn't fit | doesn't fit | doesn't fit | doesn't fit | ~177 tok/s | ~61 tok/s |
| Recursive Flint 40B-A2.5B | doesn't fit | doesn't fit | doesn't fit | doesn't fit | ~366 tok/s | ~98 tok/s |
| Qwen3-Coder-Next | doesn't fit | doesn't fit | doesn't fit | doesn't fit | doesn't fit | ~74 tok/s |
| Aleph Alpha Kolibri-1 78B-A3.5B | doesn't fit | doesn't fit | doesn't fit | doesn't fit | doesn't fit | ~78 tok/s |
| NVIDIA Nemotron 3.5 Bolt 74B-A9B | doesn't fit | doesn't fit | doesn't fit | doesn't fit | doesn't fit | ~30 tok/s |
| gpt-oss-120b | doesn't fit | doesn't fit | doesn't fit | doesn't fit | doesn't fit | ~65 tok/s |
| NVIDIA Nemotron 3.5 Super VL 120B-A12B | doesn't fit | doesn't fit | doesn't fit | doesn't fit | doesn't fit | ~29 tok/s |
| Mistral Small 4 119B | doesn't fit | doesn't fit | doesn't fit | doesn't fit | doesn't fit | ~47 tok/s |
| Qwen3.5-122B-A10B | doesn't fit | doesn't fit | doesn't fit | doesn't fit | doesn't fit | ~29 tok/s |
| Ornith-1.5-122B | doesn't fit | doesn't fit | doesn't fit | doesn't fit | doesn't fit | ~29 tok/s |
| Qwen3.8-Flash-Next | doesn't fit | doesn't fit | doesn't fit | doesn't fit | doesn't fit | ~42 tok/s |
| Swift 1.5 Qwen3.8-Flash-Next | doesn't fit | doesn't fit | doesn't fit | doesn't fit | doesn't fit | ~42 tok/s |
| StepFun Step 3.7 Flash | doesn't fit | doesn't fit | doesn't fit | doesn't fit | doesn't fit | ~29 tok/s |
| Qwen3-235B-A22B | doesn't fit | doesn't fit | doesn't fit | doesn't fit | doesn't fit | ~15 tok/s |
| MiniMax-M3.1-Flash Unsloth 2-bit | doesn't fit | doesn't fit | doesn't fit | doesn't fit | doesn't fit | ~25 tok/s |
| DeepSeek-V4-Flash | doesn't fit | doesn't fit | doesn't fit | doesn't fit | doesn't fit | ~28 tok/s |
| MiniMax-M3.1-Flash Q2_K | doesn't fit | doesn't fit | doesn't fit | doesn't fit | doesn't fit | ~24 tok/s |
| Xiaomi MiMo-V2.6-Flash 309B-A15B | doesn't fit | doesn't fit | doesn't fit | doesn't fit | doesn't fit | ~24 tok/s |
| GLM-5.3-Flash | doesn't fit | doesn't fit | doesn't fit | doesn't fit | doesn't fit | doesn't fit |
| MiniMax-M3.1-Flash | doesn't fit | doesn't fit | doesn't fit | doesn't fit | doesn't fit | doesn't fit |
| Reflection Beam 1.0 501B-A23B | doesn't fit | doesn't fit | doesn't fit | doesn't fit | doesn't fit | doesn't fit |
| Thinking Machines Inkling 2 Small | doesn't fit | doesn't fit | doesn't fit | doesn't fit | doesn't fit | doesn't fit |
| inclusionAI Ling-3.1-flash 560B-A25B | doesn't fit | doesn't fit | doesn't fit | doesn't fit | doesn't fit | doesn't fit |
| DeepSeek-V4.1-Flash (1017) | doesn't fit | doesn't fit | doesn't fit | doesn't fit | doesn't fit | doesn't fit |
| Meta Muse Spark Open 816B-A42B | doesn't fit | doesn't fit | doesn't fit | doesn't fit | doesn't fit | doesn't fit |
| Mistral Large 4 | doesn't fit | doesn't fit | doesn't fit | doesn't fit | doesn't fit | doesn't fit |
All speeds are estimates, short context, one device.
Why a faster card helps MoE less
| Model | GeForce RTX 3090 estimate | GeForce RTX 5090 estimate | Speed-up |
|---|---|---|---|
| K2-Horizon-32B (dense, for comparison) | ~37 tok/s | ~69 tok/s | 1.9× |
| LFM2.5-8B-A1B | ~192 tok/s | ~214 tok/s | 1.1× |
| JetBrains Mellum2.1 12B-A2.5B Thinking | ~269 tok/s | ~363 tok/s | 1.3× |
| gpt-oss-20b | ~242 tok/s | ~336 tok/s | 1.4× |
| LFM2.5-24B-A2B | ~179 tok/s | ~205 tok/s | 1.1× |
| Gemma 4 26B A4B | ~211 tok/s | ~304 tok/s | 1.4× |
| GLM-4.7-Flash | ~261 tok/s | ~355 tok/s | 1.4× |
| Raon-1 mini preview | ~130 tok/s | ~168 tok/s | 1.3× |
| Xing4.0-29B-A4B | ~207 tok/s | ~300 tok/s | 1.4× |
| MiniCPM-V 4.7 35B-A3B | ~159 tok/s | ~191 tok/s | 1.2× |
| NVIDIA Nemotron 3.5 Bolt 74B-A9B 24GB (2-bit) | ~137 tok/s | ~173 tok/s | 1.3× |
| Qwen3.5-35B-A3B | ~157 tok/s | ~189 tok/s | 1.2× |
| Qwen3.6-35B-A3B | ~157 tok/s | ~189 tok/s | 1.2× |
| K2-Horizon-MoVA-36B-A4B | ~217 tok/s | ~311 tok/s | 1.4× |
Bandwidth: 936 GB/s vs 1792 GB/s (1.9×). Models that fit both cards.
Who this is for
You run, or want to run, mixture-of-experts models and are deciding between more memory and more bandwidth. For these models memory is the constraint and bandwidth is a smaller lever than usual. For the biggest MoE models, see the gpt-oss-120b guide and Strix Halo vs a GPU; the formula is on our methodology page.
Questions people ask
What is the best GPU for MoE models?
Memory first, bandwidth second. K2-Horizon-MoVA-36B-A4B needs about 23.9 GB for its whole file, yet runs at about 217 tok/s estimate on the GeForce RTX 3090, against 37 for the dense K2-Horizon-32B. The GeForce RTX 5090 has 1.9× the GeForce RTX 3090's bandwidth but is only 1.1× to 1.4× faster on MoE models.
Why do MoE models run faster than dense models of the same size?
Each token only reads the active experts. K2-Horizon-MoVA-36B-A4B is a 22.37 GB file, but only 4B of its 37.44B parameters are active, so it reads about 2.39 GB per token. On the NVIDIA GeForce RTX 3090 that works out to an estimated 217 tok/s, against 37 tok/s for the dense K2-Horizon-32B (21.08 GB). estimate
Do MoE models need less VRAM?
No. The whole file has to be in memory, because any token can be routed to any expert. K2-Horizon-MoVA-36B-A4B needs about 23.87 GB at Q4_K_M, the same as a dense model with a file that size. Offloading experts to system RAM is possible but much slower, and we don't estimate it.
Is a faster GPU worth it for MoE models?
Less than for dense models. The NVIDIA GeForce RTX 5090 has 1.9x the bandwidth of the NVIDIA GeForce RTX 3090. Our estimate for the dense K2-Horizon-32B scales 1.9x, but for the MoE models that fit both it scales only 1.1x to 1.4x, because part of each token is fixed routing and kernel overhead that bandwidth doesn't shorten.
Related
- GPUWhat the NVIDIA GeForce RTX 5060 Ti 16 GB can run
- GPUWhat the AMD Radeon RX 9070 can run
- GPUWhat the NVIDIA GeForce RTX 3090 can run
- GPUWhat the AMD Radeon RX 7900 XTX can run
- GPUWhat the NVIDIA GeForce RTX 5090 can run
- ModelWhat GPU you need for K2-Horizon-MoVA-36B-A4B
- ModelWhat GPU you need for Xiaomi MiMo-V2.6-Flash 309B-A15B
- Can it runCan the GeForce RTX 5060 Ti 16 GB run Mistral-Small-3.2-24B-Instruct-2506?
- Can it runCan the GeForce RTX 5060 Ti 16 GB run gpt-oss-20b?
- Can it runCan the Radeon RX 9070 run Mistral-Small-3.2-24B-Instruct-2506?
- Can it runCan the Radeon RX 9070 run gpt-oss-20b?
- Can it runCan the GeForce RTX 3090 run K2-Horizon-MoVA-36B-A4B?
- Can it runCan the GeForce RTX 3090 run Qwen3.6-35B-A3B?
- Build16 GB starter: Radeon RX 9070$2,101.57, 16 GB
- Build16 GB NVIDIA: RTX 5060 Ti 16 GB$1,951.57, 16 GB
- Build24 GB: one used RTX 3090$2,650.58, 24 GB
- Build32 GB: two Radeon RX 9070$3,589.93, 32 GB
- GuideRTX 3090 vs RTX 5090 for local LLMs
- GuideCan an RTX 5060 Ti 16 GB run Qwen3 Coder 30B?
- GuideStrix Halo vs a GPU for local AI