Best GPU for local LLMs (2026)
Ranked by memory (what fits) and bandwidth (how fast it generates). Prices only where we have a dated source.
Cheapest GPU that runs Qwen3-32B: the AMD Radeon RX 7900 XTX at $1,350 (new). Qwen3-32B is 19.76 GB at Q4_K_M (source) and needs about 21.3 GB of VRAM with our 1.5 GB headroom allowance, so nothing cheaper in our price data has the memory. Estimated ~37 tok/s estimate.
Best cost per token/sec: the AMD Radeon RX 7900 XTX at about $36 per tok/s ($1,350 ÷ ~37 tok/s), against $94 for the NVIDIA GeForce RTX 5090. Fastest: the NVIDIA GeForce RTX 5090, ~69 tok/s from 1790 GB/s.
If 16 GB is enough: 17 of our 27 sized models fit, up to Mistral-Small-3.2-24B-Instruct-2506 (14.33 GB) — cheapest priced 16 GB card is the AMD Radeon RX 9070 at $570, ~34 tok/s on it.
Last updated Oct 16, 2026. Speeds are bandwidth-based estimates, not benchmarks. Prices are dated snapshots and move daily; "check price" means we have no verified price.
Every GPU and system we track, by memory then bandwidth
| GPU | VRAM | Bandwidth | Qwen3-32B estimate | Low price | $ per tok/s |
|---|---|---|---|---|---|
| Framework Desktop, AMD Ryzen AI Max+ PRO 495 (Strix Halo), 192 GB | 192 GB | 273 GB/s | ~10 tok/s | check price | — |
| Apple Mac Studio M5 Ultra (64-core GPU, 96 GB) | 96 GB | 1200 GB/s | ~46 tok/s | check price | — |
| Apple Mac Studio M5 Max (40-core GPU, 64 GB) | 64 GB | 614 GB/s | ~24 tok/s | check price | — |
| NVIDIA RTX 6000 Ada Generation | 48 GB | 960 GB/s | ~37 tok/s | check price | — |
| NVIDIA RTX A6000 | 48 GB | 768 GB/s | ~30 tok/s | check price | — |
| Apple Mac Studio M5 Max (32-core GPU, 36 GB) | 36 GB | 460 GB/s | ~18 tok/s | check price | — |
| NVIDIA GeForce RTX 5090 | 32 GB | 1790 GB/s | ~69 tok/s | low $6,500, avg $8,757 (new) source, 2026-10-15 | $94 |
| AMD Radeon AI PRO R9700 | 32 GB | 644.6 GB/s | ~25 tok/s | check price | — |
| NVIDIA GeForce RTX 4090 | 24 GB | 1010 GB/s | ~39 tok/s | low $3,200, avg $4,663 (used) source, 2026-10-15 | $82 |
| NVIDIA GeForce RTX 3090 Ti | 24 GB | 1010 GB/s | ~39 tok/s | check price | — |
| AMD Radeon RX 7900 XTX | 24 GB | 960 GB/s | ~37 tok/s | low $1,350, avg $1,552 (new) source, 2026-10-15 | $36 |
| NVIDIA GeForce RTX 3090 | 24 GB | 936.2 GB/s | ~36 tok/s | low $1,889, avg $2,404 (used) source, 2026-10-15 | $52 |
| NVIDIA Tesla P40 | 24 GB | 347.1 GB/s | ~13 tok/s | check price | — |
| AMD Radeon RX 7900 XT | 20 GB | 800 GB/s | doesn't fit | check price | — |
| NVIDIA GeForce RTX 5080 | 16 GB | 960 GB/s | doesn't fit | low $1,600, avg $2,399 (new) source, 2026-10-15 | $43 |
| NVIDIA GeForce RTX 5070 Ti | 16 GB | 896 GB/s | doesn't fit | low $1,120, avg $1,792 (new) source, 2026-10-15 | $33 |
| NVIDIA GeForce RTX 4080 SUPER | 16 GB | 736.3 GB/s | doesn't fit | check price | — |
| NVIDIA GeForce RTX 4080 | 16 GB | 716.8 GB/s | doesn't fit | check price | — |
| NVIDIA GeForce RTX 4070 Ti SUPER | 16 GB | 672.3 GB/s | doesn't fit | check price | — |
| AMD Radeon RX 9070 XT | 16 GB | 644.6 GB/s | doesn't fit | low $810, avg $1,194 (new) source, 2026-10-15 | $32 |
| AMD Radeon RX 9070 | 16 GB | 644.6 GB/s | doesn't fit | low $570, avg $1,052 (new) source, 2026-10-15 | $23 |
| AMD Radeon RX 7800 XT | 16 GB | 624.1 GB/s | doesn't fit | low $599, avg $834 (new) source, 2026-10-15 | $25 |
| NVIDIA GeForce RTX 5060 Ti 16 GB | 16 GB | 448 GB/s | doesn't fit | low $780, avg $1,146 (new) source, 2026-10-15 | $46 |
| AMD Radeon RX 9060 XT 16 GB | 16 GB | 322.3 GB/s | doesn't fit | low $620, avg $812 (new) source, 2026-10-15 | $52 |
| NVIDIA GeForce RTX 4060 Ti 16 GB | 16 GB | 288 GB/s | doesn't fit | check price | — |
| AMD Radeon RX 7600 XT | 16 GB | 288 GB/s | doesn't fit | check price | — |
| NVIDIA GeForce RTX 3080 Ti | 12 GB | 912.4 GB/s | doesn't fit | check price | — |
| NVIDIA GeForce RTX 5070 | 12 GB | 672 GB/s | doesn't fit | low $800, avg $1,086 (new) source, 2026-10-15 | $31 |
| NVIDIA GeForce RTX 4070 SUPER | 12 GB | 504.2 GB/s | doesn't fit | low $966, avg $1,156 (used) source, 2026-10-15 | $51 |
| NVIDIA GeForce RTX 4070 | 12 GB | 504.2 GB/s | doesn't fit | check price | — |
| Intel Arc B580 | 12 GB | 456 GB/s | doesn't fit | low $330, avg $446 (new) source, 2026-10-15 | $18 |
| NVIDIA GeForce RTX 3060 12 GB | 12 GB | 360 GB/s | doesn't fit | low $455, avg $588 (used) source, 2026-10-15 | $33 |
| NVIDIA GeForce RTX 3080 | 10 GB | 760.3 GB/s | doesn't fit | check price | — |
| NVIDIA GeForce RTX 5060 Ti 8 GB | 8 GB | 448 GB/s | doesn't fit | low $450, avg $809 (new) source, 2026-10-15 | $26 |
| NVIDIA GeForce RTX 5060 | 8 GB | 448 GB/s | doesn't fit | low $410, avg $550 (new) source, 2026-10-15 | $24 |
| NVIDIA GeForce RTX 4060 | 8 GB | 272 GB/s | doesn't fit | check price | — |
Specs from each GPU's source page (TechPowerUp for the add-in cards, vendor spec pages for the Apple and Framework systems), accessed 2026-10-15. Unified-memory systems share that memory with the OS, so their fit results are optimistic.
What fits at each VRAM size
| VRAM | Models that fit | Largest that fits | Cheapest priced card at this size |
|---|---|---|---|
| 8 GB | 8 of 27 | Qwen3-8B (5.03 GB) | GeForce RTX 5060 $410 (new) |
| 10 GB | 11 of 27 | Mistral-Nemo-Instruct-2407 (7.48 GB) | no dated price |
| 12 GB | 14 of 27 | Qwen3-14B (9 GB) | Arc B580 $330 (new) |
| 16 GB | 17 of 27 | Mistral-Small-3.2-24B-Instruct-2506 (14.33 GB) | Radeon RX 9070 $570 (new) |
| 20 GB | 18 of 27 | gemma-3-27b-it (16.55 GB) | no dated price |
| 24 GB | 22 of 27 | Qwen2.5-Coder-32B-Instruct (19.85 GB) | Radeon RX 7900 XTX $1,350 (new) |
| 32 GB | 22 of 27 | Qwen2.5-Coder-32B-Instruct (19.85 GB) | GeForce RTX 5090 $6,500 (new) |
| 36 GB | 22 of 27 | Qwen2.5-Coder-32B-Instruct (19.85 GB) | no dated price |
| 48 GB | 24 of 27 | Llama-3.3-70B-Instruct (42.52 GB) | no dated price |
| 64 GB | 25 of 27 | Qwen2.5-72B-Instruct (47.42 GB) | no dated price |
| 96 GB | 27 of 27 | GLM-4.5-Air (73.5 GB) | no dated price |
| 192 GB | 27 of 27 | GLM-4.5-Air (73.5 GB) | no dated price |
Questions people ask
What is the cheapest GPU that can run Qwen3-32B?
The AMD Radeon RX 7900 XTX at $1,350 (new, priced 2026-10-15). Qwen3-32B is 19.76 GB at Q4_K_M and needs about 21.3 GB of VRAM with our 1.5 GB headroom allowance, so no cheaper card in our price data has the memory. Estimated speed: about 37 tokens per second.
Which GPU gives the most tokens per second per dollar?
The AMD Radeon RX 7900 XTX, at roughly $36 per token per second on Qwen3-32B ($1,350 divided by an estimated 37 tok/s). Of the 4 priced GPUs that fit the model, the most expensive per token per second is the NVIDIA GeForce RTX 5090 at about $94.
Do you need 24 GB of VRAM for local AI?
Not for most models. 17 of the 27 models we have Q4_K_M file sizes for fit in 16 GB, up to Mistral-Small-3.2-24B-Instruct-2506 (14.33 GB), and the cheapest priced card with that much memory is the AMD Radeon RX 9070 at $570. 24 GB raises that to 22 models, up to Qwen2.5-Coder-32B-Instruct. At 48 GB the largest that fits is Llama-3.3-70B-Instruct, a 42.52 GB file that needs about 44 GB.
What is the fastest GPU for local LLMs in this data?
The NVIDIA GeForce RTX 5090: 1790 GB/s of memory bandwidth works out to an estimated 69 tok/s on Qwen3-32B. Every speed on this page is calculated from bandwidth (0.76 x bandwidth / GB read per token), not measured in a benchmark.
Related
- What the AMD Radeon RX 7900 XTX can run
- What the NVIDIA GeForce RTX 5090 can run
- What the AMD Radeon RX 9070 can run
- What GPU you need for Qwen3-32B
- Can the Radeon RX 7900 XTX run Qwen2.5-Coder-32B-Instruct?
- Can the Radeon RX 7900 XTX run Qwen3-32B?
- Can the GeForce RTX 5090 run Qwen2.5-Coder-32B-Instruct?
- Can the GeForce RTX 5090 run Qwen3-32B?
- Can the Radeon RX 9070 run Mistral-Small-3.2-24B-Instruct-2506?
- Can the Radeon RX 9070 run gpt-oss-20b?
- Build: 16 GB starter: Radeon RX 9070 — $1,744.44, 16 GB
- Build: 16 GB NVIDIA: RTX 5060 Ti 16 GB — $1,834.44, 16 GB
- Build: 24 GB: one used RTX 3090 — $2,933.45, 24 GB
- Build: 32 GB: two Radeon RX 9070 — $3,065.92, 32 GB
- Guide: RTX 3090 vs RTX 5090 for local LLMs
- Guide: Mac vs PC for local LLMs
- Guide: How much VRAM do you need for a 70B model?