Can the GeForce RTX 4090 run Qwen2.5-Coder-32B-Instruct?
Yes. The Q4_K_M GGUF of Qwen2.5-Coder-32B-Instruct is 19.85 GB (source). With our 1.5 GB allowance for runtime and short-context cache it needs about 21.4 GB, leaving 2.6 GB of the card's 24 GB free. Expected speed: ~39 tok/s estimate. Lowest tracked price: low $3,200, avg $4,663 (used) source, 2026-10-16, which works out to $82 per tok/s (#3 of 5 priced GPUs that fit this model).
Last updated Oct 16, 2026. Speed is a bandwidth-based estimate (1010 GB/s, TechPowerUp), not a benchmark. Long contexts need more memory than our allowance.
Tight fit: under 3 GB spare means long prompts or a bigger context window may not fit. Consider a smaller quant or a card with more VRAM.
Next size up: Llama-3.3-70B-Instruct (42.52 GB at Q4_K_M) needs ~44 GB and does not fit on one NVIDIA GeForce RTX 4090. See what runs Llama-3.3-70B-Instruct.
Every priced GPU that runs Qwen2.5-Coder-32B-Instruct, by cost per token/sec
| GPU | VRAM | Speed estimate | Low price | $ per tok/s |
|---|---|---|---|---|
| AMD Radeon RX 7900 XTX | 24 GB | ~37 tok/s | $1,350 (new) | $36 |
| NVIDIA GeForce RTX 3090 | 24 GB | ~36 tok/s | $1,889 (used) | $52 |
| NVIDIA GeForce RTX 4090 | 24 GB | ~39 tok/s | $3,200 (used) | $82 |
| NVIDIA GeForce RTX 5090 | 32 GB | ~69 tok/s | $6,500 (new) | $94 |
| NVIDIA RTX A6000 | 48 GB | ~29 tok/s | $6,170 (new) | $213 |
Prices: picksbycard read of Newegg listings, 2026-10-16. Prices move daily.
Buy on Amazon As an Amazon Associate I earn from qualifying purchases.