Can the RTX A6000 run Qwen2.5-Coder-32B-Instruct?
Yes. The Q4_K_M GGUF of Qwen2.5-Coder-32B-Instruct is 19.85 GB (source). With our 1.5 GB allowance for runtime and short-context cache it needs about 21.4 GB, leaving 26.6 GB of the card's 48 GB free. Expected speed: ~29 tok/s estimate. Lowest tracked price: low $6,170 (new) source, 2026-10-16, which works out to $213 per tok/s (#5 of 5 priced GPUs that fit this model).
Last updated Oct 16, 2026. Speed is a bandwidth-based estimate (768 GB/s, TechPowerUp), not a benchmark. Long contexts need more memory than our allowance.
Next size up: Qwen2.5-72B-Instruct (47.42 GB at Q4_K_M) needs ~48.9 GB and does not fit on one NVIDIA RTX A6000. See what runs Qwen2.5-72B-Instruct.
Every priced GPU that runs Qwen2.5-Coder-32B-Instruct, by cost per token/sec
| GPU | VRAM | Speed estimate | Low price | $ per tok/s |
|---|---|---|---|---|
| AMD Radeon RX 7900 XTX | 24 GB | ~37 tok/s | $1,350 (new) | $36 |
| NVIDIA GeForce RTX 3090 | 24 GB | ~36 tok/s | $1,889 (used) | $52 |
| NVIDIA GeForce RTX 4090 | 24 GB | ~39 tok/s | $3,200 (used) | $82 |
| NVIDIA GeForce RTX 5090 | 32 GB | ~69 tok/s | $6,500 (new) | $94 |
| NVIDIA RTX A6000 | 48 GB | ~29 tok/s | $6,170 (new) | $213 |
Prices: picksbycard read of Newegg listings, 2026-10-16. Prices move daily.
Buy on Amazon As an Amazon Associate I earn from qualifying purchases.