Can the RTX A6000 run NVIDIA Nemotron 3.5 Lightning 30B-A3B?
Yes. A new RTX A6000 ($8,363) runs NVIDIA Nemotron 3.5 Lightning 30B-A3B at Q4 with room for its full 256K context, at about 130 tok/s estimate. That is $64 per tok/s, #2 of 2 priced GPUs that fit it; the GeForce RTX 5090 ranks first at $37.
Yes, with room for long context. On the NVIDIA RTX A6000 (48 GB) there is room for about its full 256K tokens of context with the default FP16 KV cacheestimate: what is left after the weights and a 1 GB runtime reserve. The Q4_K_M GGUF of NVIDIA Nemotron 3.5 Lightning 30B-A3B is 25.48 GB (source); with our 1.5 GB allowance for the runtime and a short context it needs about 27 GB.
NVIDIA Nemotron 3.5 Lightning 30B-A3B fits 2 of the 18 GPUs we have a dated price for; the smallest of them has 32 GB. The RTX A6000's 768 GB/s gives ~130 tok/s estimate, 73% of the fastest card, the GeForce RTX 5090 at ~177 tok/s. Only 3B of its 31.58B parameters are active per token, so the estimate reads 2.42 GB per token rather than the whole 25.48 GB file, plus a per-token routing and expert-gather cost calibrated to 45 measured llama.cpp benchmarks (2025–26). On the same card, the dense Ripple-65B Preview (38.8 GB) estimates ~17 tok/s.
At $8,363 (new) it works out to $64 per tok/s, #2 of 2, the most expensive of them. The GeForce RTX 5090 is the cheapest per tok/s at $37 ($6,500, new). The cheapest complete build we priced that runs NVIDIA Nemotron 3.5 Lightning 30B-A3B is our 32 GB: two Radeon RX 9070 build at $3,589.93.
Last updated Oct 16, 2026. Speed is a bandwidth-based estimate (768 GB/s, NVIDIA), scaled by per-platform constants calibrated to 45 measured llama.cpp benchmarks (2025–26), not a benchmark of this pair. Max context is an estimate from the model's architecture for llama.cpp with flash attention.
Next size up: Qwen3-Coder-Next (48.53 GB at Q4_K_M) needs ~50 GB and does not fit on one NVIDIA RTX A6000. See what runs Qwen3-Coder-Next.
Every priced GPU that runs NVIDIA Nemotron 3.5 Lightning 30B-A3B, by cost per token/sec
| GPU | VRAM | Speed estimate | Max context estimate | Low price | $ per tok/s |
|---|---|---|---|---|---|
| NVIDIA GeForce RTX 5090 | 32 GB | ~177 tok/s | 256K (full) | $6,500 (new) | $37 |
| NVIDIA RTX A6000 | 48 GB | ~130 tok/s | 256K (full) | $8,363 (new) | $64 |
Prices: gpuprix.com, 2026-11-02; see how prices are sourced. Prices move daily. Bar length shows value: longer is cheaper per tok/s.