Can the GeForce RTX 4090 run K2-Horizon-MoVA-36B-A4B?
Yes, with short context. A used GeForce RTX 4090 ($2,910) runs K2-Horizon-MoVA-36B-A4B at Q4 with room for only about 3K tokens of context (5K with 8-bit KV cache), at about 228 tok/s estimate. That is $13 per tok/s, #3 of 5 priced GPUs that fit it; the GeForce RTX 3090 ranks first at $7.
Yes, with short context only. On the NVIDIA GeForce RTX 4090 (24 GB) there is room for about 3K tokens of context with the default FP16 KV cache, or 5K with an 8-bit (q8_0) KV cacheestimate: what is left after the weights and a 1 GB runtime reserve. That is under 8K tokens at FP16, enough for short chats; long documents need the 8-bit KV cache or a smaller quant. The Q4_K_M GGUF of K2-Horizon-MoVA-36B-A4B is 22.37 GB (source); with our 1.5 GB allowance for the runtime and a short context it needs about 23.9 GB.
K2-Horizon-MoVA-36B-A4B fits 5 of the 18 GPUs we have a dated price for; the smallest of them has 24 GB. The GeForce RTX 4090's 1008 GB/s gives ~228 tok/s estimate, 73% of the fastest card, the GeForce RTX 5090 at ~311 tok/s. Only 4B of its 37.44B parameters are active per token, so the estimate reads 2.39 GB per token rather than the whole 22.37 GB file, plus a per-token routing and expert-gather cost calibrated to 45 measured llama.cpp benchmarks (2025–26). On the same card, the mixture-of-experts Qwen3.6-35B-A3B (3B of 35.95B active, 22.29 GB) estimates ~161 tok/s.
At $2,910 (used) it works out to $13 per tok/s, #3 of 5. The GeForce RTX 3090 is the cheapest per tok/s at $7 ($1,499, used). The cheapest complete build we priced that runs K2-Horizon-MoVA-36B-A4B is our 24 GB: one used RTX 3090 build at $2,650.58.
Last updated Oct 16, 2026. Speed is a bandwidth-based estimate (1008 GB/s, NVIDIA), scaled by per-platform constants calibrated to 45 measured llama.cpp benchmarks (2025–26), not a benchmark of this pair. Max context is an estimate from the model's architecture for llama.cpp with flash attention.
Next size up: NVIDIA Nemotron 3.5 Lightning 30B-A3B (25.48 GB at Q4_K_M) needs ~27 GB and does not fit on one NVIDIA GeForce RTX 4090. See what runs NVIDIA Nemotron 3.5 Lightning 30B-A3B.
Every priced GPU that runs K2-Horizon-MoVA-36B-A4B, by cost per token/sec
| GPU | VRAM | Speed estimate | Max context estimate | Low price | $ per tok/s |
|---|---|---|---|---|---|
| NVIDIA GeForce RTX 3090 | 24 GB | ~217 tok/s | 3K · 5K (8-bit) Tight | $1,499 (used) | $7 |
| AMD Radeon RX 7900 XTX | 24 GB | ~178 tok/s | 3K · 5K (8-bit) Tight | $1,350 (new) | $8 |
| NVIDIA GeForce RTX 4090 | 24 GB | ~228 tok/s | 3K · 5K (8-bit) Tight | $2,910 (used) | $13 |
| NVIDIA GeForce RTX 5090 | 32 GB | ~311 tok/s | 42K · 80K (8-bit) | $6,500 (new) | $21 |
| NVIDIA RTX A6000 | 48 GB | ~191 tok/s | 122K · 230K (8-bit) | $8,363 (new) | $44 |
Prices: gpusold.com, 2026-11-02; see how prices are sourced. Prices move daily. Bar length shows value: longer is cheaper per tok/s.