Can the GeForce RTX 3090 run K2-Horizon-MoVA-36B-A4B?

Yes, with short context only. On the NVIDIA GeForce RTX 3090 (24 GB) there is room for about 3K tokens of context with the default FP16 KV cache, or 5K with an 8-bit (q8_0) KV cacheestimate: what is left after the weights and a 1 GB runtime reserve. That is under 8K tokens at FP16, enough for short chats; long documents need the 8-bit KV cache or a smaller quant. The Q4_K_M GGUF of K2-Horizon-MoVA-36B-A4B is 22.37 GB (source); with our 1.5 GB allowance for the runtime and a short context it needs about 23.9 GB.

K2-Horizon-MoVA-36B-A4B fits 5 of the 18 GPUs we have a dated price for; the smallest of them has 24 GB. The GeForce RTX 3090's 936 GB/s gives ~217 tok/s estimate, 70% of the fastest card, the GeForce RTX 5090 at ~311 tok/s. Only 4B of its 37.44B parameters are active per token, so the estimate reads 2.39 GB per token rather than the whole 22.37 GB file, plus a per-token routing and expert-gather cost calibrated to 45 measured llama.cpp benchmarks (2025–26). On the same card, the mixture-of-experts Qwen3.6-35B-A3B (3B of 35.95B active, 22.29 GB) estimates ~157 tok/s.

The cheapest way to run it per tok/s. At $1,499 (used) the GeForce RTX 3090 works out to $7 per tok/s; the next best is the Radeon RX 7900 XTX at $8. Our 24 GB: one used RTX 3090 build is a complete PC with this card for $2,650.58, about $12 per tok/s for the whole machine.