What local AI models can the NVIDIA RTX A400 run?
15 of our 115 models, up to LFM3-1.7B with long context. It is the largest current model that keeps room for long context on the RTX A400's 4 GB: at Q4 it fits with room for about 150K tokens of context (its full 256K with 8-bit KV cache), at about 60 tok/s estimate. The largest that fits at all is IBM Granite 4.2 3B with room for about 9K tokens of context (17K with 8-bit KV cache).
Memory type: GDDR6 (source)
- bus_bits: line card gives none.
- year: no NVIDIA launch page was opened (announced April 2024 per search results).
Specs from first-party pages (valuehub.hp.com), accessed 2026-10-18. Speeds use the NVIDIA (CUDA) constants (how).
As an Amazon Associate I earn from qualifying purchases.
Every model, largest that fits first
| Model | Quant | File | Fits? | Speed estimate | Max context estimate |
|---|---|---|---|---|---|
| IBM Granite 4.2 3B | Q4_K_M | 2.24 GB | Fits | ~36 tok/s | 9K · 17K (8-bit) |
| Ternary Bonsai 2 9B | PTQ1_0 ternary | 2 GB | Fits | ~37 tok/s | 29K · 56K (8-bit) |
| PLaMo 3 Translate 2B Translation only | Q4_K_M | 1.81 GB | Fits | ~44 tok/s | 32K (full) |
| LFM3-1.7B | Q4_K_M | 1.1 GB | Fits | ~60 tok/s | 150K · 256K (8-bit) |
| LFM3-VL-2B | Q4_K_M | 1.1 GB | Fits | ~60 tok/s | 128K (full) |
| Ternary Bonsai 2 4B | PTQ1_0 ternary | 0.95 GB | Fits | ~67 tok/s | 61K · 115K (8-bit) |
| MiniCPM-V 4.7 1B | Q4_K_M | 0.67 GB | Fits | ~87 tok/s | 185K · 256K (8-bit) |
| PLaMo 3 Translate 0.8B Translation only | Q4_K_M | 0.57 GB | Fits | ~126 tok/s | 32K (full) |
| BotPsy-0.8B | Q4_K_M | 0.53 GB | Fits | ~101 tok/s | 196K · 256K (8-bit) |
| Ternary Bonsai 2 2B | PTQ1_0 ternary | 0.43 GB | Fits | ~115 tok/s | 204K · 256K (8-bit) |
| gemma-3-4b-it Older | Q4_K_M | 2.49 GB | Fits | ~33 tok/s | 15K · 37K (8-bit) |
| Llama-3.2-3B-Instruct Older | Q4_K_M | 2.02 GB | Fits | ~40 tok/s | 8K · 15K (8-bit) |
| Qwen3-0.6B Older | Q4_K_M | 0.48 GB | Fits | ~145 tok/s | 21K · 40K (8-bit) |
| Raon-1 nano Early support | Q4_K_M | 2.08 GB | Weights fit context unknown | ~35 tok/s | Unknown (no KV data) |
| Qwen3-4B Older | Q4_K_M | 2.5 GB | Fits | ~32 tok/s | 3K · 6K (8-bit) Tight |
| ZoomGain-4B | Q4_K_M | 2.71 GB | No | — | — |
| Microsoft Fara1.5 4B | Q4_K_M | 2.88 GB | No | — | — |
| NVIDIA Nemotron 3 Nano 4B | Q4_K_M | 2.9 GB | No | — | — |
| Qwen3.5-4B | Q4_K_M | 3.01 GB | No | — | — |
| Gemma 4 E2B | Q4_K_M | 3.46 GB | No | — | — |
| Spark-X2.6-4B | Q4_K_M | 3.54 GB | No | — | — |
| Mistral-7B-Instruct-v0.3 Older | Q4_K_M | 4.37 GB | No | — | — |
| Qwen2.5-Coder-7B-Instruct Older | Q4_K_M | 4.68 GB | No | — | — |
| Gemma 4 E4B | Q4_K_M | 4.98 GB | No | — | — |
| Llama-3.1-8B-Instruct Older | Q4_K_M | 4.92 GB | No | — | — |
| Qwen3-8B Older | Q4_K_M | 5.03 GB | No | — | — |
| LFM2.5-8B-A1B | Q4_K_M | 5.16 GB | No | — | — |
| PLaMo 3 Translate 8B Translation only | Q4_K_M | 5.28 GB | No | — | — |
| webAI TwIL-8B Math and code reasoning | Q4_K_M | 5.35 GB | No | — | — |
| IBM Granite 4.2 8B | Q4_K_M | 5.35 GB | No | — | — |
| K2-Horizon-7B | Q4_K_M | 5.59 GB | No | — | — |
| ZoomGain-9B | Q4_K_M | 5.63 GB | No | — | — |
| Xiaomi MiMo-V2.6-Distill-Qwen-9B | Q4_K_M | 5.84 GB | No | — | — |
| Ternary Bonsai 2 27B | PTQ1_0 ternary | 5.95 GB | No | — | — |
| Microsoft Fara1.5 9B | Q4_K_M | 5.91 GB | No | — | — |
| NVIDIA Nemotron 3.5 Bolt E4B | Q4_K_M | 6.18 GB | No | — | — |
| Qwen3.5-9B | Q4_K_M | 6.17 GB | No | — | — |
| NVIDIA-Nemotron-Nano-9B-v2 Older | Q4_K_M | 6.53 GB | No | — | — |
| Gemma 4 12B | Q4_K_M | 7.12 GB | No | — | — |
| gemma-3-12b-it Older | Q4_K_M | 7.3 GB | No | — | — |
| Mistral-Nemo-Instruct-2407 Older | Q4_K_M | 7.48 GB | No | — | — |
| JetBrains Mellum2.1 12B-A2.5B Thinking | Q4_K_M | 8.07 GB | No | — | — |
| Qwen3-14B Older | Q4_K_M | 9 GB | No | — | — |
| Qwen2.5-Coder-14B-Instruct Older | Q4_K_M | 8.99 GB | No | — | — |
| DeepSeek-R1-Distill-Qwen-14B Older | Q4_K_M | 8.99 GB | No | — | — |
| phi-4 | Q4_K_M | 9.05 GB | No | — | — |
| gpt-oss-20b | Q4_K_M | 11.67 GB | No | — | — |
| Mistral-Small-3.2-24B-Instruct-2506 | Q4_K_M | 14.33 GB | No | — | — |
| Devstral-Small-2505 | Q4_K_M | 14.33 GB | No | — | — |
| LFM2-24B-A2B Older | Q4_K_M | 14.42 GB | No | — | — |
| LFM2.5-24B-A2B | Q4_K_M | 14.53 GB | No | — | — |
| gemma-3-27b-it Older | Q4_K_M | 16.55 GB | No | — | — |
| Kvark Sparkle 1.1 27B | Q4_K_M | 16.81 GB | No | — | — |
| Gemma 4 26B A4B | Q4_K_M | 17.04 GB | No | — | — |
| Qwen3.8-27B | Q4_K_M | 17.44 GB | No | — | — |
| Microsoft Fara1.5 27B | Q4_K_M | 17.53 GB | No | — | — |
| IBM Granite 4.2 30B | Q4_K_M | 17.72 GB | No | — | — |
| Qwen3.6-27B | Q4_K_M | 17.98 GB | No | — | — |
| Qwen3.5-27B | Q4_K_M | 17.98 GB | No | — | — |
| Meta Muse Glimmer 30B | Q4_K_M | 18.39 GB | No | — | — |
| GLM-4.7-Flash | Q4_K_M | 18.31 GB | No | — | — |
| Qwen3-Coder-30B-A3B-Instruct Older | Q4_K_M | 18.56 GB | No | — | — |
| Gemma 4 31B | Q4_K_M | 18.32 GB | No | — | — |
| Qwen3-30B-A3B Older | Q4_K_M | 18.63 GB | No | — | — |
| PLaMo 3 Translate 31B Translation only | Q4_K_M | 19.53 GB | No | — | — |
| Raon-1 mini preview Early support | Q4_K_M | 19.69 GB | No | — | — |
| Qwen3-32B Older | Q4_K_M | 19.76 GB | No | — | — |
| Qwen2.5-Coder-32B-Instruct Older | Q4_K_M | 19.85 GB | No | — | — |
| DeepSeek-R1-Distill-Qwen-32B Older | Q4_K_M | 19.85 GB | No | — | — |
| Xing4.0-29B-A4B | Q4_K_M | 20.06 GB | No | — | — |
| Nanbeige4.3-33B | Q4_K_M | 20.79 GB | No | — | — |
| MiniCPM-V 4.7 35B-A3B | Q4_K_M | 21.17 GB | No | — | — |
| K2-Horizon-32B | Q4_K_M | 21.08 GB | No | — | — |
| NVIDIA Nemotron 3.5 Bolt 74B-A9B 24GB (2-bit) | native 2-bit | 22.02 GB | No | — | — |
| Qwen3.6-35B-A3B | Q4_K_M | 22.29 GB | No | — | — |
| Qwen3.5-35B-A3B | Q4_K_M | 22.29 GB | No | — | — |
| K2-Horizon-MoVA-36B-A4B | Q4_K_M | 22.37 GB | No | — | — |
| NVIDIA Nemotron 3.5 Lightning 30B-A3B | Q4_K_M | 25.48 GB | No | — | — |
| Recursive Flint 40B-A2.5B | Q4_1 | 25.99 GB | No | — | — |
| Ripple-65B Preview | Q4_K_M | 38.8 GB | No | — | — |
| Llama-3.3-70B-Instruct Older | Q4_K_M | 42.52 GB | No | — | — |
| DeepSeek-R1-Distill-Llama-70B Older | Q4_K_M | 42.52 GB | No | — | — |
| Qwen2.5-72B-Instruct Older | Q4_K_M | 47.42 GB | No | — | — |
| Qwen3-Coder-Next | Q4_K_M | 48.53 GB | No | — | — |
| Aleph Alpha Kolibri-1 78B-A3.5B | Q4_K_M | 49.76 GB | No | — | — |
| NVIDIA Nemotron 3.5 Bolt 74B-A9B | UD-Q4_K_XL | 50.22 GB | No | — | — |
| gpt-oss-120b | Q4_K_M | 62.84 GB | No | — | — |
| NVIDIA Nemotron 3.5 Super VL 120B-A12B | Q4_K_M | 70.1 GB | No | — | — |
| Mistral Small 4 119B | Q4_K_M | 72.64 GB | No | — | — |
| GLM-4.5-Air Older | Q4_K_M | 73.5 GB | No | — | — |
| Qwen3.5-122B-A10B | Q4_K_M | 77.62 GB | No | — | — |
| Mistral Medium 3.5 128B | Q4_K_M | 78.41 GB | No | — | — |
| Ornith-1.5-122B | Q4_K_M | 78.92 GB | No | — | — |
| NVIDIA Nemotron 3 Super 120B-A12B Older | UD-Q4_K_M | 82.54 GB | No | — | — |
| Swift 1.5 Qwen3.8-Flash-Next | Q4_K_M | 119.6 GB | No | — | — |
| Qwen3.8-Flash-Next | Q4_K_M | 119.6 GB | No | — | — |
| StepFun Step 3.7 Flash | UD-Q4_K_M | 122.09 GB | No | — | — |
| Qwen3-235B-A22B | Q4_K_M | 142.15 GB | No | — | — |
| MiniMax-M3.1-Flash Unsloth 2-bit | UD-Q2_K_XL | 143.91 GB | No | — | — |
| DeepSeek-V4-Flash | UD-Q4_K_XL | 155.1 GB | No | — | — |
| MiniMax-M3.1-Flash Q2_K | Q2_K | 155.91 GB | No | — | — |
| Thinking Machines Inkling-Small Older | Q4_K_M | 159.34 GB | No | — | — |
| Xiaomi MiMo-V2.6-Flash 309B-A15B | MXFP4 | 167.37 GB | No | — | — |
| GLM-5.3-Flash | UD-Q4_K_XL | 199.71 GB | No | — | — |
| MiniMax-M3.1-Flash | Q4_K_M | 258.01 GB | No | — | — |
| Reflection Beam 1.0 501B-A23B | Q4_K_M | 302.41 GB | No | — | — |
| Thinking Machines Inkling 2 Small Early support | UD-Q4_K_XL | 354.15 GB | No | — | — |
| inclusionAI Ling-3.1-flash 560B-A25B | Q4_K_M | 357.97 GB | No | — | — |
| DeepSeek-V4.1-Flash (1017) | UD-Q4_K_XL | 500.71 GB | No | — | — |
| Meta Muse Spark Open 816B-A42B | UD-Q4_K_XL | 505.5 GB | No | — | — |
| Mistral Large 4 | UD-Q4_K_XL | 605 GB | No | — | — |
Q4_K_M unless the row says otherwise; UD- sizes are unsloth dynamic quants, the only Q4 build published for those models. Max context is an estimate of the longest context whose KV cache fits beside the model: default FP16 cache, then 8-bit (q8_0).
Speed calculator
estimateEstimate = bandwidth ÷ GB read per token (the model's listed GGUF file: Q4_K_M for most models, otherwise the quant named in the list; scaled by active/total parameters for MoE), scaled by a per-platform efficiency and fixed per-token costs calibrated to 45 measured llama.cpp benchmarks (2025–26). Fit = file size + 1.5 GB headroom ≤ total VRAM (on Macs, the part of unified memory macOS lets the GPU use by default), and room for at least 4K tokens of context with an 8-bit KV cache. For a model with no KV cache data, only the file and headroom are checked, so a fit says nothing about context. Multi-GPU: VRAM adds, speed stays one card's (a layer split runs the cards in turn); max context uses the total. Speed is for short context. Max context = what is left after the file and a 1 GB runtime reserve ÷ KV cache bytes per token, default FP16 and 8-bit (q8_0) cache. How speed works · how context works