What local AI models can the AMD Radeon PRO V620 run?
79 of our 115 models, up to Recursive Flint 40B-A2.5B. That is the largest current model that fits the Radeon PRO V620's 32 GB at Q4_1 with room for about 168K tokens of context (its full 256K with 8-bit KV cache), at about 153 tok/s estimate.
Memory type: GDDR6 (source) · Bus width: 256-bit (source) · Released: 2023 (source)
- RDNA 2: no calibration point; uses the AMD constants fitted on RDNA 3/3.5/4.
Specs from first-party pages (AMD), accessed 2026-10-18. Speeds use the AMD (ROCm/Vulkan) constants (how).
As an Amazon Associate I earn from qualifying purchases.
Every model, largest that fits first
| Model | Quant | File | Fits? | Speed estimate | Max context estimate |
|---|---|---|---|---|---|
| Recursive Flint 40B-A2.5B | Q4_1 | 25.99 GB | Fits | ~153 tok/s | 168K · 256K (8-bit) |
| NVIDIA Nemotron 3.5 Lightning 30B-A3B | Q4_K_M | 25.48 GB | Fits | ~91 tok/s | 256K (full) |
| K2-Horizon-MoVA-36B-A4B | Q4_K_M | 22.37 GB | Fits | ~118 tok/s | 42K · 80K (8-bit) |
| Qwen3.6-35B-A3B | Q4_K_M | 22.29 GB | Fits | ~105 tok/s | 256K (full) |
| Qwen3.5-35B-A3B | Q4_K_M | 22.29 GB | Fits | ~105 tok/s | 256K (full) |
| NVIDIA Nemotron 3.5 Bolt 74B-A9B 24GB (2-bit) | native 2-bit | 22.02 GB | Fits | ~87 tok/s | 256K (full) |
| MiniCPM-V 4.7 35B-A3B | Q4_K_M | 21.17 GB | Fits | ~107 tok/s | 256K (full) |
| K2-Horizon-32B | Q4_K_M | 21.08 GB | Fits | ~19 tok/s | 36K · 69K (8-bit) |
| Nanbeige4.3-33B | Q4_K_M | 20.79 GB | Fits | ~19 tok/s | 46K · 88K (8-bit) |
| Xing4.0-29B-A4B | Q4_K_M | 20.06 GB | Fits | ~112 tok/s | 231K · 256K (8-bit) |
| Raon-1 mini preview Early support | Q4_K_M | 19.69 GB | Weights fit context unknown | ~82 tok/s | Unknown (no KV data) |
| PLaMo 3 Translate 31B Translation only | Q4_K_M | 19.53 GB | Fits | ~20 tok/s | 32K (full) |
| Meta Muse Glimmer 30B | Q4_K_M | 18.39 GB | Fits | ~21 tok/s | 128K (full) |
| Gemma 4 31B | Q4_K_M | 18.32 GB | Fits | ~22 tok/s | 136K · 256K (8-bit) |
| GLM-4.7-Flash | Q4_K_M | 18.31 GB | Fits | ~146 tok/s | 198K (full) |
| Qwen3.6-27B | Q4_K_M | 17.98 GB | Fits | ~21 tok/s | 194K · 256K (8-bit) |
| Qwen3.5-27B | Q4_K_M | 17.98 GB | Fits | ~21 tok/s | 194K · 256K (8-bit) |
| IBM Granite 4.2 30B | Q4_K_M | 17.72 GB | Fits | ~22 tok/s | 49K · 93K (8-bit) |
| Microsoft Fara1.5 27B | Q4_K_M | 17.53 GB | Fits | ~21 tok/s | 200K · 256K (8-bit) |
| Qwen3.8-27B | Q4_K_M | 17.44 GB | Fits | ~21 tok/s | 202K · 256K (8-bit) |
| Gemma 4 26B A4B | Q4_K_M | 17.04 GB | Fits | ~114 tok/s | 256K (full) |
| Kvark Sparkle 1.1 27B | Q4_K_M | 16.81 GB | Fits | ~22 tok/s | 211K · 256K (8-bit) |
| LFM2.5-24B-A2B | Q4_K_M | 14.53 GB | Fits | ~127 tok/s | 128K (full) |
| Mistral-Small-3.2-24B-Instruct-2506 | Q4_K_M | 14.33 GB | Fits | ~28 tok/s | 99K · 128K (8-bit) |
| Devstral-Small-2505 | Q4_K_M | 14.33 GB | Fits | ~28 tok/s | 99K · 128K (8-bit) |
| gpt-oss-20b | Q4_K_M | 11.67 GB | Fits | ~133 tok/s | 128K (full) |
| phi-4 | Q4_K_M | 9.05 GB | Fits | ~44 tok/s | 16K (full) |
| JetBrains Mellum2.1 12B-A2.5B Thinking | Q4_K_M | 8.07 GB | Fits | ~151 tok/s | 128K (full) |
| Gemma 4 12B | Q4_K_M | 7.12 GB | Fits | ~55 tok/s | 256K (full) |
| NVIDIA Nemotron 3.5 Bolt E4B | Q4_K_M | 6.18 GB | Fits | ~64 tok/s | 64K (full) |
| Qwen3.5-9B | Q4_K_M | 6.17 GB | Fits | ~55 tok/s | 256K (full) |
| Ternary Bonsai 2 27B | PTQ1_0 ternary | 5.95 GB | Fits | ~57 tok/s | 256K (full) |
| Microsoft Fara1.5 9B | Q4_K_M | 5.91 GB | Fits | ~58 tok/s | 256K (full) |
| Xiaomi MiMo-V2.6-Distill-Qwen-9B | Q4_K_M | 5.84 GB | Fits | ~58 tok/s | 256K (full) |
| ZoomGain-9B | Q4_K_M | 5.63 GB | Fits | ~60 tok/s | 256K (full) |
| K2-Horizon-7B | Q4_K_M | 5.59 GB | Fits | ~71 tok/s | 168K · 316K (8-bit) |
| webAI TwIL-8B Math and code reasoning | Q4_K_M | 5.35 GB | Fits | ~74 tok/s | 128K (full) |
| IBM Granite 4.2 8B | Q4_K_M | 5.35 GB | Fits | ~74 tok/s | 128K (full) |
| PLaMo 3 Translate 8B Translation only | Q4_K_M | 5.28 GB | Fits | ~75 tok/s | 32K (full) |
| LFM2.5-8B-A1B | Q4_K_M | 5.16 GB | Fits | ~140 tok/s | 128K (full) |
| Gemma 4 E4B | Q4_K_M | 4.98 GB | Fits | ~79 tok/s | 128K (full) |
| Spark-X2.6-4B | Q4_K_M | 3.54 GB | Fits | ~111 tok/s | 724K · 1M (8-bit) |
| Gemma 4 E2B | Q4_K_M | 3.46 GB | Fits | ~114 tok/s | 128K (full) |
| Qwen3.5-4B | Q4_K_M | 3.01 GB | Fits | ~100 tok/s | 256K (full) |
| NVIDIA Nemotron 3 Nano 4B | Q4_K_M | 2.9 GB | Fits | ~103 tok/s | 256K (full) |
| Microsoft Fara1.5 4B | Q4_K_M | 2.88 GB | Fits | ~103 tok/s | 256K (full) |
| ZoomGain-4B | Q4_K_M | 2.71 GB | Fits | ~108 tok/s | 256K (full) |
| IBM Granite 4.2 3B | Q4_K_M | 2.24 GB | Fits | ~176 tok/s | 128K (full) |
| Raon-1 nano Early support | Q4_K_M | 2.08 GB | Weights fit context unknown | ~130 tok/s | Unknown (no KV data) |
| Ternary Bonsai 2 9B | PTQ1_0 ternary | 2 GB | Fits | ~134 tok/s | 256K (full) |
| PLaMo 3 Translate 2B Translation only | Q4_K_M | 1.81 GB | Fits | ~218 tok/s | 32K (full) |
| LFM3-1.7B | Q4_K_M | 1.1 GB | Fits | ~193 tok/s | 256K (full) |
| LFM3-VL-2B | Q4_K_M | 1.1 GB | Fits | ~193 tok/s | 128K (full) |
| Ternary Bonsai 2 4B | PTQ1_0 ternary | 0.95 GB | Fits | ~208 tok/s | 256K (full) |
| MiniCPM-V 4.7 1B | Q4_K_M | 0.67 GB | Fits | ~244 tok/s | 256K (full) |
| PLaMo 3 Translate 0.8B Translation only | Q4_K_M | 0.57 GB | Fits | ~692 tok/s | 32K (full) |
| BotPsy-0.8B | Q4_K_M | 0.53 GB | Fits | ~267 tok/s | 256K (full) |
| Ternary Bonsai 2 2B | PTQ1_0 ternary | 0.43 GB | Fits | ~286 tok/s | 256K (full) |
| Qwen2.5-Coder-32B-Instruct Older | Q4_K_M | 19.85 GB | Fits | ~20 tok/s | 32K (full) |
| DeepSeek-R1-Distill-Qwen-32B Older | Q4_K_M | 19.85 GB | Fits | ~20 tok/s | 41K · 78K (8-bit) |
| Qwen3-32B Older | Q4_K_M | 19.76 GB | Fits | ~20 tok/s | 40K (full) |
| Qwen3-30B-A3B Older | Q4_K_M | 18.63 GB | Fits | ~133 tok/s | 40K (full) |
| Qwen3-Coder-30B-A3B-Instruct Older | Q4_K_M | 18.56 GB | Fits | ~134 tok/s | 123K · 232K (8-bit) |
| gemma-3-27b-it Older | Q4_K_M | 16.55 GB | Fits | ~24 tok/s | 128K (full) |
| LFM2-24B-A2B Older | Q4_K_M | 14.42 GB | Fits | ~127 tok/s | 128K (full) |
| Qwen3-14B Older | Q4_K_M | 9 GB | Fits | ~44 tok/s | 40K (full) |
| Qwen2.5-Coder-14B-Instruct Older | Q4_K_M | 8.99 GB | Fits | ~44 tok/s | 32K (full) |
| DeepSeek-R1-Distill-Qwen-14B Older | Q4_K_M | 8.99 GB | Fits | ~44 tok/s | 109K · 128K (8-bit) |
| Mistral-Nemo-Instruct-2407 Older | Q4_K_M | 7.48 GB | Fits | ~53 tok/s | 128K (full) |
| gemma-3-12b-it Older | Q4_K_M | 7.3 GB | Fits | ~54 tok/s | 128K (full) |
| NVIDIA-Nemotron-Nano-9B-v2 Older | Q4_K_M | 6.53 GB | Fits | ~53 tok/s | 128K (full) |
| Qwen3-8B Older | Q4_K_M | 5.03 GB | Fits | ~78 tok/s | 40K (full) |
| Llama-3.1-8B-Instruct Older | Q4_K_M | 4.92 GB | Fits | ~80 tok/s | 128K (full) |
| Qwen2.5-Coder-7B-Instruct Older | Q4_K_M | 4.68 GB | Fits | ~84 tok/s | 32K (full) |
| Mistral-7B-Instruct-v0.3 Older | Q4_K_M | 4.37 GB | Fits | ~90 tok/s | 32K (full) |
| Qwen3-4B Older | Q4_K_M | 2.5 GB | Fits | ~158 tok/s | 40K (full) |
| gemma-3-4b-it Older | Q4_K_M | 2.49 GB | Fits | ~158 tok/s | 128K (full) |
| Llama-3.2-3B-Instruct Older | Q4_K_M | 2.02 GB | Fits | ~195 tok/s | 128K (full) |
| Qwen3-0.6B Older | Q4_K_M | 0.48 GB | Fits | ~821 tok/s | 40K (full) |
| Ripple-65B Preview | Q4_K_M | 38.8 GB | No | — | — |
| Llama-3.3-70B-Instruct Older | Q4_K_M | 42.52 GB | No | — | — |
| DeepSeek-R1-Distill-Llama-70B Older | Q4_K_M | 42.52 GB | No | — | — |
| Qwen2.5-72B-Instruct Older | Q4_K_M | 47.42 GB | No | — | — |
| Qwen3-Coder-Next | Q4_K_M | 48.53 GB | No | — | — |
| Aleph Alpha Kolibri-1 78B-A3.5B | Q4_K_M | 49.76 GB | No | — | — |
| NVIDIA Nemotron 3.5 Bolt 74B-A9B | UD-Q4_K_XL | 50.22 GB | No | — | — |
| gpt-oss-120b | Q4_K_M | 62.84 GB | No | — | — |
| NVIDIA Nemotron 3.5 Super VL 120B-A12B | Q4_K_M | 70.1 GB | No | — | — |
| Mistral Small 4 119B | Q4_K_M | 72.64 GB | No | — | — |
| GLM-4.5-Air Older | Q4_K_M | 73.5 GB | No | — | — |
| Qwen3.5-122B-A10B | Q4_K_M | 77.62 GB | No | — | — |
| Mistral Medium 3.5 128B | Q4_K_M | 78.41 GB | No | — | — |
| Ornith-1.5-122B | Q4_K_M | 78.92 GB | No | — | — |
| NVIDIA Nemotron 3 Super 120B-A12B Older | UD-Q4_K_M | 82.54 GB | No | — | — |
| Swift 1.5 Qwen3.8-Flash-Next | Q4_K_M | 119.6 GB | No | — | — |
| Qwen3.8-Flash-Next | Q4_K_M | 119.6 GB | No | — | — |
| StepFun Step 3.7 Flash | UD-Q4_K_M | 122.09 GB | No | — | — |
| Qwen3-235B-A22B | Q4_K_M | 142.15 GB | No | — | — |
| MiniMax-M3.1-Flash Unsloth 2-bit | UD-Q2_K_XL | 143.91 GB | No | — | — |
| DeepSeek-V4-Flash | UD-Q4_K_XL | 155.1 GB | No | — | — |
| MiniMax-M3.1-Flash Q2_K | Q2_K | 155.91 GB | No | — | — |
| Thinking Machines Inkling-Small Older | Q4_K_M | 159.34 GB | No | — | — |
| Xiaomi MiMo-V2.6-Flash 309B-A15B | MXFP4 | 167.37 GB | No | — | — |
| GLM-5.3-Flash | UD-Q4_K_XL | 199.71 GB | No | — | — |
| MiniMax-M3.1-Flash | Q4_K_M | 258.01 GB | No | — | — |
| Reflection Beam 1.0 501B-A23B | Q4_K_M | 302.41 GB | No | — | — |
| Thinking Machines Inkling 2 Small Early support | UD-Q4_K_XL | 354.15 GB | No | — | — |
| inclusionAI Ling-3.1-flash 560B-A25B | Q4_K_M | 357.97 GB | No | — | — |
| DeepSeek-V4.1-Flash (1017) | UD-Q4_K_XL | 500.71 GB | No | — | — |
| Meta Muse Spark Open 816B-A42B | UD-Q4_K_XL | 505.5 GB | No | — | — |
| Mistral Large 4 | UD-Q4_K_XL | 605 GB | No | — | — |
Q4_K_M unless the row says otherwise; UD- sizes are unsloth dynamic quants, the only Q4 build published for those models. Max context is an estimate of the longest context whose KV cache fits beside the model: default FP16 cache, then 8-bit (q8_0).
Speed calculator
estimateEstimate = bandwidth ÷ GB read per token (the model's listed GGUF file: Q4_K_M for most models, otherwise the quant named in the list; scaled by active/total parameters for MoE), scaled by a per-platform efficiency and fixed per-token costs calibrated to 45 measured llama.cpp benchmarks (2025–26). Fit = file size + 1.5 GB headroom ≤ total VRAM (on Macs, the part of unified memory macOS lets the GPU use by default), and room for at least 4K tokens of context with an 8-bit KV cache. For a model with no KV cache data, only the file and headroom are checked, so a fit says nothing about context. Multi-GPU: VRAM adds, speed stays one card's (a layer split runs the cards in turn); max context uses the total. Speed is for short context. Max context = what is left after the file and a 1 GB runtime reserve ÷ KV cache bytes per token, default FP16 and 8-bit (q8_0) cache. How speed works · how context works