What GPU do you need to run Raon-1 mini preview?
Weights-only fit: a new Radeon RX 7900 XTX ($1,350). It is the cheapest priced graphics card that holds Raon-1 mini preview (about 21.2 GB at Q4) (weights-only fit, context unknown: no KV cache data; 2.8 GB left beyond the file and headroom), at about 114 tok/s estimate; that is the card price only, and it requires a separate compatible PC. The weights fit 54 of the 157 GPUs and systems we track; with no KV cache data, context is unknown on all of them.
Early support Runs on KRAFTON's llama.cpp fork and MLX build. Stock llama.cpp, Ollama and LM Studio can't load Raon-1 mini preview yet. Upstream llama.cpp has no Raon-1 architecture (these GGUFs use raon1). Pull request #31107, which would have added it, was closed unmerged on Oct 24, 2026, after a maintainer asked KRAFTON to submit its own code. KRAFTON's GGUF card says to build from its fork: git clone -b raon-1-mini-preview https://github.com/krafton-ai/llama.cpp. On a Mac, KRAFTON's MLX build (KRAFTON/Raon-1-mini-preview-mlx, in NVIDIA's 4-bit NVFP4 format) needs trust_remote_code=True and pins mlx==0.32.4, mlx-lm==0.32.2 and transformers==5.20.1. The fits and speeds on this page are math on the official Q4_K_M file, not a test on the fork. (source, checked Nov 11, 2026.)
28.8B parameters, 4.3B active (mixture-of-experts) (Hugging Face, accessed 2026-11-11). Q4_K_M GGUF: 19.69 GB (source). No context estimate: Raon-1 mini preview's architecture is not in upstream llama.cpp, so there is no verified llama.cpp KV formula for it yet. KRAFTON says only its 7 gated MLA layers keep a KV cache, about 8 KiB per token in BF16, and the 18 Raon Delta Attention layers keep a fixed-size state. That is KRAFTON's figure, not checked here (details).
KRAFTON AI's mixture-of-experts model, trained from scratch, for reasoning, coding and agent work with tool calling. It is a preview: KRAFTON says specs and results may change before the final version. Of its 25 layers, 18 are Raon Delta Attention (linear attention with a fixed-size state) and 7 are gated MLA (multi-head latent attention). It has 128 routed experts, and each token uses 6 of them plus 1 shared expert. KRAFTON gives 28.8B total and 4.3B active parameters. Q4_K_M is KRAFTON's official GGUF (raon-1-mini-preview-Q4_K_M.gguf, 19,686,434,240 bytes), made with KRAFTON's own llama.cpp fork; a smaller official IQ4_XS (16.07 GB) and every other size are in the table below. Before downloading, you request access once on the Hugging Face page (KRAFTON says approval is automatic) and log in. Apache 2.0, including commercial use of the downloaded weights; KRAFTON's hosted API and playground have separate terms. Release date is the Hugging Face repo creation date (Oct 21, 2026, UTC).
What we checked
Checked Nov 11, 2026. Official is the maker or the weight files. A vendor claim is that project's own documentation. A community claim is someone else's upload. Our check is a file or search we read. Unknown means we have no source, so it is not estimated. None of these rows is a speed we measured.
| Kind | What | Detail | Source |
|---|---|---|---|
| Official | Model card | KRAFTON's card: a 30B-A4B mixture-of-experts model trained from scratch. 25 layers (18 Raon Delta Attention, 7 gated MLA), hidden size 2,560, SiTU-GLU, Raon-mHC with 4 residual streams, no position encoding (NoPE), a 259,840-token vocabulary and an untied output head. 128 routed experts (expert intermediate size 1,088), 6 routed and 1 shared active per token; the first layer is a dense FFN. Native context is 262,144 tokens, with thinking built in. The card covers tool calling with Hermes Agent, Pi and OpenCode, and lists Korean, English, Japanese, Chinese, Spanish, French and Thai. It says: "This is a preview release, not a final model." | Model card, checked Nov 11, 2026 |
| Official | Parameters and config | KRAFTON's table: 28.8B total, 4.3B activated. Hugging Face's counter reads 28,783,218,402 (3,113,981,970 BF16, 25,669,140,480 F8_E4M3 and 95,952 F32), because the main repo stores the routed experts in MXFP8. config.json: 25 layers, hidden size 2,560, 128 experts, 6 per token, 1 shared, q_lora_rank 576, kv_lora_rank 512, untied embeddings, 262,144 max positions. | config.json, checked Nov 11, 2026 |
| Official | Access and license | Apache 2.0, including commercial use of the downloaded weights. The hosted API and playground have separate terms. KRAFTON's cards say: "The model repositories are gated. Request access once on this page (approval is automatic; it covers every Raon-1 mini preview repository), then log in so the examples below can download them." Every card shows that request form, though Hugging Face's API reports the repos as not gated. Expect the request step either way. This is a summary of the license, not legal advice. | Model card, checked Nov 11, 2026 |
| Official | Weight files | Seven safetensors shards sum to 32,702,145,088 bytes (32.70 GB); the index lists 32,699,649,368 bytes of tensors. These are downloads, not the memory a runtime needs. | Hugging Face API, checked Nov 11, 2026 |
| Official | GGUF files | KRAFTON's own GGUF repo (GGUF architecture raon1). The card says the files were converted from the BF16 weights of the same checkpoint, and that Q6_K and smaller used an importance matrix (imatrix). It has IQ4_XS, Q4_K_M, Q5_K_M, Q6_K and Q8_0; the fit on this page uses Q4_K_M. There is also a 732,539,232-byte DFlash2 draft model (dflash-raon-1-mini-preview-Q8_0.gguf), which is only for speculative decoding and is not a fit file. Every size is in the table below. | GGUF repo, checked Nov 11, 2026 |
| Official | MLX build | KRAFTON/Raon-1-mini-preview-mlx is NVFP4, in four shards summing to 17,537,518,992 bytes, plus a 207,872,547-byte head file and a 971,364,975-byte DFlash2 draft. It ships its own model code, so it needs trust_remote_code=True, and it pins mlx==0.32.4, mlx-lm==0.32.2 and transformers==5.20.1. KRAFTON says the DFlash2 draft needs mlx 0.32.4 and mlx-lm 0.32.x (other versions warn and run without it) and adds about 1 GB of memory. | MLX repo, checked Nov 11, 2026 |
| Vendor claim | NVFP4 on an RTX 5090 (KRAFTON's claim) | KRAFTON's card says the NVFP4 format enables inference on a single NVIDIA RTX 5090 (32 GB). For vLLM it points to a separate checkpoint, KRAFTON/Raon-1-mini-preview-nvfp4. That is KRAFTON's claim, not a fit we checked. The fits on this page use the Q4_K_M GGUF. | Model card, checked Nov 11, 2026 |
| Vendor claim | DFlash2 speculative decoding (KRAFTON's figures) | KRAFTON says its DFlash2 draft model gives up to 2.18× the output throughput of plain decoding on a single NVIDIA B300 (GSM8K, one request at a time, vLLM 0.28.0 with the MXFP8 model). Its card adds that speedups vary by hardware and workload. This is speculative decoding on a datacenter GPU, not plain decoding. The speed estimates on this page do not use it. | Model card, checked Nov 11, 2026 |
| Vendor claim | Benchmark scores (KRAFTON's) | KRAFTON's card compares the model with Qwen3.6-35B-A3B and Gemma 4 26B-A4B, with Qwen3.8-27B and Muse Glimmer 30B as dense references; gpt-5.6-luna was the judge and user simulator. Its scores for Raon-1 mini preview include MMLU Pro 76.5, GPQA Diamond 84.3, AIME 2026 93.3, IFBench 84.2, LiveCodeBench v6 81.6, SWE-Verified 70.8 (mini-SWE-agent harness), Terminal-Bench Lite 47.0, BFCL v4 59.8, BrowseComp Plus 55.7 and AA-LCR 67.3. Chatbot Korean (92.9) is KRAFTON's internal test and is not public. These are KRAFTON's scores, not checked by us. | Model card, checked Nov 11, 2026 |
| Our check | Upstream llama.cpp | Pull request #31107 ("model : add Raon-1 mini preview support") was closed without merging on Oct 24, 2026, after a maintainer asked KRAFTON to submit its own code. The matching request, issue #31106, is also closed. ggml-org/llama.cpp master has no src/models/raon1.cpp. | llama.cpp PR #31107, checked Nov 11, 2026 |
| Our check | Ollama and LM Studio | An Ollama search on 2026-11-11 found no KRAFTON Raon model, only unrelated names. We found no LM Studio listing. | Ollama search, checked Nov 11, 2026 |
GGUF files and sizes
Exact sizes from Hugging Face, accessed 2026-11-11. The fit and context tables on this page use the Q4_K_M file (19.69 GB); a smaller file needs less memory by the difference. VRAM needed is the file + the same headroom.
| File | Published by | Bytes | Size | VRAM needed | Source |
|---|---|---|---|---|---|
| IQ4_XS | KRAFTON | 16,071,801,280 | 16.07 GB | ~17.6 GB | raon-1-mini-preview-IQ4_XS.gguf |
| Q4_K_M | KRAFTON | 19,686,434,240 | 19.69 GB | ~21.2 GB | raon-1-mini-preview-Q4_K_M.gguf |
| Q5_K_M | KRAFTON | 22,183,765,440 | 22.18 GB | ~23.7 GB | raon-1-mini-preview-Q5_K_M.gguf |
| Q6_K | KRAFTON | 25,839,462,080 | 25.84 GB | ~27.3 GB | raon-1-mini-preview-Q6_K.gguf |
| Q8_0 | KRAFTON | 30,693,555,936 | 30.69 GB | ~32.2 GB | raon-1-mini-preview-Q8_0.gguf |
Other places to get it
Checked Nov 14, 2026. We list only copies we opened and checked. Check file hashes for copies from other uploaders.
- ModelScope: Other uploaderSame account name as on Hugging Face; we found no link from the lab to it
Weights-only fits, context unknown. We have no KV cache data for Raon-1 mini preview, so every fit on this page only checks that the Q4_K_M file and headroom fit. How much context fits beside the weights is unknown, so none of these is a proven fit for any context length.
Single GPUs the weights fit, cheapest first
| Single GPU that fits | VRAM | Speed estimate | Max context estimate | Price |
|---|---|---|---|---|
| AMD Radeon RX 7900 XTX | 24 GB | ~114 tok/s | Unknown (no KV data) | low $1,350 |
| NVIDIA GeForce RTX 3090 | 24 GB | ~130 tok/s | Unknown (no KV data) | low $1,499 |
| Apple Mac Studio M5 Max (32-core GPU, 36 GB) | 36 GB (~24.1 usable) | ~68 tok/s | Unknown (no KV data) | low $2,499 |
| NVIDIA GeForce RTX 4090 | 24 GB | ~135 tok/s | Unknown (no KV data) | low $2,910 |
| NVIDIA DGX Spark (64GB) | 64 GB | ~61 tok/s | Unknown (no KV data) | low $4,999 |
| Apple Mac Studio M5 Ultra (64-core GPU, 96 GB) | 96 GB (~72 usable) | ~98 tok/s | Unknown (no KV data) | low $5,499 |
| NVIDIA GeForce RTX 5090 | 32 GB | ~168 tok/s | Unknown (no KV data) | low $6,500 |
| Framework Desktop, AMD Ryzen AI Max+ PRO 495 (Strix Halo), 192 GB | 192 GB | ~53 tok/s | Unknown (no KV data) | low $6,799 |
| NVIDIA DGX Spark (128GB) | 128 GB | ~61 tok/s | Unknown (no KV data) | low $6,949.99 |
| NVIDIA RTX A6000 | 48 GB | ~118 tok/s | Unknown (no KV data) | low $8,363 |
| Intel Arc Pro B60 | 24 GB | ~64 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA A10 | 24 GB | ~103 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA GeForce RTX 3090 Ti | 24 GB | ~135 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA L4 | 24 GB | ~66 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA Quadro RTX 6000 | 24 GB | ~110 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA RTX 4500 Ada Generation | 24 GB | ~85 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA RTX A5000 | 24 GB | ~118 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA RTX A5500 | 24 GB | ~118 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA RTX PRO 4000 Blackwell | 24 GB | ~110 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA RTX PRO 4000 SFF Blackwell | 24 GB | ~85 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA Tesla P40 | 24 GB | ~58 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA TITAN RTX | 24 GB | ~110 tok/s | Unknown (no KV data) | No price yet |
| AMD Radeon PRO V710 | 28 GB | ~75 tok/s | Unknown (no KV data) | No price yet |
| AMD Radeon AI PRO R9600 | 32 GB | ~93 tok/s | Unknown (no KV data) | No price yet |
| AMD Radeon AI PRO R9600D | 32 GB | ~93 tok/s | Unknown (no KV data) | No price yet |
| AMD Radeon AI PRO R9700 | 32 GB | ~93 tok/s | Unknown (no KV data) | No price yet |
| AMD Radeon AI PRO R9700S | 32 GB | ~93 tok/s | Unknown (no KV data) | No price yet |
| AMD Radeon PRO V620 | 32 GB | ~82 tok/s | Unknown (no KV data) | No price yet |
| AMD Radeon PRO W6800 | 32 GB | ~82 tok/s | Unknown (no KV data) | No price yet |
| AMD Radeon PRO W7800 32GB | 32 GB | ~88 tok/s | Unknown (no KV data) | No price yet |
| Intel Arc Pro B70 | 32 GB | ~77 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA Quadro GV100 | 32 GB | ~126 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA RTX 5000 Ada Generation | 32 GB | ~101 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA RTX PRO 4500 Blackwell | 32 GB | ~128 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA V100 PCIe 32GB | 32 GB | ~128 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA V100S PCIe 32GB | 32 GB | ~142 tok/s | Unknown (no KV data) | No price yet |
| Apple M5 Pro 48GB | 48 GB (~36 usable) | ~55 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA A100 40GB PCIe | 40 GB | ~160 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA A800 40GB Active | 40 GB | ~160 tok/s | Unknown (no KV data) | No price yet |
| AMD Radeon PRO W7800 48GB | 48 GB | ~108 tok/s | Unknown (no KV data) | No price yet |
| AMD Radeon PRO W7900 | 48 GB | ~108 tok/s | Unknown (no KV data) | No price yet |
| AMD Radeon PRO W7900 Dual Slot | 48 GB | ~108 tok/s | Unknown (no KV data) | No price yet |
| Apple Mac Studio M5 Max (40-core GPU, 64 GB) | 64 GB (~48 usable) | ~78 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA A40 | 48 GB | ~112 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA L40 | 48 GB | ~126 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA L40S | 48 GB | ~126 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA Quadro RTX 8000 | 48 GB | ~110 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA RTX 6000 Ada Generation | 48 GB | ~132 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA RTX PRO 5000 Blackwell 48GB | 48 GB | ~152 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA RTX PRO 5000 Blackwell 72GB | 72 GB | ~152 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA A100 80GB PCIe | 80 GB | ~172 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA RTX PRO 6000 Blackwell Max-Q | 96 GB | ~168 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA RTX PRO 6000 Blackwell Server Edition | 96 GB | ~162 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA RTX PRO 6000 Blackwell Workstation Edition | 96 GB | ~168 tok/s | Unknown (no KV data) | No price yet |
Every device that runs it
Graphics cards, Macs and mini PCs from the wider hardware list behind My rig, smallest usable memory first. Not sure about yours? Check your exact rig.
Show all 104 devices that run it
| Device | Type | Memory | Speed estimate | Max context estimate | Price |
|---|---|---|---|---|---|
| Apple M1 Max 32GB | Mac | 32 GB (~21.4 usable) | ~63 tok/s | Unknown Tight | No price yet |
| Apple M1 Pro 32GB | Mac | 32 GB (~21.4 usable) | ~42 tok/s | Unknown Tight | No price yet |
| Apple M2 Max (30-core GPU) 32GB | Mac | 32 GB (~21.4 usable) | ~63 tok/s | Unknown Tight | No price yet |
| Apple M2 Max (38-core GPU) 32GB | Mac | 32 GB (~21.4 usable) | ~63 tok/s | Unknown Tight | No price yet |
| Apple M2 Pro 32GB | Mac | 32 GB (~21.4 usable) | ~42 tok/s | Unknown Tight | No price yet |
| Apple M4 32GB | Mac | 32 GB (~21.4 usable) | ~29 tok/s | Unknown Tight | No price yet |
| Apple M5 32GB | Mac | 32 GB (~21.4 usable) | ~34 tok/s | Unknown Tight | No price yet |
| Apple M6 32GB | Mac | 32 GB (~21.4 usable) | ~37 tok/s | Unknown Tight | No price yet |
| AMD Radeon RX 7900 XTX | GPU | 24 GB | ~114 tok/s | Unknown | low $1,350 |
| Intel Arc Pro B60 | Workstation GPU | 24 GB | ~64 tok/s | Unknown | No price yet |
| NVIDIA A10 | Data center GPU | 24 GB | ~103 tok/s | Unknown | No price yet |
| NVIDIA GeForce RTX 3090 | GPU | 24 GB | ~130 tok/s | Unknown | low $1,499 |
| NVIDIA GeForce RTX 3090 Ti | GPU | 24 GB | ~135 tok/s | Unknown | No price yet |
| NVIDIA GeForce RTX 4090 | GPU | 24 GB | ~135 tok/s | Unknown | low $2,910 |
| NVIDIA L4 | Data center GPU | 24 GB | ~66 tok/s | Unknown | No price yet |
| NVIDIA Quadro RTX 6000 | Workstation GPU | 24 GB | ~110 tok/s | Unknown | No price yet |
| NVIDIA RTX 4500 Ada Generation | Workstation GPU | 24 GB | ~85 tok/s | Unknown | No price yet |
| NVIDIA RTX A5000 | Workstation GPU | 24 GB | ~118 tok/s | Unknown | No price yet |
| NVIDIA RTX A5500 | Workstation GPU | 24 GB | ~118 tok/s | Unknown | No price yet |
| NVIDIA RTX PRO 4000 Blackwell | Workstation GPU | 24 GB | ~110 tok/s | Unknown | No price yet |
| NVIDIA RTX PRO 4000 SFF Blackwell | Workstation GPU | 24 GB | ~85 tok/s | Unknown | No price yet |
| NVIDIA RTX Spark N1X laptop (6144-core GPU, 24GB) | Laptop | 24 GB | ~66 tok/s | Unknown | No price yet |
| NVIDIA Tesla P40 | Data center GPU | 24 GB | ~58 tok/s | Unknown | No price yet |
| NVIDIA TITAN RTX | GPU | 24 GB | ~110 tok/s | Unknown | No price yet |
| Apple M3 Max (30-core GPU) 36GB | Mac | 36 GB (~24.1 usable) | ~54 tok/s | Unknown | No price yet |
| Apple M3 Pro 36GB | Mac | 36 GB (~24.1 usable) | ~34 tok/s | Unknown | No price yet |
| Apple M4 Max (32-core GPU) 36GB | Mac | 36 GB (~24.1 usable) | ~64 tok/s | Unknown | No price yet |
| Apple Mac Studio M5 Max (32-core GPU, 36 GB) | Mac | 36 GB (~24.1 usable) | ~68 tok/s | Unknown | low $2,499 |
| AMD Radeon PRO V710 | Data center GPU | 28 GB | ~75 tok/s | Unknown | No price yet |
| AMD Radeon AI PRO R9600 | Workstation GPU | 32 GB | ~93 tok/s | Unknown | No price yet |
| AMD Radeon AI PRO R9600D | Workstation GPU | 32 GB | ~93 tok/s | Unknown | No price yet |
| AMD Radeon AI PRO R9700 | Workstation GPU | 32 GB | ~93 tok/s | Unknown | No price yet |
| AMD Radeon AI PRO R9700S | Workstation GPU | 32 GB | ~93 tok/s | Unknown | No price yet |
| AMD Radeon PRO V620 | Data center GPU | 32 GB | ~82 tok/s | Unknown | No price yet |
| AMD Radeon PRO W6800 | Workstation GPU | 32 GB | ~82 tok/s | Unknown | No price yet |
| AMD Radeon PRO W7800 32GB | Workstation GPU | 32 GB | ~88 tok/s | Unknown | No price yet |
| AMD Ryzen AI Max 385 (32GB) | Mini PC | 32 GB | ~51 tok/s | Unknown | No price yet |
| AMD Ryzen AI Max 390 (32GB) | Mini PC | 32 GB | ~51 tok/s | Unknown | No price yet |
| AMD Ryzen AI Max+ 395 (32GB) | Mini PC | 32 GB | ~51 tok/s | Unknown | No price yet |
| Intel Arc Pro B70 | Workstation GPU | 32 GB | ~77 tok/s | Unknown | No price yet |
| NVIDIA GeForce RTX 5090 | GPU | 32 GB | ~168 tok/s | Unknown | low $6,500 |
| NVIDIA Quadro GV100 | Workstation GPU | 32 GB | ~126 tok/s | Unknown | No price yet |
| NVIDIA RTX 5000 Ada Generation | Workstation GPU | 32 GB | ~101 tok/s | Unknown | No price yet |
| NVIDIA RTX PRO 4500 Blackwell | Workstation GPU | 32 GB | ~128 tok/s | Unknown | No price yet |
| NVIDIA RTX Spark N1X laptop (6144-core GPU, 32GB) | Laptop | 32 GB | ~66 tok/s | Unknown | No price yet |
| NVIDIA V100 PCIe 32GB | Data center GPU | 32 GB | ~128 tok/s | Unknown | No price yet |
| NVIDIA V100S PCIe 32GB | Data center GPU | 32 GB | ~142 tok/s | Unknown | No price yet |
| Apple M3 Max (40-core GPU) 48GB | Mac | 48 GB (~36 usable) | ~63 tok/s | Unknown | No price yet |
| Apple M4 Max (40-core GPU) 48GB | Mac | 48 GB (~36 usable) | ~74 tok/s | Unknown | No price yet |
| Apple M4 Pro 48GB | Mac | 48 GB (~36 usable) | ~51 tok/s | Unknown | No price yet |
| Apple M5 Max (40-core GPU) 48GB | Mac | 48 GB (~36 usable) | ~78 tok/s | Unknown | No price yet |
| Apple M5 Pro 48GB | Mac | 48 GB (~36 usable) | ~55 tok/s | Unknown | No price yet |
| NVIDIA A100 40GB PCIe | Data center GPU | 40 GB | ~160 tok/s | Unknown | No price yet |
| NVIDIA A800 40GB Active | Workstation GPU | 40 GB | ~160 tok/s | Unknown | No price yet |
| AMD Radeon PRO W7800 48GB | Workstation GPU | 48 GB | ~108 tok/s | Unknown | No price yet |
| AMD Radeon PRO W7900 | Workstation GPU | 48 GB | ~108 tok/s | Unknown | No price yet |
| AMD Radeon PRO W7900 Dual Slot | Workstation GPU | 48 GB | ~108 tok/s | Unknown | No price yet |
| Apple M1 Max 64GB | Mac | 64 GB (~48 usable) | ~63 tok/s | Unknown | No price yet |
| Apple M1 Ultra 64GB | Mac | 64 GB (~48 usable) | ~86 tok/s | Unknown | No price yet |
| Apple M2 Max (30-core GPU) 64GB | Mac | 64 GB (~48 usable) | ~63 tok/s | Unknown | No price yet |
| Apple M2 Max (38-core GPU) 64GB | Mac | 64 GB (~48 usable) | ~63 tok/s | Unknown | No price yet |
| Apple M2 Ultra 64GB | Mac | 64 GB (~48 usable) | ~86 tok/s | Unknown | No price yet |
| Apple M3 Max (40-core GPU) 64GB | Mac | 64 GB (~48 usable) | ~63 tok/s | Unknown | No price yet |
| Apple M4 Max (40-core GPU) 64GB | Mac | 64 GB (~48 usable) | ~74 tok/s | Unknown | No price yet |
| Apple M4 Pro 64GB | Mac | 64 GB (~48 usable) | ~51 tok/s | Unknown | No price yet |
| Apple M5 Pro 64GB | Mac | 64 GB (~48 usable) | ~55 tok/s | Unknown | No price yet |
| Apple Mac Studio M5 Max (40-core GPU, 64 GB) | Mac | 64 GB (~48 usable) | ~78 tok/s | Unknown | No price yet |
| NVIDIA A40 | Data center GPU | 48 GB | ~112 tok/s | Unknown | No price yet |
| NVIDIA L40 | Data center GPU | 48 GB | ~126 tok/s | Unknown | No price yet |
| NVIDIA L40S | Data center GPU | 48 GB | ~126 tok/s | Unknown | No price yet |
| NVIDIA Quadro RTX 8000 | Workstation GPU | 48 GB | ~110 tok/s | Unknown | No price yet |
| NVIDIA RTX 6000 Ada Generation | Workstation GPU | 48 GB | ~132 tok/s | Unknown | No price yet |
| NVIDIA RTX A6000 | Workstation GPU | 48 GB | ~118 tok/s | Unknown | low $8,363 |
| NVIDIA RTX PRO 5000 Blackwell 48GB | Workstation GPU | 48 GB | ~152 tok/s | Unknown | No price yet |
| NVIDIA RTX Spark N1X laptop (6144-core GPU, 48GB) | Laptop | 48 GB | ~66 tok/s | Unknown | No price yet |
| AMD Ryzen AI Max 385 (64GB) | Mini PC | 64 GB | ~51 tok/s | Unknown | No price yet |
| AMD Ryzen AI Max 390 (64GB) | Mini PC | 64 GB | ~51 tok/s | Unknown | No price yet |
| AMD Ryzen AI Max+ 395 (64GB) | Mini PC | 64 GB | ~51 tok/s | Unknown | low $2,199.99 |
| NVIDIA DGX Spark (64GB) | Mini PC | 64 GB | ~61 tok/s | Unknown | low $4,999 |
| NVIDIA RTX Spark N1X laptop (6144-core GPU, 64GB) | Laptop | 64 GB | ~66 tok/s | Unknown | No price yet |
| Apple M2 Max (38-core GPU) 96GB | Mac | 96 GB (~72 usable) | ~63 tok/s | Unknown | No price yet |
| Apple M3 Max (30-core GPU) 96GB | Mac | 96 GB (~72 usable) | ~54 tok/s | Unknown | No price yet |
| Apple M3 Ultra 96GB | Mac | 96 GB (~72 usable) | ~87 tok/s | Unknown | No price yet |
| Apple Mac Studio M5 Ultra (64-core GPU, 96 GB) | Mac | 96 GB (~72 usable) | ~98 tok/s | Unknown | low $5,499 |
| NVIDIA RTX PRO 5000 Blackwell 72GB | Workstation GPU | 72 GB | ~152 tok/s | Unknown | No price yet |
| NVIDIA A100 80GB PCIe | Data center GPU | 80 GB | ~172 tok/s | Unknown | No price yet |
| Apple M1 Ultra 128GB | Mac | 128 GB (~96 usable) | ~86 tok/s | Unknown | No price yet |
| Apple M2 Ultra 128GB | Mac | 128 GB (~96 usable) | ~86 tok/s | Unknown | No price yet |
| Apple M3 Max (40-core GPU) 128GB | Mac | 128 GB (~96 usable) | ~63 tok/s | Unknown | No price yet |
| Apple M4 Max (40-core GPU) 128GB | Mac | 128 GB (~96 usable) | ~74 tok/s | Unknown | No price yet |
| Apple M5 Max (40-core GPU) 128GB | Mac | 128 GB (~96 usable) | ~78 tok/s | Unknown | No price yet |
| NVIDIA RTX PRO 6000 Blackwell Max-Q | Workstation GPU | 96 GB | ~168 tok/s | Unknown | No price yet |
| NVIDIA RTX PRO 6000 Blackwell Server Edition | Data center GPU | 96 GB | ~162 tok/s | Unknown | No price yet |
| NVIDIA RTX PRO 6000 Blackwell Workstation Edition | Workstation GPU | 96 GB | ~168 tok/s | Unknown | No price yet |
| AMD Ryzen AI Max 385 (128GB) | Mini PC | 128 GB | ~51 tok/s | Unknown | No price yet |
| AMD Ryzen AI Max 390 (128GB) | Mini PC | 128 GB | ~51 tok/s | Unknown | No price yet |
| AMD Ryzen AI Max+ 395 (128GB) | Mini PC | 128 GB | ~51 tok/s | Unknown | low $3,499.99 |
| NVIDIA DGX Spark (128GB) | Mini PC | 128 GB | ~61 tok/s | Unknown | low $6,949.99 |
| NVIDIA RTX Spark N1X laptop (6144-core GPU, 128GB) | Laptop | 128 GB | ~66 tok/s | Unknown | No price yet |
| Apple M2 Ultra 192GB | Mac | 192 GB (~144 usable) | ~86 tok/s | Unknown | No price yet |
| Apple M3 Ultra 256GB | Mac | 256 GB (~192 usable) | ~87 tok/s | Unknown | No price yet |
| Apple M5 Ultra 256GB | Mac | 256 GB (~192 usable) | ~98 tok/s | Unknown | No price yet |
| Framework Desktop, AMD Ryzen AI Max+ PRO 495 (Strix Halo), 192 GB | Mini PC | 192 GB | ~53 tok/s | Unknown | low $6,799 |
| Apple M5 Ultra 512GB | Mac | 512 GB (~384 usable) | ~98 tok/s | Unknown | No price yet |
Multi-GPU: cards that need two or more
| Cards | Total VRAM | Speed estimate | Max context estimate |
|---|---|---|---|
| 2 × Intel Arc B580 | 24 GB | ~64 tok/s | Unknown |
| 2 × NVIDIA GeForce RTX 3060 12 GB | 24 GB | ~75 tok/s | Unknown |
| 2 × NVIDIA GeForce RTX 4070 SUPER | 24 GB | ~93 tok/s | Unknown |
| 3 × NVIDIA GeForce RTX 5060 | 24 GB | ~87 tok/s | Unknown |
| 3 × NVIDIA GeForce RTX 5060 Ti 8 GB | 24 GB | ~87 tok/s | Unknown |
| 2 × NVIDIA GeForce RTX 5070 | 24 GB | ~110 tok/s | Unknown |
| 2 × AMD Radeon RX 7800 XT | 32 GB | ~92 tok/s | Unknown |
| 2 × AMD Radeon RX 9060 XT 16 GB | 32 GB | ~60 tok/s | Unknown |
| 2 × AMD Radeon RX 9070 | 32 GB | ~93 tok/s | Unknown |
| 2 × AMD Radeon RX 9070 XT | 32 GB | ~93 tok/s | Unknown |
| 2 × NVIDIA GeForce RTX 5060 Ti 16 GB | 32 GB | ~87 tok/s | Unknown |
| 2 × NVIDIA GeForce RTX 5070 Ti | 32 GB | ~128 tok/s | Unknown |
| 2 × NVIDIA GeForce RTX 5080 | 32 GB | ~132 tok/s | Unknown |
Speed calculator
estimateEstimate = bandwidth ÷ GB read per token (the model's listed GGUF file: Q4_K_M for most models, otherwise the quant named in the list; scaled by active/total parameters for MoE), scaled by a per-platform efficiency and fixed per-token costs calibrated to 45 measured llama.cpp benchmarks (2025–26). Fit = file size + 1.5 GB headroom ≤ total VRAM (on Macs, the part of unified memory macOS lets the GPU use by default), and room for at least 4K tokens of context with an 8-bit KV cache. For a model with no KV cache data, only the file and headroom are checked, so a fit says nothing about context. Multi-GPU: VRAM adds, speed stays one card's (a layer split runs the cards in turn); max context uses the total. Speed is for short context. Max context = what is left after the file and a 1 GB runtime reserve ÷ KV cache bytes per token, default FP16 and 8-bit (q8_0) cache. How speed works · how context works
Related
- BuildBuild that fits it: 24 GB: one used RTX 3090
- BuildBuild that fits it: 32 GB: two Radeon RX 9070
- BuildBuild that fits it: 48 GB: dual used RTX 3090
- ModelRaon-1 nano, the small dense model
- ModelQwen3.6-35B-A3B, one of the models KRAFTON compares against
- ModelGemma 4 26B A4B, another model KRAFTON compares against