What GPU do you need to run Raon-1 nano?
Weights-only fit: a used GeForce RTX 3060 12 GB ($305). It is the cheapest priced graphics card that holds Raon-1 nano (about 3.6 GB at Q4) (weights-only fit, context unknown: no KV cache data; 8.4 GB left beyond the file and headroom), at about 98 tok/s estimate; that is the card price only, and it requires a separate compatible PC. The weights fit 157 of the 157 GPUs and systems we track; with no KV cache data, context is unknown on all of them.
Early support Runs on KRAFTON's llama.cpp fork and MLX build. Stock llama.cpp, Ollama and LM Studio can't load Raon-1 nano yet. Upstream llama.cpp has no Raon-1 architecture (these GGUFs use raon1nano), and we found no upstream pull request for nano. KRAFTON's GGUF card builds from its fork: git clone -b raon-1-nano-preview https://github.com/krafton-ai/llama.cpp (the card also mentions a raon-1-nano-optimize branch). On a Mac, KRAFTON's MLX build (KRAFTON/Raon-1-nano-mlx) needs trust_remote_code=True and pins mlx==0.32.4, mlx-lm==0.32.2 and transformers==5.20.2. The fits and speeds on this page are math on the official Q4_K_M file, not a test on the fork. (source, checked Nov 11, 2026.)
Parameters3.1B
VRAM needed~3.6 GBfile + headroom
Single GPUs the weights fit157context unknown
Context window256K tokens
3.1B parameters (Hugging Face, accessed 2026-11-11). Q4_K_M GGUF: 2.08 GB (source). No context estimate: Raon-1 nano's architecture is not in upstream llama.cpp, so there is no verified llama.cpp KV formula for it yet. KRAFTON says only its 6 gated MLA layers keep a KV cache, about 7 KiB per token in BF16, and the 18 Raon Delta Attention layers keep a fixed-size state. That is KRAFTON's figure, not checked here (details).
KRAFTON AI's small model for chat, instruction following and tool use. KRAFTON derived it from Raon-1 mini preview (30B-A4B) by MoE-to-Dense pruning and distillation, then trained it further on 3T tokens, followed by SFT, OPD and RL. It is dense, and a hybrid: of its 24 layers, 18 are Raon Delta Attention (linear attention with a fixed-size state) and 6 are gated MLA (multi-head latent attention). Thinking is on by default, and KRAFTON says turning it off is not recommended. KRAFTON gives 3.1B parameters, including a 1.5B per-layer embedding table, and calls it an effective 1.5B. Those per-layer embeddings still sit in memory, so the fit counts the whole file. KRAFTON's other cards say 3.2B (details below). Q4_K_M is KRAFTON's official GGUF (raon-1-nano-Q4_K_M.gguf, 2,082,677,632 bytes), made with KRAFTON's own llama.cpp fork; every size is in the table below. The _MTP files add a small MTP head for speculative decoding (drafting tokens ahead); the fit and the speed estimates use the plain file and plain decoding. Apache 2.0, including commercial use of the downloaded weights; KRAFTON's hosted API and playground have separate terms. Release date is KRAFTON's announcement; the Hugging Face repo was created on Oct 29, 2026.
What we checked
Checked Nov 11, 2026. Official is the maker or the weight files. A vendor claim is that project's own documentation. A community claim is someone else's upload. Our check is a file or search we read. Unknown means we have no source, so it is not estimated. None of these rows is a speed we measured.
| Kind | What | Detail | Source |
|---|---|---|---|
| Official | Model card | KRAFTON's card: a dense hybrid with 24 layers (18 Raon Delta Attention, 6 gated MLA), hidden size 1,536, SwiGLU, Raon-mHC with 4 residual streams, no position encoding (NoPE), a 259,840-token vocabulary, a tied output head and a per-layer embedding size of 256. Native context is 262,144 tokens. Thinking is on by default. The checkpoint also has a 1-layer, 55M-parameter MTP module. | Model card, checked Nov 11, 2026 |
| Official | Parameter count | KRAFTON's announcement and main card say 3.1B in total, including a 1.5B per-layer embedding table (an effective 1.5B). The main card also says the Hugging Face checkpoint is 3.2B with the 55M MTP module. The GGUF and MLX cards say 3.2B, including a 1.6B table. Hugging Face's own counter reads 3,529,911,376, which may count the tied output head twice (not checked). This page uses KRAFTON's 3.1B. The model is dense, so no active count is used. Its per-layer embedding table is a lookup, so each token reads only a small slice of it, but the table still sits in memory. The speed estimate on this page reads the whole Q4_K_M file per token, so it may be on the slow side. | Model card, checked Nov 11, 2026 |
| Official | License | Apache 2.0, including commercial use of the downloaded weights, per the cards. KRAFTON's hosted API and playground have separate terms. The repos are not gated. This is a summary of the license, not legal advice. | Model card, checked Nov 11, 2026 |
| Official | Weight files | Three BF16 safetensors files sum to 7,060,010,384 bytes (7.06 GB): two model shards (3,068,867,712 and 3,192,914,064 bytes) and model-tied-lm-head.safetensors (798,228,608 bytes). The MTP module is a separate 109,687,880-byte file. These are downloads, not the memory a runtime needs. | Hugging Face API, checked Nov 11, 2026 |
| Official | GGUF files | KRAFTON's own GGUF repo, made with the raon-1-nano-preview branch of its llama.cpp fork (GGUF architecture raon1nano). It has Q4_K_M, Q6_K, Q8_0 and F16, each also as an _MTP file with the MTP head. The fit on this page uses the plain Q4_K_M. We found no third-party GGUFs. Every size is in the table below. | GGUF repo, checked Nov 11, 2026 |
| Official | MLX build | KRAFTON/Raon-1-nano-mlx is 4-bit (GPTQ, group size 64): model.safetensors is 1,805,577,858 bytes, plus a 62,411,311-byte MTP file. It ships its own model code (raon_mlx_model.py), so it needs trust_remote_code=True, and it pins mlx==0.32.4, mlx-lm==0.32.2 and transformers==5.20.2. KRAFTON says its fused Metal kernels "are enabled only with mlx==0.32.4 (other versions run a slower path)". KRAFTON puts a BF16 MLX export at about 6.3 GB and says its decode is slower than 4-bit. | MLX repo, checked Nov 11, 2026 |
| Our check | Upstream llama.cpp | ggml-org/llama.cpp master has no Raon-1 model file (src/models/raon1.cpp is a 404), and we found no upstream pull request for Raon-1 nano. The only related one, #31107 for Raon-1 mini preview, was closed unmerged on Oct 24, 2026. | llama.cpp PR #31107, checked Nov 11, 2026 |
| Our check | Ollama and LM Studio | An Ollama search on 2026-11-11 found no KRAFTON Raon model, only unrelated names. We found no LM Studio listing. | Ollama search, checked Nov 11, 2026 |
| Vendor claim | Other runtimes | KRAFTON also publishes a LiteRT-LM build (KRAFTON/Raon-1-nano-litert-lm). Its card says it runs with a 4,096-token context and answers without thinking, and that the first GPU compile takes about 30 to 60 seconds. Transformers needs trust_remote_code; KRAFTON notes that Raon Delta Attention prefill runs one step at a time, so long prompts can be slow to process. KRAFTON's vLLM instructions install its own fork (github.com/krafton-ai/vllm, branch raon-1-nano). We did not check upstream vLLM. | Model card, checked Nov 11, 2026 |
| Vendor claim | Benchmark scores (KRAFTON's) | KRAFTON's card compares nano with Qwen3.5 2B, Gemma 4 E2B, MiniCPM5 2B and LFM3 1.7B. Its scores for nano: MMLU-Redux 81.1, IFBench 75.9, MultiChallenge 40.5, FACTS Grounding 44.4, BFCL v4 56.3, BrowseComp Plus 34.5, NL2SH-ALFA 58.0, AndroidControl 36.0 (average of HL and LL), PersonaMem v2 38.8 (32K setting) and PrefEval 44.0. In Korean: KMMLU-Redux 50.9, HAE-RAE 1.1 44.2 and CLIcK 65.2. Office Work Bench (49.2) and Chatbot Korean (91.0) are KRAFTON's internal tests and are not public. Sampling settings differ by benchmark and are on the card. These are KRAFTON's scores, not checked by us. KRAFTON publishes no speed figures for nano, so the speed estimates on this page are our own formula for plain decoding with the Q4_K_M file, without MTP drafting. | Model card, checked Nov 11, 2026 |
GGUF files and sizes
Exact sizes from Hugging Face, accessed 2026-11-11. The fit and context tables on this page use the Q4_K_M file (2.08 GB); a smaller file needs less memory by the difference. VRAM needed is the file + the same headroom.
| File | Published by | Bytes | Size | VRAM needed | Source |
|---|---|---|---|---|---|
| Q4_K_M | KRAFTON | 2,082,677,632 | 2.08 GB | ~3.6 GB | raon-1-nano-Q4_K_M.gguf |
| Q4_K_M + MTP head | KRAFTON | 2,145,934,208 | 2.15 GB | ~3.6 GB | raon-1-nano-Q4_K_M_MTP.gguf |
| Q6_K | KRAFTON | 2,646,016,384 | 2.65 GB | ~4.2 GB | raon-1-nano-Q6_K.gguf |
| Q6_K + MTP head | KRAFTON | 2,709,272,960 | 2.71 GB | ~4.2 GB | raon-1-nano-Q6_K_MTP.gguf |
| Q8_0 | KRAFTON | 3,391,016,320 | 3.39 GB | ~4.9 GB | raon-1-nano-Q8_0.gguf |
| Q8_0 + MTP head | KRAFTON | 3,454,272,896 | 3.45 GB | ~5 GB | raon-1-nano-Q8_0_MTP.gguf |
| F16 | KRAFTON | 6,274,887,040 | 6.27 GB | ~7.8 GB | raon-1-nano-F16.gguf |
| F16 + MTP head | KRAFTON | 6,384,592,256 | 6.38 GB | ~7.9 GB | raon-1-nano-F16_MTP.gguf |
Other places to get it
Checked Nov 14, 2026. We list only copies we opened and checked. Check file hashes for copies from other uploaders.
- ModelScope: Other uploaderSame account name as on Hugging Face; we found no link from the lab to it
Weights-only fits, context unknown. We have no KV cache data for Raon-1 nano, so every fit on this page only checks that the Q4_K_M file and headroom fit. How much context fits beside the weights is unknown, so none of these is a proven fit for any context length.
Single GPUs the weights fit, cheapest first
| Single GPU that fits | VRAM | Speed estimate | Max context estimate | Price |
|---|---|---|---|---|
| NVIDIA GeForce RTX 3060 12 GB | 12 GB | ~98 tok/s | Unknown (no KV data) | low $305 |
| Intel Arc B580 | 12 GB | ~99 tok/s | Unknown (no KV data) | low $330 |
| NVIDIA GeForce RTX 5060 | 8 GB | ~113 tok/s | Unknown (no KV data) | low $410 |
| NVIDIA GeForce RTX 5060 Ti 8 GB | 8 GB | ~113 tok/s | Unknown (no KV data) | low $450 |
| AMD Radeon RX 9070 | 16 GB | ~151 tok/s | Unknown (no KV data) | low $570 |
| AMD Radeon RX 7800 XT | 16 GB | ~149 tok/s | Unknown (no KV data) | low $599 |
| NVIDIA GeForce RTX 4070 SUPER | 12 GB | ~121 tok/s | Unknown (no KV data) | low $620 |
| AMD Radeon RX 9060 XT 16 GB | 16 GB | ~92 tok/s | Unknown (no KV data) | low $620 |
| NVIDIA GeForce RTX 5060 Ti 16 GB | 16 GB | ~113 tok/s | Unknown (no KV data) | low $780 |
| NVIDIA GeForce RTX 5070 | 12 GB | ~141 tok/s | Unknown (no KV data) | low $800 |
| AMD Radeon RX 9070 XT | 16 GB | ~151 tok/s | Unknown (no KV data) | low $810 |
| NVIDIA GeForce RTX 5070 Ti | 16 GB | ~160 tok/s | Unknown (no KV data) | low $1,120 |
| AMD Radeon RX 7900 XTX | 24 GB | ~192 tok/s | Unknown (no KV data) | low $1,350 |
| NVIDIA GeForce RTX 3090 | 24 GB | ~163 tok/s | Unknown (no KV data) | low $1,499 |
| NVIDIA GeForce RTX 5080 | 16 GB | ~165 tok/s | Unknown (no KV data) | low $1,600 |
| Apple Mac Studio M5 Max (32-core GPU, 36 GB) | 36 GB (~24.1 usable) | ~107 tok/s | Unknown (no KV data) | low $2,499 |
| NVIDIA GeForce RTX 4090 | 24 GB | ~168 tok/s | Unknown (no KV data) | low $2,910 |
| NVIDIA DGX Spark (64GB) | 64 GB | ~82 tok/s | Unknown (no KV data) | low $4,999 |
| Apple Mac Studio M5 Ultra (64-core GPU, 96 GB) | 96 GB (~72 usable) | ~160 tok/s | Unknown (no KV data) | low $5,499 |
| NVIDIA GeForce RTX 5090 | 32 GB | ~203 tok/s | Unknown (no KV data) | low $6,500 |
| Framework Desktop, AMD Ryzen AI Max+ PRO 495 (Strix Halo), 192 GB | 192 GB | ~81 tok/s | Unknown (no KV data) | low $6,799 |
| NVIDIA DGX Spark (128GB) | 128 GB | ~82 tok/s | Unknown (no KV data) | low $6,949.99 |
| NVIDIA RTX A6000 | 48 GB | ~150 tok/s | Unknown (no KV data) | low $8,363 |
| AMD Radeon PRO W6400 | 4 GB | ~43 tok/s | Unknown (no KV data) | No price yet |
| AMD Radeon RX 6400 | 4 GB | ~43 tok/s | Unknown (no KV data) | No price yet |
| AMD Radeon RX 6500 XT 4GB | 4 GB | ~47 tok/s | Unknown (no KV data) | No price yet |
| AMD Radeon RX 9050 4GB | 4 GB | ~47 tok/s | Unknown (no KV data) | No price yet |
| Intel Arc A310 | 4 GB | ~20 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA GeForce GTX 1050 Ti | 4 GB | ~31 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA GeForce GTX 1650 (GDDR5) | 4 GB | ~45 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA GeForce GTX 1650 (GDDR6) | 4 GB | ~63 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA RTX A400 | 4 GB | ~35 tok/s | Unknown (no KV data) | No price yet |
| Intel Arc A380 | 6 GB | ~30 tok/s | Unknown (no KV data) | No price yet |
| Intel Arc Pro A40 | 6 GB | ~31 tok/s | Unknown (no KV data) | No price yet |
| Intel Arc Pro A50 | 6 GB | ~31 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA GeForce GTX 1060 6GB | 6 GB | ~49 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA GeForce GTX 1660 | 6 GB | ~63 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA GeForce GTX 1660 SUPER | 6 GB | ~94 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA GeForce GTX 1660 Ti | 6 GB | ~85 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA GeForce RTX 2060 6GB | 6 GB | ~94 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA RTX A2000 6GB | 6 GB | ~85 tok/s | Unknown (no KV data) | No price yet |
| AMD Radeon PRO W6600 | 8 GB | ~69 tok/s | Unknown (no KV data) | No price yet |
| AMD Radeon PRO W7400 | 8 GB | ~55 tok/s | Unknown (no KV data) | No price yet |
| AMD Radeon PRO W7500 | 8 GB | ~77 tok/s | Unknown (no KV data) | No price yet |
| AMD Radeon PRO W7600 | 8 GB | ~85 tok/s | Unknown (no KV data) | No price yet |
| AMD Radeon RX 6500 XT 8GB | 8 GB | ~47 tok/s | Unknown (no KV data) | No price yet |
| AMD Radeon RX 6600 | 8 GB | ~69 tok/s | Unknown (no KV data) | No price yet |
| AMD Radeon RX 6600 XT | 8 GB | ~77 tok/s | Unknown (no KV data) | No price yet |
| AMD Radeon RX 6650 XT | 8 GB | ~83 tok/s | Unknown (no KV data) | No price yet |
| AMD Radeon RX 7400 | 8 GB | ~56 tok/s | Unknown (no KV data) | No price yet |
| AMD Radeon RX 7600 | 8 GB | ~85 tok/s | Unknown (no KV data) | No price yet |
| AMD Radeon RX 9050 | 8 GB | ~85 tok/s | Unknown (no KV data) | No price yet |
| AMD Radeon RX 9060 | 8 GB | ~85 tok/s | Unknown (no KV data) | No price yet |
| AMD Radeon RX 9060 XT 8GB | 8 GB | ~92 tok/s | Unknown (no KV data) | No price yet |
| Intel Arc A580 | 8 GB | ~73 tok/s | Unknown (no KV data) | No price yet |
| Intel Arc A750 | 8 GB | ~73 tok/s | Unknown (no KV data) | No price yet |
| Intel Arc A770 8GB | 8 GB | ~73 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA GeForce GTX 1070 | 8 GB | ~62 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA GeForce GTX 1070 Ti | 8 GB | ~62 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA GeForce GTX 1080 | 8 GB | ~74 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA GeForce RTX 2060 SUPER | 8 GB | ~113 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA GeForce RTX 2070 | 8 GB | ~113 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA GeForce RTX 2070 SUPER | 8 GB | ~113 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA GeForce RTX 2080 | 8 GB | ~113 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA GeForce RTX 2080 SUPER | 8 GB | ~120 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA GeForce RTX 3070 | 8 GB | ~113 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA GeForce RTX 3070 Ti | 8 GB | ~134 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA GeForce RTX 4060 | 8 GB | ~81 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA GeForce RTX 4060 Ti 8GB | 8 GB | ~85 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA GeForce RTX 5050 | 8 GB | ~91 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA RTX A1000 | 8 GB | ~63 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA T1000 8GB | 8 GB | ~54 tok/s | Unknown (no KV data) | No price yet |
| AMD Radeon RX 6700 | 10 GB | ~92 tok/s | Unknown (no KV data) | No price yet |
| Intel Arc B570 | 10 GB | ~86 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA GeForce RTX 3080 | 10 GB | ~149 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA GeForce GTX 1080 Ti | 11 GB | ~98 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA GeForce RTX 2080 Ti | 11 GB | ~134 tok/s | Unknown (no KV data) | No price yet |
| AMD Radeon RX 6700 XT | 12 GB | ~106 tok/s | Unknown (no KV data) | No price yet |
| AMD Radeon RX 6750 XT | 12 GB | ~116 tok/s | Unknown (no KV data) | No price yet |
| AMD Radeon RX 7700 XT | 12 GB | ~116 tok/s | Unknown (no KV data) | No price yet |
| AMD Radeon RX 9070 GRE | 12 GB | ~116 tok/s | Unknown (no KV data) | No price yet |
| Intel Arc Pro A60 | 12 GB | ~57 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA GeForce RTX 3080 Ti | 12 GB | ~162 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA GeForce RTX 4070 | 12 GB | ~121 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA GeForce RTX 4070 Ti | 12 GB | ~121 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA RTX A2000 12GB | 12 GB | ~85 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA Tesla P100 PCIe 12GB | 12 GB | ~106 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA TITAN Xp | 12 GB | ~106 tok/s | Unknown (no KV data) | No price yet |
| AMD Radeon PRO W7700 | 16 GB | ~141 tok/s | Unknown (no KV data) | No price yet |
| AMD Radeon RX 6800 | 16 GB | ~130 tok/s | Unknown (no KV data) | No price yet |
| AMD Radeon RX 6800 XT | 16 GB | ~130 tok/s | Unknown (no KV data) | No price yet |
| AMD Radeon RX 6900 XT | 16 GB | ~130 tok/s | Unknown (no KV data) | No price yet |
| AMD Radeon RX 6950 XT | 16 GB | ~141 tok/s | Unknown (no KV data) | No price yet |
| AMD Radeon RX 7600 XT | 16 GB | ~85 tok/s | Unknown (no KV data) | No price yet |
| AMD Radeon RX 7700 | 16 GB | ~149 tok/s | Unknown (no KV data) | No price yet |
| AMD Radeon RX 7900 GRE | 16 GB | ~141 tok/s | Unknown (no KV data) | No price yet |
| AMD Radeon RX 9060 XT LP | 16 GB | ~92 tok/s | Unknown (no KV data) | No price yet |
| Intel Arc A770 16GB | 16 GB | ~79 tok/s | Unknown (no KV data) | No price yet |
| Intel Arc Pro B50 | 16 GB | ~55 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA GeForce RTX 4060 Ti 16 GB | 16 GB | ~85 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA GeForce RTX 4070 Ti SUPER | 16 GB | ~141 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA GeForce RTX 4080 | 16 GB | ~145 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA GeForce RTX 4080 SUPER | 16 GB | ~147 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA RTX 2000 Ada Generation | 16 GB | ~71 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA RTX A4000 | 16 GB | ~113 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA RTX PRO 2000 Blackwell | 16 GB | ~85 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA T4 | 16 GB | ~87 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA Tesla P100 PCIe 16GB | 16 GB | ~125 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA V100 PCIe 16GB | 16 GB | ~161 tok/s | Unknown (no KV data) | No price yet |
| AMD Radeon RX 7900 XT | 20 GB | ~173 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA RTX 4000 Ada Generation | 20 GB | ~98 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA RTX 4000 SFF Ada Generation | 20 GB | ~83 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA RTX A4500 | 20 GB | ~137 tok/s | Unknown (no KV data) | No price yet |
| Intel Arc Pro B60 | 24 GB | ~99 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA A10 | 24 GB | ~133 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA GeForce RTX 3090 Ti | 24 GB | ~168 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA L4 | 24 GB | ~87 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA Quadro RTX 6000 | 24 GB | ~141 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA RTX 4500 Ada Generation | 24 GB | ~110 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA RTX A5000 | 24 GB | ~150 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA RTX A5500 | 24 GB | ~150 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA RTX PRO 4000 Blackwell | 24 GB | ~141 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA RTX PRO 4000 SFF Blackwell | 24 GB | ~110 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA Tesla P40 | 24 GB | ~78 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA TITAN RTX | 24 GB | ~141 tok/s | Unknown (no KV data) | No price yet |
| AMD Radeon PRO V710 | 28 GB | ~119 tok/s | Unknown (no KV data) | No price yet |
| AMD Radeon AI PRO R9600 | 32 GB | ~151 tok/s | Unknown (no KV data) | No price yet |
| AMD Radeon AI PRO R9600D | 32 GB | ~151 tok/s | Unknown (no KV data) | No price yet |
| AMD Radeon AI PRO R9700 | 32 GB | ~151 tok/s | Unknown (no KV data) | No price yet |
| AMD Radeon AI PRO R9700S | 32 GB | ~151 tok/s | Unknown (no KV data) | No price yet |
| AMD Radeon PRO V620 | 32 GB | ~130 tok/s | Unknown (no KV data) | No price yet |
| AMD Radeon PRO W6800 | 32 GB | ~130 tok/s | Unknown (no KV data) | No price yet |
| AMD Radeon PRO W7800 32GB | 32 GB | ~141 tok/s | Unknown (no KV data) | No price yet |
| Intel Arc Pro B70 | 32 GB | ~122 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA Quadro GV100 | 32 GB | ~158 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA RTX 5000 Ada Generation | 32 GB | ~130 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA RTX PRO 4500 Blackwell | 32 GB | ~160 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA V100 PCIe 32GB | 32 GB | ~161 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA V100S PCIe 32GB | 32 GB | ~176 tok/s | Unknown (no KV data) | No price yet |
| Apple M5 Pro 48GB | 48 GB (~36 usable) | ~84 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA A100 40GB PCIe | 40 GB | ~195 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA A800 40GB Active | 40 GB | ~195 tok/s | Unknown (no KV data) | No price yet |
| AMD Radeon PRO W7800 48GB | 48 GB | ~181 tok/s | Unknown (no KV data) | No price yet |
| AMD Radeon PRO W7900 | 48 GB | ~181 tok/s | Unknown (no KV data) | No price yet |
| AMD Radeon PRO W7900 Dual Slot | 48 GB | ~181 tok/s | Unknown (no KV data) | No price yet |
| Apple Mac Studio M5 Max (40-core GPU, 64 GB) | 64 GB (~48 usable) | ~123 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA A40 | 48 GB | ~143 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA L40 | 48 GB | ~158 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA L40S | 48 GB | ~158 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA Quadro RTX 8000 | 48 GB | ~141 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA RTX 6000 Ada Generation | 48 GB | ~165 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA RTX PRO 5000 Blackwell 48GB | 48 GB | ~187 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA RTX PRO 5000 Blackwell 72GB | 72 GB | ~187 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA A100 80GB PCIe | 80 GB | ~207 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA RTX PRO 6000 Blackwell Max-Q | 96 GB | ~203 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA RTX PRO 6000 Blackwell Server Edition | 96 GB | ~197 tok/s | Unknown (no KV data) | No price yet |
| NVIDIA RTX PRO 6000 Blackwell Workstation Edition | 96 GB | ~203 tok/s | Unknown (no KV data) | No price yet |
Every device that runs it
Graphics cards, Macs and mini PCs from the wider hardware list behind My rig, smallest usable memory first. Not sure about yours? Check your exact rig.
Show all 224 devices that run it
| Device | Type | Memory | Speed estimate | Max context estimate | Price |
|---|---|---|---|---|---|
| AMD Radeon PRO W6400 | Workstation GPU | 4 GB | ~43 tok/s | Unknown Tight | No price yet |
| AMD Radeon RX 6400 | GPU | 4 GB | ~43 tok/s | Unknown Tight | No price yet |
| AMD Radeon RX 6500 XT 4GB | GPU | 4 GB | ~47 tok/s | Unknown Tight | No price yet |
| AMD Radeon RX 9050 4GB | GPU | 4 GB | ~47 tok/s | Unknown Tight | No price yet |
| Intel Arc A310 | GPU | 4 GB | ~20 tok/s | Unknown Tight | No price yet |
| NVIDIA GeForce GTX 1050 Ti | GPU | 4 GB | ~31 tok/s | Unknown Tight | No price yet |
| NVIDIA GeForce GTX 1650 (GDDR5) | GPU | 4 GB | ~45 tok/s | Unknown Tight | No price yet |
| NVIDIA GeForce GTX 1650 (GDDR6) | GPU | 4 GB | ~63 tok/s | Unknown Tight | No price yet |
| NVIDIA RTX A400 | Workstation GPU | 4 GB | ~35 tok/s | Unknown Tight | No price yet |
| Apple M2 8GB | Mac | 8 GB (~5.4 usable) | ~36 tok/s | Unknown | No price yet |
| Apple M3 8GB | Mac | 8 GB (~5.4 usable) | ~36 tok/s | Unknown | No price yet |
| Intel Arc A380 | GPU | 6 GB | ~30 tok/s | Unknown | No price yet |
| Intel Arc Pro A40 | Workstation GPU | 6 GB | ~31 tok/s | Unknown | No price yet |
| Intel Arc Pro A50 | Workstation GPU | 6 GB | ~31 tok/s | Unknown | No price yet |
| NVIDIA GeForce GTX 1060 6GB | GPU | 6 GB | ~49 tok/s | Unknown | No price yet |
| NVIDIA GeForce GTX 1660 | GPU | 6 GB | ~63 tok/s | Unknown | No price yet |
| NVIDIA GeForce GTX 1660 SUPER | GPU | 6 GB | ~94 tok/s | Unknown | No price yet |
| NVIDIA GeForce GTX 1660 Ti | GPU | 6 GB | ~85 tok/s | Unknown | No price yet |
| NVIDIA GeForce RTX 2060 6GB | GPU | 6 GB | ~94 tok/s | Unknown | No price yet |
| NVIDIA RTX A2000 6GB | Workstation GPU | 6 GB | ~85 tok/s | Unknown | No price yet |
| AMD Radeon PRO W6600 | Workstation GPU | 8 GB | ~69 tok/s | Unknown | No price yet |
| AMD Radeon PRO W7400 | Workstation GPU | 8 GB | ~55 tok/s | Unknown | No price yet |
| AMD Radeon PRO W7500 | Workstation GPU | 8 GB | ~77 tok/s | Unknown | No price yet |
| AMD Radeon PRO W7600 | Workstation GPU | 8 GB | ~85 tok/s | Unknown | No price yet |
| AMD Radeon RX 6500 XT 8GB | GPU | 8 GB | ~47 tok/s | Unknown | No price yet |
| AMD Radeon RX 6600 | GPU | 8 GB | ~69 tok/s | Unknown | No price yet |
| AMD Radeon RX 6600 XT | GPU | 8 GB | ~77 tok/s | Unknown | No price yet |
| AMD Radeon RX 6650 XT | GPU | 8 GB | ~83 tok/s | Unknown | No price yet |
| AMD Radeon RX 7400 | GPU | 8 GB | ~56 tok/s | Unknown | No price yet |
| AMD Radeon RX 7600 | GPU | 8 GB | ~85 tok/s | Unknown | No price yet |
| AMD Radeon RX 9050 | GPU | 8 GB | ~85 tok/s | Unknown | No price yet |
| AMD Radeon RX 9060 | GPU | 8 GB | ~85 tok/s | Unknown | No price yet |
| AMD Radeon RX 9060 XT 8GB | GPU | 8 GB | ~92 tok/s | Unknown | No price yet |
| Intel Arc A580 | GPU | 8 GB | ~73 tok/s | Unknown | No price yet |
| Intel Arc A750 | GPU | 8 GB | ~73 tok/s | Unknown | No price yet |
| Intel Arc A770 8GB | GPU | 8 GB | ~73 tok/s | Unknown | No price yet |
| NVIDIA GeForce GTX 1070 | GPU | 8 GB | ~62 tok/s | Unknown | No price yet |
| NVIDIA GeForce GTX 1070 Ti | GPU | 8 GB | ~62 tok/s | Unknown | No price yet |
| NVIDIA GeForce GTX 1080 | GPU | 8 GB | ~74 tok/s | Unknown | No price yet |
| NVIDIA GeForce RTX 2060 SUPER | GPU | 8 GB | ~113 tok/s | Unknown | No price yet |
| NVIDIA GeForce RTX 2070 | GPU | 8 GB | ~113 tok/s | Unknown | No price yet |
| NVIDIA GeForce RTX 2070 SUPER | GPU | 8 GB | ~113 tok/s | Unknown | No price yet |
| NVIDIA GeForce RTX 2080 | GPU | 8 GB | ~113 tok/s | Unknown | No price yet |
| NVIDIA GeForce RTX 2080 SUPER | GPU | 8 GB | ~120 tok/s | Unknown | No price yet |
| NVIDIA GeForce RTX 3070 | GPU | 8 GB | ~113 tok/s | Unknown | No price yet |
| NVIDIA GeForce RTX 3070 Ti | GPU | 8 GB | ~134 tok/s | Unknown | No price yet |
| NVIDIA GeForce RTX 4060 | GPU | 8 GB | ~81 tok/s | Unknown | No price yet |
| NVIDIA GeForce RTX 4060 Ti 8GB | GPU | 8 GB | ~85 tok/s | Unknown | No price yet |
| NVIDIA GeForce RTX 5050 | GPU | 8 GB | ~91 tok/s | Unknown | No price yet |
| NVIDIA GeForce RTX 5060 | GPU | 8 GB | ~113 tok/s | Unknown | low $410 |
| NVIDIA GeForce RTX 5060 Ti 8 GB | GPU | 8 GB | ~113 tok/s | Unknown | low $450 |
| NVIDIA RTX A1000 | Workstation GPU | 8 GB | ~63 tok/s | Unknown | No price yet |
| NVIDIA T1000 8GB | Workstation GPU | 8 GB | ~54 tok/s | Unknown | No price yet |
| AMD Radeon RX 6700 | GPU | 10 GB | ~92 tok/s | Unknown | No price yet |
| Intel Arc B570 | GPU | 10 GB | ~86 tok/s | Unknown | No price yet |
| NVIDIA GeForce RTX 3080 | GPU | 10 GB | ~149 tok/s | Unknown | No price yet |
| Apple M1 Pro 16GB | Mac | 16 GB (~10.7 usable) | ~63 tok/s | Unknown | No price yet |
| Apple M2 16GB | Mac | 16 GB (~10.7 usable) | ~36 tok/s | Unknown | No price yet |
| Apple M2 Pro 16GB | Mac | 16 GB (~10.7 usable) | ~63 tok/s | Unknown | No price yet |
| Apple M3 16GB | Mac | 16 GB (~10.7 usable) | ~36 tok/s | Unknown | No price yet |
| Apple M4 16GB | Mac | 16 GB (~10.7 usable) | ~42 tok/s | Unknown | No price yet |
| Apple M5 16GB | Mac | 16 GB (~10.7 usable) | ~51 tok/s | Unknown | No price yet |
| Apple M6 16GB | Mac | 16 GB (~10.7 usable) | ~51 tok/s | Unknown | No price yet |
| NVIDIA GeForce GTX 1080 Ti | GPU | 11 GB | ~98 tok/s | Unknown | No price yet |
| NVIDIA GeForce RTX 2080 Ti | GPU | 11 GB | ~134 tok/s | Unknown | No price yet |
| AMD Radeon RX 6700 XT | GPU | 12 GB | ~106 tok/s | Unknown | No price yet |
| AMD Radeon RX 6750 XT | GPU | 12 GB | ~116 tok/s | Unknown | No price yet |
| AMD Radeon RX 7700 XT | GPU | 12 GB | ~116 tok/s | Unknown | No price yet |
| AMD Radeon RX 9070 GRE | GPU | 12 GB | ~116 tok/s | Unknown | No price yet |
| Intel Arc B580 | GPU | 12 GB | ~99 tok/s | Unknown | low $330 |
| Intel Arc Pro A60 | Workstation GPU | 12 GB | ~57 tok/s | Unknown | No price yet |
| NVIDIA GeForce RTX 3060 12 GB | GPU | 12 GB | ~98 tok/s | Unknown | low $305 |
| NVIDIA GeForce RTX 3080 Ti | GPU | 12 GB | ~162 tok/s | Unknown | No price yet |
| NVIDIA GeForce RTX 4070 | GPU | 12 GB | ~121 tok/s | Unknown | No price yet |
| NVIDIA GeForce RTX 4070 SUPER | GPU | 12 GB | ~121 tok/s | Unknown | low $620 |
| NVIDIA GeForce RTX 4070 Ti | GPU | 12 GB | ~121 tok/s | Unknown | No price yet |
| NVIDIA GeForce RTX 5070 | GPU | 12 GB | ~141 tok/s | Unknown | low $800 |
| NVIDIA RTX A2000 12GB | Workstation GPU | 12 GB | ~85 tok/s | Unknown | No price yet |
| NVIDIA Tesla P100 PCIe 12GB | Data center GPU | 12 GB | ~106 tok/s | Unknown | No price yet |
| NVIDIA TITAN Xp | GPU | 12 GB | ~106 tok/s | Unknown | No price yet |
| Apple M3 Pro 18GB | Mac | 18 GB (~12.1 usable) | ~50 tok/s | Unknown | No price yet |
| AMD Radeon PRO W7700 | Workstation GPU | 16 GB | ~141 tok/s | Unknown | No price yet |
| AMD Radeon RX 6800 | GPU | 16 GB | ~130 tok/s | Unknown | No price yet |
| AMD Radeon RX 6800 XT | GPU | 16 GB | ~130 tok/s | Unknown | No price yet |
| AMD Radeon RX 6900 XT | GPU | 16 GB | ~130 tok/s | Unknown | No price yet |
| AMD Radeon RX 6950 XT | GPU | 16 GB | ~141 tok/s | Unknown | No price yet |
| AMD Radeon RX 7600 XT | GPU | 16 GB | ~85 tok/s | Unknown | No price yet |
| AMD Radeon RX 7700 | GPU | 16 GB | ~149 tok/s | Unknown | No price yet |
| AMD Radeon RX 7800 XT | GPU | 16 GB | ~149 tok/s | Unknown | low $599 |
| AMD Radeon RX 7900 GRE | GPU | 16 GB | ~141 tok/s | Unknown | No price yet |
| AMD Radeon RX 9060 XT 16 GB | GPU | 16 GB | ~92 tok/s | Unknown | low $620 |
| AMD Radeon RX 9060 XT LP | GPU | 16 GB | ~92 tok/s | Unknown | No price yet |
| AMD Radeon RX 9070 | GPU | 16 GB | ~151 tok/s | Unknown | low $570 |
| AMD Radeon RX 9070 XT | GPU | 16 GB | ~151 tok/s | Unknown | low $810 |
| Intel Arc A770 16GB | GPU | 16 GB | ~79 tok/s | Unknown | No price yet |
| Intel Arc Pro B50 | Workstation GPU | 16 GB | ~55 tok/s | Unknown | No price yet |
| NVIDIA GeForce RTX 4060 Ti 16 GB | GPU | 16 GB | ~85 tok/s | Unknown | No price yet |
| NVIDIA GeForce RTX 4070 Ti SUPER | GPU | 16 GB | ~141 tok/s | Unknown | No price yet |
| NVIDIA GeForce RTX 4080 | GPU | 16 GB | ~145 tok/s | Unknown | No price yet |
| NVIDIA GeForce RTX 4080 SUPER | GPU | 16 GB | ~147 tok/s | Unknown | No price yet |
| NVIDIA GeForce RTX 5060 Ti 16 GB | GPU | 16 GB | ~113 tok/s | Unknown | low $780 |
| NVIDIA GeForce RTX 5070 Ti | GPU | 16 GB | ~160 tok/s | Unknown | low $1,120 |
| NVIDIA GeForce RTX 5080 | GPU | 16 GB | ~165 tok/s | Unknown | low $1,600 |
| NVIDIA RTX 2000 Ada Generation | Workstation GPU | 16 GB | ~71 tok/s | Unknown | No price yet |
| NVIDIA RTX A4000 | Workstation GPU | 16 GB | ~113 tok/s | Unknown | No price yet |
| NVIDIA RTX PRO 2000 Blackwell | Workstation GPU | 16 GB | ~85 tok/s | Unknown | No price yet |
| NVIDIA T4 | Data center GPU | 16 GB | ~87 tok/s | Unknown | No price yet |
| NVIDIA Tesla P100 PCIe 16GB | Data center GPU | 16 GB | ~125 tok/s | Unknown | No price yet |
| NVIDIA V100 PCIe 16GB | Data center GPU | 16 GB | ~161 tok/s | Unknown | No price yet |
| Apple M2 24GB | Mac | 24 GB (~16.1 usable) | ~36 tok/s | Unknown | No price yet |
| Apple M3 24GB | Mac | 24 GB (~16.1 usable) | ~36 tok/s | Unknown | No price yet |
| Apple M4 24GB | Mac | 24 GB (~16.1 usable) | ~42 tok/s | Unknown | No price yet |
| Apple M4 Pro 24GB | Mac | 24 GB (~16.1 usable) | ~78 tok/s | Unknown | No price yet |
| Apple M5 24GB | Mac | 24 GB (~16.1 usable) | ~51 tok/s | Unknown | No price yet |
| Apple M5 Pro 24GB | Mac | 24 GB (~16.1 usable) | ~84 tok/s | Unknown | No price yet |
| Apple M6 24GB | Mac | 24 GB (~16.1 usable) | ~55 tok/s | Unknown | No price yet |
| AMD Radeon RX 7900 XT | GPU | 20 GB | ~173 tok/s | Unknown | No price yet |
| NVIDIA RTX 4000 Ada Generation | Workstation GPU | 20 GB | ~98 tok/s | Unknown | No price yet |
| NVIDIA RTX 4000 SFF Ada Generation | Workstation GPU | 20 GB | ~83 tok/s | Unknown | No price yet |
| NVIDIA RTX A4500 | Workstation GPU | 20 GB | ~137 tok/s | Unknown | No price yet |
| Apple M1 Max 32GB | Mac | 32 GB (~21.4 usable) | ~99 tok/s | Unknown | No price yet |
| Apple M1 Pro 32GB | Mac | 32 GB (~21.4 usable) | ~63 tok/s | Unknown | No price yet |
| Apple M2 Max (30-core GPU) 32GB | Mac | 32 GB (~21.4 usable) | ~99 tok/s | Unknown | No price yet |
| Apple M2 Max (38-core GPU) 32GB | Mac | 32 GB (~21.4 usable) | ~99 tok/s | Unknown | No price yet |
| Apple M2 Pro 32GB | Mac | 32 GB (~21.4 usable) | ~63 tok/s | Unknown | No price yet |
| Apple M4 32GB | Mac | 32 GB (~21.4 usable) | ~42 tok/s | Unknown | No price yet |
| Apple M5 32GB | Mac | 32 GB (~21.4 usable) | ~51 tok/s | Unknown | No price yet |
| Apple M6 32GB | Mac | 32 GB (~21.4 usable) | ~55 tok/s | Unknown | No price yet |
| AMD Radeon RX 7900 XTX | GPU | 24 GB | ~192 tok/s | Unknown | low $1,350 |
| Intel Arc Pro B60 | Workstation GPU | 24 GB | ~99 tok/s | Unknown | No price yet |
| NVIDIA A10 | Data center GPU | 24 GB | ~133 tok/s | Unknown | No price yet |
| NVIDIA GeForce RTX 3090 | GPU | 24 GB | ~163 tok/s | Unknown | low $1,499 |
| NVIDIA GeForce RTX 3090 Ti | GPU | 24 GB | ~168 tok/s | Unknown | No price yet |
| NVIDIA GeForce RTX 4090 | GPU | 24 GB | ~168 tok/s | Unknown | low $2,910 |
| NVIDIA L4 | Data center GPU | 24 GB | ~87 tok/s | Unknown | No price yet |
| NVIDIA Quadro RTX 6000 | Workstation GPU | 24 GB | ~141 tok/s | Unknown | No price yet |
| NVIDIA RTX 4500 Ada Generation | Workstation GPU | 24 GB | ~110 tok/s | Unknown | No price yet |
| NVIDIA RTX A5000 | Workstation GPU | 24 GB | ~150 tok/s | Unknown | No price yet |
| NVIDIA RTX A5500 | Workstation GPU | 24 GB | ~150 tok/s | Unknown | No price yet |
| NVIDIA RTX PRO 4000 Blackwell | Workstation GPU | 24 GB | ~141 tok/s | Unknown | No price yet |
| NVIDIA RTX PRO 4000 SFF Blackwell | Workstation GPU | 24 GB | ~110 tok/s | Unknown | No price yet |
| NVIDIA RTX Spark N1X laptop (6144-core GPU, 24GB) | Laptop | 24 GB | ~87 tok/s | Unknown | No price yet |
| NVIDIA Tesla P40 | Data center GPU | 24 GB | ~78 tok/s | Unknown | No price yet |
| NVIDIA TITAN RTX | GPU | 24 GB | ~141 tok/s | Unknown | No price yet |
| Apple M3 Max (30-core GPU) 36GB | Mac | 36 GB (~24.1 usable) | ~83 tok/s | Unknown | No price yet |
| Apple M3 Pro 36GB | Mac | 36 GB (~24.1 usable) | ~50 tok/s | Unknown | No price yet |
| Apple M4 Max (32-core GPU) 36GB | Mac | 36 GB (~24.1 usable) | ~100 tok/s | Unknown | No price yet |
| Apple Mac Studio M5 Max (32-core GPU, 36 GB) | Mac | 36 GB (~24.1 usable) | ~107 tok/s | Unknown | low $2,499 |
| AMD Radeon PRO V710 | Data center GPU | 28 GB | ~119 tok/s | Unknown | No price yet |
| AMD Radeon AI PRO R9600 | Workstation GPU | 32 GB | ~151 tok/s | Unknown | No price yet |
| AMD Radeon AI PRO R9600D | Workstation GPU | 32 GB | ~151 tok/s | Unknown | No price yet |
| AMD Radeon AI PRO R9700 | Workstation GPU | 32 GB | ~151 tok/s | Unknown | No price yet |
| AMD Radeon AI PRO R9700S | Workstation GPU | 32 GB | ~151 tok/s | Unknown | No price yet |
| AMD Radeon PRO V620 | Data center GPU | 32 GB | ~130 tok/s | Unknown | No price yet |
| AMD Radeon PRO W6800 | Workstation GPU | 32 GB | ~130 tok/s | Unknown | No price yet |
| AMD Radeon PRO W7800 32GB | Workstation GPU | 32 GB | ~141 tok/s | Unknown | No price yet |
| AMD Ryzen AI Max 385 (32GB) | Mini PC | 32 GB | ~77 tok/s | Unknown | No price yet |
| AMD Ryzen AI Max 390 (32GB) | Mini PC | 32 GB | ~77 tok/s | Unknown | No price yet |
| AMD Ryzen AI Max+ 395 (32GB) | Mini PC | 32 GB | ~77 tok/s | Unknown | No price yet |
| Intel Arc Pro B70 | Workstation GPU | 32 GB | ~122 tok/s | Unknown | No price yet |
| NVIDIA GeForce RTX 5090 | GPU | 32 GB | ~203 tok/s | Unknown | low $6,500 |
| NVIDIA Quadro GV100 | Workstation GPU | 32 GB | ~158 tok/s | Unknown | No price yet |
| NVIDIA RTX 5000 Ada Generation | Workstation GPU | 32 GB | ~130 tok/s | Unknown | No price yet |
| NVIDIA RTX PRO 4500 Blackwell | Workstation GPU | 32 GB | ~160 tok/s | Unknown | No price yet |
| NVIDIA RTX Spark N1X laptop (6144-core GPU, 32GB) | Laptop | 32 GB | ~87 tok/s | Unknown | No price yet |
| NVIDIA V100 PCIe 32GB | Data center GPU | 32 GB | ~161 tok/s | Unknown | No price yet |
| NVIDIA V100S PCIe 32GB | Data center GPU | 32 GB | ~176 tok/s | Unknown | No price yet |
| Apple M3 Max (40-core GPU) 48GB | Mac | 48 GB (~36 usable) | ~99 tok/s | Unknown | No price yet |
| Apple M4 Max (40-core GPU) 48GB | Mac | 48 GB (~36 usable) | ~117 tok/s | Unknown | No price yet |
| Apple M4 Pro 48GB | Mac | 48 GB (~36 usable) | ~78 tok/s | Unknown | No price yet |
| Apple M5 Max (40-core GPU) 48GB | Mac | 48 GB (~36 usable) | ~123 tok/s | Unknown | No price yet |
| Apple M5 Pro 48GB | Mac | 48 GB (~36 usable) | ~84 tok/s | Unknown | No price yet |
| NVIDIA A100 40GB PCIe | Data center GPU | 40 GB | ~195 tok/s | Unknown | No price yet |
| NVIDIA A800 40GB Active | Workstation GPU | 40 GB | ~195 tok/s | Unknown | No price yet |
| AMD Radeon PRO W7800 48GB | Workstation GPU | 48 GB | ~181 tok/s | Unknown | No price yet |
| AMD Radeon PRO W7900 | Workstation GPU | 48 GB | ~181 tok/s | Unknown | No price yet |
| AMD Radeon PRO W7900 Dual Slot | Workstation GPU | 48 GB | ~181 tok/s | Unknown | No price yet |
| Apple M1 Max 64GB | Mac | 64 GB (~48 usable) | ~99 tok/s | Unknown | No price yet |
| Apple M1 Ultra 64GB | Mac | 64 GB (~48 usable) | ~138 tok/s | Unknown | No price yet |
| Apple M2 Max (30-core GPU) 64GB | Mac | 64 GB (~48 usable) | ~99 tok/s | Unknown | No price yet |
| Apple M2 Max (38-core GPU) 64GB | Mac | 64 GB (~48 usable) | ~99 tok/s | Unknown | No price yet |
| Apple M2 Ultra 64GB | Mac | 64 GB (~48 usable) | ~138 tok/s | Unknown | No price yet |
| Apple M3 Max (40-core GPU) 64GB | Mac | 64 GB (~48 usable) | ~99 tok/s | Unknown | No price yet |
| Apple M4 Max (40-core GPU) 64GB | Mac | 64 GB (~48 usable) | ~117 tok/s | Unknown | No price yet |
| Apple M4 Pro 64GB | Mac | 64 GB (~48 usable) | ~78 tok/s | Unknown | No price yet |
| Apple M5 Pro 64GB | Mac | 64 GB (~48 usable) | ~84 tok/s | Unknown | No price yet |
| Apple Mac Studio M5 Max (40-core GPU, 64 GB) | Mac | 64 GB (~48 usable) | ~123 tok/s | Unknown | No price yet |
| NVIDIA A40 | Data center GPU | 48 GB | ~143 tok/s | Unknown | No price yet |
| NVIDIA L40 | Data center GPU | 48 GB | ~158 tok/s | Unknown | No price yet |
| NVIDIA L40S | Data center GPU | 48 GB | ~158 tok/s | Unknown | No price yet |
| NVIDIA Quadro RTX 8000 | Workstation GPU | 48 GB | ~141 tok/s | Unknown | No price yet |
| NVIDIA RTX 6000 Ada Generation | Workstation GPU | 48 GB | ~165 tok/s | Unknown | No price yet |
| NVIDIA RTX A6000 | Workstation GPU | 48 GB | ~150 tok/s | Unknown | low $8,363 |
| NVIDIA RTX PRO 5000 Blackwell 48GB | Workstation GPU | 48 GB | ~187 tok/s | Unknown | No price yet |
| NVIDIA RTX Spark N1X laptop (6144-core GPU, 48GB) | Laptop | 48 GB | ~87 tok/s | Unknown | No price yet |
| AMD Ryzen AI Max 385 (64GB) | Mini PC | 64 GB | ~77 tok/s | Unknown | No price yet |
| AMD Ryzen AI Max 390 (64GB) | Mini PC | 64 GB | ~77 tok/s | Unknown | No price yet |
| AMD Ryzen AI Max+ 395 (64GB) | Mini PC | 64 GB | ~77 tok/s | Unknown | low $2,199.99 |
| NVIDIA DGX Spark (64GB) | Mini PC | 64 GB | ~82 tok/s | Unknown | low $4,999 |
| NVIDIA RTX Spark N1X laptop (6144-core GPU, 64GB) | Laptop | 64 GB | ~87 tok/s | Unknown | No price yet |
| Apple M2 Max (38-core GPU) 96GB | Mac | 96 GB (~72 usable) | ~99 tok/s | Unknown | No price yet |
| Apple M3 Max (30-core GPU) 96GB | Mac | 96 GB (~72 usable) | ~83 tok/s | Unknown | No price yet |
| Apple M3 Ultra 96GB | Mac | 96 GB (~72 usable) | ~140 tok/s | Unknown | No price yet |
| Apple Mac Studio M5 Ultra (64-core GPU, 96 GB) | Mac | 96 GB (~72 usable) | ~160 tok/s | Unknown | low $5,499 |
| NVIDIA RTX PRO 5000 Blackwell 72GB | Workstation GPU | 72 GB | ~187 tok/s | Unknown | No price yet |
| NVIDIA A100 80GB PCIe | Data center GPU | 80 GB | ~207 tok/s | Unknown | No price yet |
| Apple M1 Ultra 128GB | Mac | 128 GB (~96 usable) | ~138 tok/s | Unknown | No price yet |
| Apple M2 Ultra 128GB | Mac | 128 GB (~96 usable) | ~138 tok/s | Unknown | No price yet |
| Apple M3 Max (40-core GPU) 128GB | Mac | 128 GB (~96 usable) | ~99 tok/s | Unknown | No price yet |
| Apple M4 Max (40-core GPU) 128GB | Mac | 128 GB (~96 usable) | ~117 tok/s | Unknown | No price yet |
| Apple M5 Max (40-core GPU) 128GB | Mac | 128 GB (~96 usable) | ~123 tok/s | Unknown | No price yet |
| NVIDIA RTX PRO 6000 Blackwell Max-Q | Workstation GPU | 96 GB | ~203 tok/s | Unknown | No price yet |
| NVIDIA RTX PRO 6000 Blackwell Server Edition | Data center GPU | 96 GB | ~197 tok/s | Unknown | No price yet |
| NVIDIA RTX PRO 6000 Blackwell Workstation Edition | Workstation GPU | 96 GB | ~203 tok/s | Unknown | No price yet |
| AMD Ryzen AI Max 385 (128GB) | Mini PC | 128 GB | ~77 tok/s | Unknown | No price yet |
| AMD Ryzen AI Max 390 (128GB) | Mini PC | 128 GB | ~77 tok/s | Unknown | No price yet |
| AMD Ryzen AI Max+ 395 (128GB) | Mini PC | 128 GB | ~77 tok/s | Unknown | low $3,499.99 |
| NVIDIA DGX Spark (128GB) | Mini PC | 128 GB | ~82 tok/s | Unknown | low $6,949.99 |
| NVIDIA RTX Spark N1X laptop (6144-core GPU, 128GB) | Laptop | 128 GB | ~87 tok/s | Unknown | No price yet |
| Apple M2 Ultra 192GB | Mac | 192 GB (~144 usable) | ~138 tok/s | Unknown | No price yet |
| Apple M3 Ultra 256GB | Mac | 256 GB (~192 usable) | ~140 tok/s | Unknown | No price yet |
| Apple M5 Ultra 256GB | Mac | 256 GB (~192 usable) | ~160 tok/s | Unknown | No price yet |
| Framework Desktop, AMD Ryzen AI Max+ PRO 495 (Strix Halo), 192 GB | Mini PC | 192 GB | ~81 tok/s | Unknown | low $6,799 |
| Apple M5 Ultra 512GB | Mac | 512 GB (~384 usable) | ~160 tok/s | Unknown | No price yet |
Speed calculator
estimateEstimate = bandwidth ÷ GB read per token (the model's listed GGUF file: Q4_K_M for most models, otherwise the quant named in the list; scaled by active/total parameters for MoE), scaled by a per-platform efficiency and fixed per-token costs calibrated to 45 measured llama.cpp benchmarks (2025–26). Fit = file size + 1.5 GB headroom ≤ total VRAM (on Macs, the part of unified memory macOS lets the GPU use by default), and room for at least 4K tokens of context with an 8-bit KV cache. For a model with no KV cache data, only the file and headroom are checked, so a fit says nothing about context. Multi-GPU: VRAM adds, speed stays one card's (a layer split runs the cards in turn); max context uses the total. Speed is for short context. Max context = what is left after the file and a 1 GB runtime reserve ÷ KV cache bytes per token, default FP16 and 8-bit (q8_0) cache. How speed works · how context works
Related
- BuildBuild that fits it: 16 GB starter: Radeon RX 9070
- BuildBuild that fits it: 16 GB NVIDIA: RTX 5060 Ti 16 GB
- BuildBuild that fits it: 24 GB: one used RTX 3090
- BuildBuild that fits it: 32 GB: two Radeon RX 9070
- BuildBuild that fits it: 48 GB: dual used RTX 3090
- ModelRaon-1 mini preview, the larger MoE model
- ModelLFM3-1.7B, one of the models KRAFTON compares against
- ModelGemma 4 E2B, another per-layer embedding model