What GPU do you need to run PLaMo 3 Translate 8B?
A used GeForce RTX 3060 12 GB ($305). It is the cheapest priced graphics card that holds PLaMo 3 Translate 8B (about 6.8 GB at Q4) with room for its full 32K context, at about 56 tok/s estimate; that is the card price only, and it requires a separate compatible PC. 139 of the 157 GPUs and systems we track fit it.
Translation only PLaMo 3 Translate 8B is a translation model, not a general chat assistant. The fits and speeds below say whether it runs on a machine, not that it is a good everyday chat model, so the site's automatic picks (the largest model a card runs, can-run pages) leave it out. License: PLaMo Community License.
8.53B parameters (Hugging Face, accessed 2026-11-10). Q4_K_M GGUF: 5.28 GB (source). Architecture config (Hugging Face config.json; 32 layers, window_size 2048, sliding_window_pattern 8: llama.cpp plamo3 (load_swa_pattern(ml, 8), LLAMA_SWA_TYPE_STANDARD) makes layer il sliding-window unless il % 8 == 7, so 4 full-attention + 28 sliding-window layers; the official GGUF metadata says the same (sliding_window 2048, sliding_window_pattern 8). max_ctx is the model card's 32,768-token training limit for input and output combined (the context field), not config.json's max_position_embeddings 262,144): 4 global attention layers × 4 KV heads × 128 head dim, plus 28 sliding-window layers (2048-token window), so about 8 KB of FP16 KV cache per token of context (how we estimate context).
Translation model, not a general chat assistant: it takes text in one language and returns a translation, through its own translation prompt format. Dense model, one of four PLaMo 3 Translate sizes. Context is 32,768 tokens because the model card says training used at most 32,768 tokens of input and output combined and asks you to split anything longer; config.json allows 262,144 positions, which this page does not treat as usable. Commercial use needs advance registration with Preferred Networks (see License below). Release date is the date of PFN's release blog post.
What we checked
Checked Nov 10, 2026. Official is the maker or the weight files. A vendor claim is that project's own documentation. A community claim is someone else's upload. Our check is a file or search we read. Unknown means we have no source, so it is not estimated. None of these rows is a speed we measured.
| Kind | What | Detail | Source |
|---|---|---|---|
| Official | Model card | PFN's card describes PLaMo 3 Translate as a translation-specialized model, tagged translation on Hugging Face, not a general chat or coding assistant. It uses a translation chat template (the PFML3 prompt format). The release blog covers four sizes (0.8B, 2B, 8B, 31B) and 53 languages, and says the model accepts surrounding context and glossaries and can write plain or polite Japanese. The card says the longest training sequence was 32,768 tokens for input and output combined and to split anything longer, so this page uses 32,768 as the context, not config.json's 262,144. Safetensors total 8,531,750,912 parameters (8.53B), BF16, dense. The repo is not gated. | Model card, checked Nov 10, 2026 |
| Official | GGUF files | PFN publishes the GGUF build itself (pfnet/plamo-3-translate-8b-GGUF), one file per quant: Q4_K_M 5,280,445,728 bytes, Q3_K_M 4,310,218,016 bytes, Q2_K 3,450,746,144 bytes. This page's fit and speed estimates use the Q4_K_M file. The 31B card's example runs llama-server with -c 16384, --jinja and --no-prefill-assistant; 16,384 is only their example. We read the sizes from the Hugging Face file list and did not download them. | GGUF repo, checked Nov 10, 2026 |
| Official | MLX 4-bit build | PFN also publishes an MLX 4-bit build for Apple silicon (pfnet/plamo-3-translate-8b-MLX-4bit), 6,066,104,664 bytes of safetensors. Its card says to run it with mlx-lm with trust_remote_code=True. Other official formats in the release are BF16, FP8, W4A16, W4AFP8 and NVFP4. We have not run the MLX build on a Mac, and no MLX speed is listed. | MLX-4bit repo, checked Nov 10, 2026 |
| Official | License: commercial use needs registration | PLaMo Community License. Personal and non-commercial use, including academic use, is free with no registration. Commercial use is free only if you (with your affiliates) had revenue of JPY 1 billion or less in the most recent fiscal year AND you register with PFN in advance through its form (https://forms.gle/mTL8tBLrMYXKNZD56). Above JPY 1 billion you need a separate license agreement with PFN (https://translate.preferredai.jp/llm/contact/). This is a summary of the license, not legal advice. | PLaMo Community License, checked Nov 10, 2026 |
| Vendor claim | PFN's benchmark claims | PFN's own claim: even the 0.8B model outperforms existing translation-specialized models (TranslateGemma, Hy-MT2) on Japanese-English and on Japanese to and from other languages, and PFN calls PLaMo 3 Translate the world's best translation-specialized LLM. Its benchmarks are PFTB and SaaS (PFN internal sets), WMT24++ (COMET-22) and FLORES+ (chrF++). PFN's card also says the Q2_K GGUF loses accuracy on some metrics while the other formats stay close to BF16. These are PFN's results; we have not reproduced any of them. | PFN release blog, checked Nov 10, 2026 |
| Our check | llama.cpp and mlx-lm support | llama.cpp master has a plamo3 architecture (src/models/plamo3.cpp), and the official GGUF header says general.architecture plamo3. With a sliding window set, llama.cpp loads a sliding-window pattern of 8: in each group of 8 layers, 7 use a 2,048-token sliding window and 1 uses full attention. For this size that is 4 full-attention and 28 sliding-window layers of 32, with 4 KV heads of size 128, which is what the context estimates use. mlx-lm master has mlx_lm/models/plamo3.py. | llama.cpp plamo3.cpp, checked Nov 10, 2026 |
| Unknown | Not tested by us | We have not run any PLaMo 3 Translate model. No speed on this page is measured: tok/s figures are the site's bandwidth estimates. Translation quality is not tested by us, and the sizes are not compared with each other or with other models here. | Model card does not give this, checked Nov 10, 2026 |
GGUF files and sizes
Exact sizes from Hugging Face, accessed 2026-11-10. The fit and context tables on this page use the Q4_K_M file (5.28 GB); a smaller file needs less memory by the difference. VRAM needed is the file + the same headroom.
| File | Published by | Bytes | Size | VRAM needed | Source |
|---|---|---|---|---|---|
| Q2_K | Preferred Networks | 3,450,746,144 | 3.45 GB | ~5 GB | Hugging Face |
| Q3_K_M | Preferred Networks | 4,310,218,016 | 4.31 GB | ~5.8 GB | Hugging Face |
| Q4_K_M | Preferred Networks | 5,280,445,728 | 5.28 GB | ~6.8 GB | Hugging Face |
Other places to get it
Checked Nov 14, 2026. We list only copies we opened and checked. Check file hashes for copies from other uploaders.
- ModelScope: Other uploaderSame account name as on Hugging Face; we found no link from the lab to it
Single GPUs that fit, cheapest first
| Single GPU that fits | VRAM | Speed estimate | Max context estimate | Price |
|---|---|---|---|---|
| NVIDIA GeForce RTX 3060 12 GB | 12 GB | ~56 tok/s | 32K (full) | low $305 |
| Intel Arc B580 | 12 GB | ~51 tok/s | 32K (full) | low $330 |
| NVIDIA GeForce RTX 5060 | 8 GB | ~69 tok/s | 32K (full) | low $410 |
| NVIDIA GeForce RTX 5060 Ti 8 GB | 8 GB | ~69 tok/s | 32K (full) | low $450 |
| AMD Radeon RX 9070 | 16 GB | ~93 tok/s | 32K (full) | low $570 |
| AMD Radeon RX 7800 XT | 16 GB | ~91 tok/s | 32K (full) | low $599 |
| NVIDIA GeForce RTX 4070 SUPER | 12 GB | ~76 tok/s | 32K (full) | low $620 |
| AMD Radeon RX 9060 XT 16 GB | 16 GB | ~47 tok/s | 32K (full) | low $620 |
| NVIDIA GeForce RTX 5060 Ti 16 GB | 16 GB | ~69 tok/s | 32K (full) | low $780 |
| NVIDIA GeForce RTX 5070 | 12 GB | ~99 tok/s | 32K (full) | low $800 |
| AMD Radeon RX 9070 XT | 16 GB | ~93 tok/s | 32K (full) | low $810 |
| NVIDIA GeForce RTX 5070 Ti | 16 GB | ~127 tok/s | 32K (full) | low $1,120 |
| AMD Radeon RX 7900 XTX | 24 GB | ~140 tok/s | 32K (full) | low $1,350 |
| NVIDIA GeForce RTX 3090 | 24 GB | ~131 tok/s | 32K (full) | low $1,499 |
| NVIDIA GeForce RTX 5080 | 16 GB | ~134 tok/s | 32K (full) | low $1,600 |
| Apple Mac Studio M5 Max (32-core GPU, 36 GB) | 36 GB (~24.1 usable) | ~68 tok/s | 32K (full) | low $2,499 |
| NVIDIA GeForce RTX 4090 | 24 GB | ~140 tok/s | 32K (full) | low $2,910 |
| NVIDIA DGX Spark (64GB) | 64 GB | ~43 tok/s | 32K (full) | low $4,999 |
| Apple Mac Studio M5 Ultra (64-core GPU, 96 GB) | 96 GB (~72 usable) | ~146 tok/s | 32K (full) | low $5,499 |
| NVIDIA GeForce RTX 5090 | 32 GB | ~220 tok/s | 32K (full) | low $6,500 |
| Framework Desktop, AMD Ryzen AI Max+ PRO 495 (Strix Halo), 192 GB | 192 GB | ~40 tok/s | 32K (full) | low $6,799 |
| NVIDIA DGX Spark (128GB) | 128 GB | ~43 tok/s | 32K (full) | low $6,949.99 |
| NVIDIA RTX A6000 | 48 GB | ~111 tok/s | 32K (full) | low $8,363 |
| AMD Radeon PRO W6600 | 8 GB | ~33 tok/s | 32K (full) | No price yet |
| AMD Radeon PRO W7400 | 8 GB | ~25 tok/s | 32K (full) | No price yet |
| AMD Radeon PRO W7500 | 8 GB | ~37 tok/s | 32K (full) | No price yet |
| AMD Radeon PRO W7600 | 8 GB | ~42 tok/s | 32K (full) | No price yet |
| AMD Radeon RX 6500 XT 8GB | 8 GB | ~21 tok/s | 32K (full) | No price yet |
| AMD Radeon RX 6600 | 8 GB | ~33 tok/s | 32K (full) | No price yet |
| AMD Radeon RX 6600 XT | 8 GB | ~37 tok/s | 32K (full) | No price yet |
| AMD Radeon RX 6650 XT | 8 GB | ~41 tok/s | 32K (full) | No price yet |
| AMD Radeon RX 7400 | 8 GB | ~25 tok/s | 32K (full) | No price yet |
| AMD Radeon RX 7600 | 8 GB | ~42 tok/s | 32K (full) | No price yet |
| AMD Radeon RX 9050 | 8 GB | ~42 tok/s | 32K (full) | No price yet |
| AMD Radeon RX 9060 | 8 GB | ~42 tok/s | 32K (full) | No price yet |
| AMD Radeon RX 9060 XT 8GB | 8 GB | ~47 tok/s | 32K (full) | No price yet |
| Intel Arc A580 | 8 GB | ~35 tok/s | 32K (full) | No price yet |
| Intel Arc A750 | 8 GB | ~35 tok/s | 32K (full) | No price yet |
| Intel Arc A770 8GB | 8 GB | ~35 tok/s | 32K (full) | No price yet |
| NVIDIA GeForce GTX 1070 | 8 GB | ~30 tok/s | 32K (full) | No price yet |
| NVIDIA GeForce GTX 1070 Ti | 8 GB | ~30 tok/s | 32K (full) | No price yet |
| NVIDIA GeForce GTX 1080 | 8 GB | ~38 tok/s | 32K (full) | No price yet |
| NVIDIA GeForce RTX 2060 SUPER | 8 GB | ~69 tok/s | 32K (full) | No price yet |
| NVIDIA GeForce RTX 2070 | 8 GB | ~69 tok/s | 32K (full) | No price yet |
| NVIDIA GeForce RTX 2070 SUPER | 8 GB | ~69 tok/s | 32K (full) | No price yet |
| NVIDIA GeForce RTX 2080 | 8 GB | ~69 tok/s | 32K (full) | No price yet |
| NVIDIA GeForce RTX 2080 SUPER | 8 GB | ~75 tok/s | 32K (full) | No price yet |
| NVIDIA GeForce RTX 3070 | 8 GB | ~69 tok/s | 32K (full) | No price yet |
| NVIDIA GeForce RTX 3070 Ti | 8 GB | ~90 tok/s | 32K (full) | No price yet |
| NVIDIA GeForce RTX 4060 | 8 GB | ~43 tok/s | 32K (full) | No price yet |
| NVIDIA GeForce RTX 4060 Ti 8GB | 8 GB | ~45 tok/s | 32K (full) | No price yet |
| NVIDIA GeForce RTX 5050 | 8 GB | ~50 tok/s | 32K (full) | No price yet |
| NVIDIA RTX A1000 | 8 GB | ~31 tok/s | 32K (full) | No price yet |
| NVIDIA T1000 8GB | 8 GB | ~26 tok/s | 32K (full) | No price yet |
| AMD Radeon RX 6700 | 10 GB | ~47 tok/s | 32K (full) | No price yet |
| Intel Arc B570 | 10 GB | ~42 tok/s | 32K (full) | No price yet |
| NVIDIA GeForce RTX 3080 | 10 GB | ~110 tok/s | 32K (full) | No price yet |
| NVIDIA GeForce GTX 1080 Ti | 11 GB | ~56 tok/s | 32K (full) | No price yet |
| NVIDIA GeForce RTX 2080 Ti | 11 GB | ~91 tok/s | 32K (full) | No price yet |
| AMD Radeon RX 6700 XT | 12 GB | ~56 tok/s | 32K (full) | No price yet |
| AMD Radeon RX 6750 XT | 12 GB | ~63 tok/s | 32K (full) | No price yet |
| AMD Radeon RX 7700 XT | 12 GB | ~63 tok/s | 32K (full) | No price yet |
| AMD Radeon RX 9070 GRE | 12 GB | ~63 tok/s | 32K (full) | No price yet |
| Intel Arc Pro A60 | 12 GB | ~26 tok/s | 32K (full) | No price yet |
| NVIDIA GeForce RTX 3080 Ti | 12 GB | ~129 tok/s | 32K (full) | No price yet |
| NVIDIA GeForce RTX 4070 | 12 GB | ~76 tok/s | 32K (full) | No price yet |
| NVIDIA GeForce RTX 4070 Ti | 12 GB | ~76 tok/s | 32K (full) | No price yet |
| NVIDIA RTX A2000 12GB | 12 GB | ~45 tok/s | 32K (full) | No price yet |
| NVIDIA Tesla P100 PCIe 12GB | 12 GB | ~63 tok/s | 32K (full) | No price yet |
| NVIDIA TITAN Xp | 12 GB | ~62 tok/s | 32K (full) | No price yet |
| AMD Radeon PRO W7700 | 16 GB | ~84 tok/s | 32K (full) | No price yet |
| AMD Radeon RX 6800 | 16 GB | ~75 tok/s | 32K (full) | No price yet |
| AMD Radeon RX 6800 XT | 16 GB | ~75 tok/s | 32K (full) | No price yet |
| AMD Radeon RX 6900 XT | 16 GB | ~75 tok/s | 32K (full) | No price yet |
| AMD Radeon RX 6950 XT | 16 GB | ~84 tok/s | 32K (full) | No price yet |
| AMD Radeon RX 7600 XT | 16 GB | ~42 tok/s | 32K (full) | No price yet |
| AMD Radeon RX 7700 | 16 GB | ~91 tok/s | 32K (full) | No price yet |
| AMD Radeon RX 7900 GRE | 16 GB | ~84 tok/s | 32K (full) | No price yet |
| AMD Radeon RX 9060 XT LP | 16 GB | ~47 tok/s | 32K (full) | No price yet |
| Intel Arc A770 16GB | 16 GB | ~38 tok/s | 32K (full) | No price yet |
| Intel Arc Pro B50 | 16 GB | ~25 tok/s | 32K (full) | No price yet |
| NVIDIA GeForce RTX 4060 Ti 16 GB | 16 GB | ~45 tok/s | 32K (full) | No price yet |
| NVIDIA GeForce RTX 4070 Ti SUPER | 16 GB | ~99 tok/s | 32K (full) | No price yet |
| NVIDIA GeForce RTX 4080 | 16 GB | ~104 tok/s | 32K (full) | No price yet |
| NVIDIA GeForce RTX 4080 SUPER | 16 GB | ~107 tok/s | 32K (full) | No price yet |
| NVIDIA RTX 2000 Ada Generation | 16 GB | ~36 tok/s | 32K (full) | No price yet |
| NVIDIA RTX A4000 | 16 GB | ~69 tok/s | 32K (full) | No price yet |
| NVIDIA RTX PRO 2000 Blackwell | 16 GB | ~45 tok/s | 32K (full) | No price yet |
| NVIDIA T4 | 16 GB | ~47 tok/s | 32K (full) | No price yet |
| NVIDIA Tesla P100 PCIe 16GB | 16 GB | ~81 tok/s | 32K (full) | No price yet |
| NVIDIA V100 PCIe 16GB | 16 GB | ~127 tok/s | 32K (full) | No price yet |
| AMD Radeon RX 7900 XT | 20 GB | ~117 tok/s | 32K (full) | No price yet |
| NVIDIA RTX 4000 Ada Generation | 20 GB | ~56 tok/s | 32K (full) | No price yet |
| NVIDIA RTX 4000 SFF Ada Generation | 20 GB | ~44 tok/s | 32K (full) | No price yet |
| NVIDIA RTX A4500 | 20 GB | ~95 tok/s | 32K (full) | No price yet |
| Intel Arc Pro B60 | 24 GB | ~51 tok/s | 32K (full) | No price yet |
| NVIDIA A10 | 24 GB | ~89 tok/s | 32K (full) | No price yet |
| NVIDIA GeForce RTX 3090 Ti | 24 GB | ~140 tok/s | 32K (full) | No price yet |
| NVIDIA L4 | 24 GB | ~47 tok/s | 32K (full) | No price yet |
| NVIDIA Quadro RTX 6000 | 24 GB | ~99 tok/s | 32K (full) | No price yet |
| NVIDIA RTX 4500 Ada Generation | 24 GB | ~66 tok/s | 32K (full) | No price yet |
| NVIDIA RTX A5000 | 24 GB | ~111 tok/s | 32K (full) | No price yet |
| NVIDIA RTX A5500 | 24 GB | ~111 tok/s | 32K (full) | No price yet |
| NVIDIA RTX PRO 4000 Blackwell | 24 GB | ~99 tok/s | 32K (full) | No price yet |
| NVIDIA RTX PRO 4000 SFF Blackwell | 24 GB | ~66 tok/s | 32K (full) | No price yet |
| NVIDIA Tesla P40 | 24 GB | ~41 tok/s | 32K (full) | No price yet |
| NVIDIA TITAN RTX | 24 GB | ~99 tok/s | 32K (full) | No price yet |
| AMD Radeon PRO V710 | 28 GB | ~65 tok/s | 32K (full) | No price yet |
| AMD Radeon AI PRO R9600 | 32 GB | ~93 tok/s | 32K (full) | No price yet |
| AMD Radeon AI PRO R9600D | 32 GB | ~93 tok/s | 32K (full) | No price yet |
| AMD Radeon AI PRO R9700 | 32 GB | ~93 tok/s | 32K (full) | No price yet |
| AMD Radeon AI PRO R9700S | 32 GB | ~93 tok/s | 32K (full) | No price yet |
| AMD Radeon PRO V620 | 32 GB | ~75 tok/s | 32K (full) | No price yet |
| AMD Radeon PRO W6800 | 32 GB | ~75 tok/s | 32K (full) | No price yet |
| AMD Radeon PRO W7800 32GB | 32 GB | ~84 tok/s | 32K (full) | No price yet |
| Intel Arc Pro B70 | 32 GB | ~68 tok/s | 32K (full) | No price yet |
| NVIDIA Quadro GV100 | 32 GB | ~124 tok/s | 32K (full) | No price yet |
| NVIDIA RTX 5000 Ada Generation | 32 GB | ~86 tok/s | 32K (full) | No price yet |
| NVIDIA RTX PRO 4500 Blackwell | 32 GB | ~127 tok/s | 32K (full) | No price yet |
| NVIDIA V100 PCIe 32GB | 32 GB | ~127 tok/s | 32K (full) | No price yet |
| NVIDIA V100S PCIe 32GB | 32 GB | ~154 tok/s | 32K (full) | No price yet |
| Apple M5 Pro 48GB | 48 GB (~36 usable) | ~47 tok/s | 32K (full) | No price yet |
| NVIDIA A100 40GB PCIe | 40 GB | ~198 tok/s | 32K (full) | No price yet |
| NVIDIA A800 40GB Active | 40 GB | ~198 tok/s | 32K (full) | No price yet |
| AMD Radeon PRO W7800 48GB | 48 GB | ~126 tok/s | 32K (full) | No price yet |
| AMD Radeon PRO W7900 | 48 GB | ~126 tok/s | 32K (full) | No price yet |
| AMD Radeon PRO W7900 Dual Slot | 48 GB | ~126 tok/s | 32K (full) | No price yet |
| Apple Mac Studio M5 Max (40-core GPU, 64 GB) | 64 GB (~48 usable) | ~86 tok/s | 32K (full) | No price yet |
| NVIDIA A40 | 48 GB | ~102 tok/s | 32K (full) | No price yet |
| NVIDIA L40 | 48 GB | ~123 tok/s | 32K (full) | No price yet |
| NVIDIA L40S | 48 GB | ~123 tok/s | 32K (full) | No price yet |
| NVIDIA Quadro RTX 8000 | 48 GB | ~99 tok/s | 32K (full) | No price yet |
| NVIDIA RTX 6000 Ada Generation | 48 GB | ~134 tok/s | 32K (full) | No price yet |
| NVIDIA RTX PRO 5000 Blackwell 48GB | 48 GB | ~177 tok/s | 32K (full) | No price yet |
| NVIDIA RTX PRO 5000 Blackwell 72GB | 72 GB | ~177 tok/s | 32K (full) | No price yet |
| NVIDIA A100 80GB PCIe | 80 GB | ~233 tok/s | 32K (full) | No price yet |
| NVIDIA RTX PRO 6000 Blackwell Max-Q | 96 GB | ~220 tok/s | 32K (full) | No price yet |
| NVIDIA RTX PRO 6000 Blackwell Server Edition | 96 GB | ~203 tok/s | 32K (full) | No price yet |
| NVIDIA RTX PRO 6000 Blackwell Workstation Edition | 96 GB | ~220 tok/s | 32K (full) | No price yet |
Every device that runs it
Graphics cards, Macs and mini PCs from the wider hardware list behind My rig, smallest usable memory first. Not sure about yours? Check your exact rig.
Show all 204 devices that run it
| Device | Type | Memory | Speed estimate | Max context estimate | Price |
|---|---|---|---|---|---|
| AMD Radeon PRO W6600 | Workstation GPU | 8 GB | ~33 tok/s | 32K (full) | No price yet |
| AMD Radeon PRO W7400 | Workstation GPU | 8 GB | ~25 tok/s | 32K (full) | No price yet |
| AMD Radeon PRO W7500 | Workstation GPU | 8 GB | ~37 tok/s | 32K (full) | No price yet |
| AMD Radeon PRO W7600 | Workstation GPU | 8 GB | ~42 tok/s | 32K (full) | No price yet |
| AMD Radeon RX 6500 XT 8GB | GPU | 8 GB | ~21 tok/s | 32K (full) | No price yet |
| AMD Radeon RX 6600 | GPU | 8 GB | ~33 tok/s | 32K (full) | No price yet |
| AMD Radeon RX 6600 XT | GPU | 8 GB | ~37 tok/s | 32K (full) | No price yet |
| AMD Radeon RX 6650 XT | GPU | 8 GB | ~41 tok/s | 32K (full) | No price yet |
| AMD Radeon RX 7400 | GPU | 8 GB | ~25 tok/s | 32K (full) | No price yet |
| AMD Radeon RX 7600 | GPU | 8 GB | ~42 tok/s | 32K (full) | No price yet |
| AMD Radeon RX 9050 | GPU | 8 GB | ~42 tok/s | 32K (full) | No price yet |
| AMD Radeon RX 9060 | GPU | 8 GB | ~42 tok/s | 32K (full) | No price yet |
| AMD Radeon RX 9060 XT 8GB | GPU | 8 GB | ~47 tok/s | 32K (full) | No price yet |
| Intel Arc A580 | GPU | 8 GB | ~35 tok/s | 32K (full) | No price yet |
| Intel Arc A750 | GPU | 8 GB | ~35 tok/s | 32K (full) | No price yet |
| Intel Arc A770 8GB | GPU | 8 GB | ~35 tok/s | 32K (full) | No price yet |
| NVIDIA GeForce GTX 1070 | GPU | 8 GB | ~30 tok/s | 32K (full) | No price yet |
| NVIDIA GeForce GTX 1070 Ti | GPU | 8 GB | ~30 tok/s | 32K (full) | No price yet |
| NVIDIA GeForce GTX 1080 | GPU | 8 GB | ~38 tok/s | 32K (full) | No price yet |
| NVIDIA GeForce RTX 2060 SUPER | GPU | 8 GB | ~69 tok/s | 32K (full) | No price yet |
| NVIDIA GeForce RTX 2070 | GPU | 8 GB | ~69 tok/s | 32K (full) | No price yet |
| NVIDIA GeForce RTX 2070 SUPER | GPU | 8 GB | ~69 tok/s | 32K (full) | No price yet |
| NVIDIA GeForce RTX 2080 | GPU | 8 GB | ~69 tok/s | 32K (full) | No price yet |
| NVIDIA GeForce RTX 2080 SUPER | GPU | 8 GB | ~75 tok/s | 32K (full) | No price yet |
| NVIDIA GeForce RTX 3070 | GPU | 8 GB | ~69 tok/s | 32K (full) | No price yet |
| NVIDIA GeForce RTX 3070 Ti | GPU | 8 GB | ~90 tok/s | 32K (full) | No price yet |
| NVIDIA GeForce RTX 4060 | GPU | 8 GB | ~43 tok/s | 32K (full) | No price yet |
| NVIDIA GeForce RTX 4060 Ti 8GB | GPU | 8 GB | ~45 tok/s | 32K (full) | No price yet |
| NVIDIA GeForce RTX 5050 | GPU | 8 GB | ~50 tok/s | 32K (full) | No price yet |
| NVIDIA GeForce RTX 5060 | GPU | 8 GB | ~69 tok/s | 32K (full) | low $410 |
| NVIDIA GeForce RTX 5060 Ti 8 GB | GPU | 8 GB | ~69 tok/s | 32K (full) | low $450 |
| NVIDIA RTX A1000 | Workstation GPU | 8 GB | ~31 tok/s | 32K (full) | No price yet |
| NVIDIA T1000 8GB | Workstation GPU | 8 GB | ~26 tok/s | 32K (full) | No price yet |
| AMD Radeon RX 6700 | GPU | 10 GB | ~47 tok/s | 32K (full) | No price yet |
| Intel Arc B570 | GPU | 10 GB | ~42 tok/s | 32K (full) | No price yet |
| NVIDIA GeForce RTX 3080 | GPU | 10 GB | ~110 tok/s | 32K (full) | No price yet |
| Apple M1 Pro 16GB | Mac | 16 GB (~10.7 usable) | ~32 tok/s | 32K (full) | No price yet |
| Apple M2 16GB | Mac | 16 GB (~10.7 usable) | ~16 tok/s | 32K (full) | No price yet |
| Apple M2 Pro 16GB | Mac | 16 GB (~10.7 usable) | ~32 tok/s | 32K (full) | No price yet |
| Apple M3 16GB | Mac | 16 GB (~10.7 usable) | ~16 tok/s | 32K (full) | No price yet |
| Apple M4 16GB | Mac | 16 GB (~10.7 usable) | ~19 tok/s | 32K (full) | No price yet |
| Apple M5 16GB | Mac | 16 GB (~10.7 usable) | ~25 tok/s | 32K (full) | No price yet |
| Apple M6 16GB | Mac | 16 GB (~10.7 usable) | ~25 tok/s | 32K (full) | No price yet |
| NVIDIA GeForce GTX 1080 Ti | GPU | 11 GB | ~56 tok/s | 32K (full) | No price yet |
| NVIDIA GeForce RTX 2080 Ti | GPU | 11 GB | ~91 tok/s | 32K (full) | No price yet |
| AMD Radeon RX 6700 XT | GPU | 12 GB | ~56 tok/s | 32K (full) | No price yet |
| AMD Radeon RX 6750 XT | GPU | 12 GB | ~63 tok/s | 32K (full) | No price yet |
| AMD Radeon RX 7700 XT | GPU | 12 GB | ~63 tok/s | 32K (full) | No price yet |
| AMD Radeon RX 9070 GRE | GPU | 12 GB | ~63 tok/s | 32K (full) | No price yet |
| Intel Arc B580 | GPU | 12 GB | ~51 tok/s | 32K (full) | low $330 |
| Intel Arc Pro A60 | Workstation GPU | 12 GB | ~26 tok/s | 32K (full) | No price yet |
| NVIDIA GeForce RTX 3060 12 GB | GPU | 12 GB | ~56 tok/s | 32K (full) | low $305 |
| NVIDIA GeForce RTX 3080 Ti | GPU | 12 GB | ~129 tok/s | 32K (full) | No price yet |
| NVIDIA GeForce RTX 4070 | GPU | 12 GB | ~76 tok/s | 32K (full) | No price yet |
| NVIDIA GeForce RTX 4070 SUPER | GPU | 12 GB | ~76 tok/s | 32K (full) | low $620 |
| NVIDIA GeForce RTX 4070 Ti | GPU | 12 GB | ~76 tok/s | 32K (full) | No price yet |
| NVIDIA GeForce RTX 5070 | GPU | 12 GB | ~99 tok/s | 32K (full) | low $800 |
| NVIDIA RTX A2000 12GB | Workstation GPU | 12 GB | ~45 tok/s | 32K (full) | No price yet |
| NVIDIA Tesla P100 PCIe 12GB | Data center GPU | 12 GB | ~63 tok/s | 32K (full) | No price yet |
| NVIDIA TITAN Xp | GPU | 12 GB | ~62 tok/s | 32K (full) | No price yet |
| Apple M3 Pro 18GB | Mac | 18 GB (~12.1 usable) | ~24 tok/s | 32K (full) | No price yet |
| AMD Radeon PRO W7700 | Workstation GPU | 16 GB | ~84 tok/s | 32K (full) | No price yet |
| AMD Radeon RX 6800 | GPU | 16 GB | ~75 tok/s | 32K (full) | No price yet |
| AMD Radeon RX 6800 XT | GPU | 16 GB | ~75 tok/s | 32K (full) | No price yet |
| AMD Radeon RX 6900 XT | GPU | 16 GB | ~75 tok/s | 32K (full) | No price yet |
| AMD Radeon RX 6950 XT | GPU | 16 GB | ~84 tok/s | 32K (full) | No price yet |
| AMD Radeon RX 7600 XT | GPU | 16 GB | ~42 tok/s | 32K (full) | No price yet |
| AMD Radeon RX 7700 | GPU | 16 GB | ~91 tok/s | 32K (full) | No price yet |
| AMD Radeon RX 7800 XT | GPU | 16 GB | ~91 tok/s | 32K (full) | low $599 |
| AMD Radeon RX 7900 GRE | GPU | 16 GB | ~84 tok/s | 32K (full) | No price yet |
| AMD Radeon RX 9060 XT 16 GB | GPU | 16 GB | ~47 tok/s | 32K (full) | low $620 |
| AMD Radeon RX 9060 XT LP | GPU | 16 GB | ~47 tok/s | 32K (full) | No price yet |
| AMD Radeon RX 9070 | GPU | 16 GB | ~93 tok/s | 32K (full) | low $570 |
| AMD Radeon RX 9070 XT | GPU | 16 GB | ~93 tok/s | 32K (full) | low $810 |
| Intel Arc A770 16GB | GPU | 16 GB | ~38 tok/s | 32K (full) | No price yet |
| Intel Arc Pro B50 | Workstation GPU | 16 GB | ~25 tok/s | 32K (full) | No price yet |
| NVIDIA GeForce RTX 4060 Ti 16 GB | GPU | 16 GB | ~45 tok/s | 32K (full) | No price yet |
| NVIDIA GeForce RTX 4070 Ti SUPER | GPU | 16 GB | ~99 tok/s | 32K (full) | No price yet |
| NVIDIA GeForce RTX 4080 | GPU | 16 GB | ~104 tok/s | 32K (full) | No price yet |
| NVIDIA GeForce RTX 4080 SUPER | GPU | 16 GB | ~107 tok/s | 32K (full) | No price yet |
| NVIDIA GeForce RTX 5060 Ti 16 GB | GPU | 16 GB | ~69 tok/s | 32K (full) | low $780 |
| NVIDIA GeForce RTX 5070 Ti | GPU | 16 GB | ~127 tok/s | 32K (full) | low $1,120 |
| NVIDIA GeForce RTX 5080 | GPU | 16 GB | ~134 tok/s | 32K (full) | low $1,600 |
| NVIDIA RTX 2000 Ada Generation | Workstation GPU | 16 GB | ~36 tok/s | 32K (full) | No price yet |
| NVIDIA RTX A4000 | Workstation GPU | 16 GB | ~69 tok/s | 32K (full) | No price yet |
| NVIDIA RTX PRO 2000 Blackwell | Workstation GPU | 16 GB | ~45 tok/s | 32K (full) | No price yet |
| NVIDIA T4 | Data center GPU | 16 GB | ~47 tok/s | 32K (full) | No price yet |
| NVIDIA Tesla P100 PCIe 16GB | Data center GPU | 16 GB | ~81 tok/s | 32K (full) | No price yet |
| NVIDIA V100 PCIe 16GB | Data center GPU | 16 GB | ~127 tok/s | 32K (full) | No price yet |
| Apple M2 24GB | Mac | 24 GB (~16.1 usable) | ~16 tok/s | 32K (full) | No price yet |
| Apple M3 24GB | Mac | 24 GB (~16.1 usable) | ~16 tok/s | 32K (full) | No price yet |
| Apple M4 24GB | Mac | 24 GB (~16.1 usable) | ~19 tok/s | 32K (full) | No price yet |
| Apple M4 Pro 24GB | Mac | 24 GB (~16.1 usable) | ~42 tok/s | 32K (full) | No price yet |
| Apple M5 24GB | Mac | 24 GB (~16.1 usable) | ~25 tok/s | 32K (full) | No price yet |
| Apple M5 Pro 24GB | Mac | 24 GB (~16.1 usable) | ~47 tok/s | 32K (full) | No price yet |
| Apple M6 24GB | Mac | 24 GB (~16.1 usable) | ~27 tok/s | 32K (full) | No price yet |
| AMD Radeon RX 7900 XT | GPU | 20 GB | ~117 tok/s | 32K (full) | No price yet |
| NVIDIA RTX 4000 Ada Generation | Workstation GPU | 20 GB | ~56 tok/s | 32K (full) | No price yet |
| NVIDIA RTX 4000 SFF Ada Generation | Workstation GPU | 20 GB | ~44 tok/s | 32K (full) | No price yet |
| NVIDIA RTX A4500 | Workstation GPU | 20 GB | ~95 tok/s | 32K (full) | No price yet |
| Apple M1 Max 32GB | Mac | 32 GB (~21.4 usable) | ~60 tok/s | 32K (full) | No price yet |
| Apple M1 Pro 32GB | Mac | 32 GB (~21.4 usable) | ~32 tok/s | 32K (full) | No price yet |
| Apple M2 Max (30-core GPU) 32GB | Mac | 32 GB (~21.4 usable) | ~60 tok/s | 32K (full) | No price yet |
| Apple M2 Max (38-core GPU) 32GB | Mac | 32 GB (~21.4 usable) | ~60 tok/s | 32K (full) | No price yet |
| Apple M2 Pro 32GB | Mac | 32 GB (~21.4 usable) | ~32 tok/s | 32K (full) | No price yet |
| Apple M4 32GB | Mac | 32 GB (~21.4 usable) | ~19 tok/s | 32K (full) | No price yet |
| Apple M5 32GB | Mac | 32 GB (~21.4 usable) | ~25 tok/s | 32K (full) | No price yet |
| Apple M6 32GB | Mac | 32 GB (~21.4 usable) | ~27 tok/s | 32K (full) | No price yet |
| AMD Radeon RX 7900 XTX | GPU | 24 GB | ~140 tok/s | 32K (full) | low $1,350 |
| Intel Arc Pro B60 | Workstation GPU | 24 GB | ~51 tok/s | 32K (full) | No price yet |
| NVIDIA A10 | Data center GPU | 24 GB | ~89 tok/s | 32K (full) | No price yet |
| NVIDIA GeForce RTX 3090 | GPU | 24 GB | ~131 tok/s | 32K (full) | low $1,499 |
| NVIDIA GeForce RTX 3090 Ti | GPU | 24 GB | ~140 tok/s | 32K (full) | No price yet |
| NVIDIA GeForce RTX 4090 | GPU | 24 GB | ~140 tok/s | 32K (full) | low $2,910 |
| NVIDIA L4 | Data center GPU | 24 GB | ~47 tok/s | 32K (full) | No price yet |
| NVIDIA Quadro RTX 6000 | Workstation GPU | 24 GB | ~99 tok/s | 32K (full) | No price yet |
| NVIDIA RTX 4500 Ada Generation | Workstation GPU | 24 GB | ~66 tok/s | 32K (full) | No price yet |
| NVIDIA RTX A5000 | Workstation GPU | 24 GB | ~111 tok/s | 32K (full) | No price yet |
| NVIDIA RTX A5500 | Workstation GPU | 24 GB | ~111 tok/s | 32K (full) | No price yet |
| NVIDIA RTX PRO 4000 Blackwell | Workstation GPU | 24 GB | ~99 tok/s | 32K (full) | No price yet |
| NVIDIA RTX PRO 4000 SFF Blackwell | Workstation GPU | 24 GB | ~66 tok/s | 32K (full) | No price yet |
| NVIDIA RTX Spark N1X laptop (6144-core GPU, 24GB) | Laptop | 24 GB | ~47 tok/s | 32K (full) | No price yet |
| NVIDIA Tesla P40 | Data center GPU | 24 GB | ~41 tok/s | 32K (full) | No price yet |
| NVIDIA TITAN RTX | GPU | 24 GB | ~99 tok/s | 32K (full) | No price yet |
| Apple M3 Max (30-core GPU) 36GB | Mac | 36 GB (~24.1 usable) | ~46 tok/s | 32K (full) | No price yet |
| Apple M3 Pro 36GB | Mac | 36 GB (~24.1 usable) | ~24 tok/s | 32K (full) | No price yet |
| Apple M4 Max (32-core GPU) 36GB | Mac | 36 GB (~24.1 usable) | ~61 tok/s | 32K (full) | No price yet |
| Apple Mac Studio M5 Max (32-core GPU, 36 GB) | Mac | 36 GB (~24.1 usable) | ~68 tok/s | 32K (full) | low $2,499 |
| AMD Radeon PRO V710 | Data center GPU | 28 GB | ~65 tok/s | 32K (full) | No price yet |
| AMD Radeon AI PRO R9600 | Workstation GPU | 32 GB | ~93 tok/s | 32K (full) | No price yet |
| AMD Radeon AI PRO R9600D | Workstation GPU | 32 GB | ~93 tok/s | 32K (full) | No price yet |
| AMD Radeon AI PRO R9700 | Workstation GPU | 32 GB | ~93 tok/s | 32K (full) | No price yet |
| AMD Radeon AI PRO R9700S | Workstation GPU | 32 GB | ~93 tok/s | 32K (full) | No price yet |
| AMD Radeon PRO V620 | Data center GPU | 32 GB | ~75 tok/s | 32K (full) | No price yet |
| AMD Radeon PRO W6800 | Workstation GPU | 32 GB | ~75 tok/s | 32K (full) | No price yet |
| AMD Radeon PRO W7800 32GB | Workstation GPU | 32 GB | ~84 tok/s | 32K (full) | No price yet |
| AMD Ryzen AI Max 385 (32GB) | Mini PC | 32 GB | ~37 tok/s | 32K (full) | No price yet |
| AMD Ryzen AI Max 390 (32GB) | Mini PC | 32 GB | ~37 tok/s | 32K (full) | No price yet |
| AMD Ryzen AI Max+ 395 (32GB) | Mini PC | 32 GB | ~37 tok/s | 32K (full) | No price yet |
| Intel Arc Pro B70 | Workstation GPU | 32 GB | ~68 tok/s | 32K (full) | No price yet |
| NVIDIA GeForce RTX 5090 | GPU | 32 GB | ~220 tok/s | 32K (full) | low $6,500 |
| NVIDIA Quadro GV100 | Workstation GPU | 32 GB | ~124 tok/s | 32K (full) | No price yet |
| NVIDIA RTX 5000 Ada Generation | Workstation GPU | 32 GB | ~86 tok/s | 32K (full) | No price yet |
| NVIDIA RTX PRO 4500 Blackwell | Workstation GPU | 32 GB | ~127 tok/s | 32K (full) | No price yet |
| NVIDIA RTX Spark N1X laptop (6144-core GPU, 32GB) | Laptop | 32 GB | ~47 tok/s | 32K (full) | No price yet |
| NVIDIA V100 PCIe 32GB | Data center GPU | 32 GB | ~127 tok/s | 32K (full) | No price yet |
| NVIDIA V100S PCIe 32GB | Data center GPU | 32 GB | ~154 tok/s | 32K (full) | No price yet |
| Apple M3 Max (40-core GPU) 48GB | Mac | 48 GB (~36 usable) | ~60 tok/s | 32K (full) | No price yet |
| Apple M4 Max (40-core GPU) 48GB | Mac | 48 GB (~36 usable) | ~78 tok/s | 32K (full) | No price yet |
| Apple M4 Pro 48GB | Mac | 48 GB (~36 usable) | ~42 tok/s | 32K (full) | No price yet |
| Apple M5 Max (40-core GPU) 48GB | Mac | 48 GB (~36 usable) | ~86 tok/s | 32K (full) | No price yet |
| Apple M5 Pro 48GB | Mac | 48 GB (~36 usable) | ~47 tok/s | 32K (full) | No price yet |
| NVIDIA A100 40GB PCIe | Data center GPU | 40 GB | ~198 tok/s | 32K (full) | No price yet |
| NVIDIA A800 40GB Active | Workstation GPU | 40 GB | ~198 tok/s | 32K (full) | No price yet |
| AMD Radeon PRO W7800 48GB | Workstation GPU | 48 GB | ~126 tok/s | 32K (full) | No price yet |
| AMD Radeon PRO W7900 | Workstation GPU | 48 GB | ~126 tok/s | 32K (full) | No price yet |
| AMD Radeon PRO W7900 Dual Slot | Workstation GPU | 48 GB | ~126 tok/s | 32K (full) | No price yet |
| Apple M1 Max 64GB | Mac | 64 GB (~48 usable) | ~60 tok/s | 32K (full) | No price yet |
| Apple M1 Ultra 64GB | Mac | 64 GB (~48 usable) | ~107 tok/s | 32K (full) | No price yet |
| Apple M2 Max (30-core GPU) 64GB | Mac | 64 GB (~48 usable) | ~60 tok/s | 32K (full) | No price yet |
| Apple M2 Max (38-core GPU) 64GB | Mac | 64 GB (~48 usable) | ~60 tok/s | 32K (full) | No price yet |
| Apple M2 Ultra 64GB | Mac | 64 GB (~48 usable) | ~107 tok/s | 32K (full) | No price yet |
| Apple M3 Max (40-core GPU) 64GB | Mac | 64 GB (~48 usable) | ~60 tok/s | 32K (full) | No price yet |
| Apple M4 Max (40-core GPU) 64GB | Mac | 64 GB (~48 usable) | ~78 tok/s | 32K (full) | No price yet |
| Apple M4 Pro 64GB | Mac | 64 GB (~48 usable) | ~42 tok/s | 32K (full) | No price yet |
| Apple M5 Pro 64GB | Mac | 64 GB (~48 usable) | ~47 tok/s | 32K (full) | No price yet |
| Apple Mac Studio M5 Max (40-core GPU, 64 GB) | Mac | 64 GB (~48 usable) | ~86 tok/s | 32K (full) | No price yet |
| NVIDIA A40 | Data center GPU | 48 GB | ~102 tok/s | 32K (full) | No price yet |
| NVIDIA L40 | Data center GPU | 48 GB | ~123 tok/s | 32K (full) | No price yet |
| NVIDIA L40S | Data center GPU | 48 GB | ~123 tok/s | 32K (full) | No price yet |
| NVIDIA Quadro RTX 8000 | Workstation GPU | 48 GB | ~99 tok/s | 32K (full) | No price yet |
| NVIDIA RTX 6000 Ada Generation | Workstation GPU | 48 GB | ~134 tok/s | 32K (full) | No price yet |
| NVIDIA RTX A6000 | Workstation GPU | 48 GB | ~111 tok/s | 32K (full) | low $8,363 |
| NVIDIA RTX PRO 5000 Blackwell 48GB | Workstation GPU | 48 GB | ~177 tok/s | 32K (full) | No price yet |
| NVIDIA RTX Spark N1X laptop (6144-core GPU, 48GB) | Laptop | 48 GB | ~47 tok/s | 32K (full) | No price yet |
| AMD Ryzen AI Max 385 (64GB) | Mini PC | 64 GB | ~37 tok/s | 32K (full) | No price yet |
| AMD Ryzen AI Max 390 (64GB) | Mini PC | 64 GB | ~37 tok/s | 32K (full) | No price yet |
| AMD Ryzen AI Max+ 395 (64GB) | Mini PC | 64 GB | ~37 tok/s | 32K (full) | low $2,199.99 |
| NVIDIA DGX Spark (64GB) | Mini PC | 64 GB | ~43 tok/s | 32K (full) | low $4,999 |
| NVIDIA RTX Spark N1X laptop (6144-core GPU, 64GB) | Laptop | 64 GB | ~47 tok/s | 32K (full) | No price yet |
| Apple M2 Max (38-core GPU) 96GB | Mac | 96 GB (~72 usable) | ~60 tok/s | 32K (full) | No price yet |
| Apple M3 Max (30-core GPU) 96GB | Mac | 96 GB (~72 usable) | ~46 tok/s | 32K (full) | No price yet |
| Apple M3 Ultra 96GB | Mac | 96 GB (~72 usable) | ~109 tok/s | 32K (full) | No price yet |
| Apple Mac Studio M5 Ultra (64-core GPU, 96 GB) | Mac | 96 GB (~72 usable) | ~146 tok/s | 32K (full) | low $5,499 |
| NVIDIA RTX PRO 5000 Blackwell 72GB | Workstation GPU | 72 GB | ~177 tok/s | 32K (full) | No price yet |
| NVIDIA A100 80GB PCIe | Data center GPU | 80 GB | ~233 tok/s | 32K (full) | No price yet |
| Apple M1 Ultra 128GB | Mac | 128 GB (~96 usable) | ~107 tok/s | 32K (full) | No price yet |
| Apple M2 Ultra 128GB | Mac | 128 GB (~96 usable) | ~107 tok/s | 32K (full) | No price yet |
| Apple M3 Max (40-core GPU) 128GB | Mac | 128 GB (~96 usable) | ~60 tok/s | 32K (full) | No price yet |
| Apple M4 Max (40-core GPU) 128GB | Mac | 128 GB (~96 usable) | ~78 tok/s | 32K (full) | No price yet |
| Apple M5 Max (40-core GPU) 128GB | Mac | 128 GB (~96 usable) | ~86 tok/s | 32K (full) | No price yet |
| NVIDIA RTX PRO 6000 Blackwell Max-Q | Workstation GPU | 96 GB | ~220 tok/s | 32K (full) | No price yet |
| NVIDIA RTX PRO 6000 Blackwell Server Edition | Data center GPU | 96 GB | ~203 tok/s | 32K (full) | No price yet |
| NVIDIA RTX PRO 6000 Blackwell Workstation Edition | Workstation GPU | 96 GB | ~220 tok/s | 32K (full) | No price yet |
| AMD Ryzen AI Max 385 (128GB) | Mini PC | 128 GB | ~37 tok/s | 32K (full) | No price yet |
| AMD Ryzen AI Max 390 (128GB) | Mini PC | 128 GB | ~37 tok/s | 32K (full) | No price yet |
| AMD Ryzen AI Max+ 395 (128GB) | Mini PC | 128 GB | ~37 tok/s | 32K (full) | low $3,499.99 |
| NVIDIA DGX Spark (128GB) | Mini PC | 128 GB | ~43 tok/s | 32K (full) | low $6,949.99 |
| NVIDIA RTX Spark N1X laptop (6144-core GPU, 128GB) | Laptop | 128 GB | ~47 tok/s | 32K (full) | No price yet |
| Apple M2 Ultra 192GB | Mac | 192 GB (~144 usable) | ~107 tok/s | 32K (full) | No price yet |
| Apple M3 Ultra 256GB | Mac | 256 GB (~192 usable) | ~109 tok/s | 32K (full) | No price yet |
| Apple M5 Ultra 256GB | Mac | 256 GB (~192 usable) | ~146 tok/s | 32K (full) | No price yet |
| Framework Desktop, AMD Ryzen AI Max+ PRO 495 (Strix Halo), 192 GB | Mini PC | 192 GB | ~40 tok/s | 32K (full) | low $6,799 |
| Apple M5 Ultra 512GB | Mac | 512 GB (~384 usable) | ~146 tok/s | 32K (full) | No price yet |
Speed calculator
estimateEstimate = bandwidth ÷ GB read per token (the model's listed GGUF file: Q4_K_M for most models, otherwise the quant named in the list; scaled by active/total parameters for MoE), scaled by a per-platform efficiency and fixed per-token costs calibrated to 45 measured llama.cpp benchmarks (2025–26). Fit = file size + 1.5 GB headroom ≤ total VRAM (on Macs, the part of unified memory macOS lets the GPU use by default), and room for at least 4K tokens of context with an 8-bit KV cache. For a model with no KV cache data, only the file and headroom are checked, so a fit says nothing about context. Multi-GPU: VRAM adds, speed stays one card's (a layer split runs the cards in turn); max context uses the total. Speed is for short context. Max context = what is left after the file and a 1 GB runtime reserve ÷ KV cache bytes per token, default FP16 and 8-bit (q8_0) cache. How speed works · how context works
Related
- BuildBuild that fits it: 16 GB starter: Radeon RX 9070
- BuildBuild that fits it: 16 GB NVIDIA: RTX 5060 Ti 16 GB
- BuildBuild that fits it: 24 GB: one used RTX 3090
- BuildBuild that fits it: 32 GB: two Radeon RX 9070
- BuildBuild that fits it: 48 GB: dual used RTX 3090
- ModelPLaMo 3 Translate 0.8B, another size of the same translation model
- ModelPLaMo 3 Translate 2B, another size of the same translation model
- ModelPLaMo 3 Translate 31B, another size of the same translation model
- CatalogEvery model in the catalog