What GPU do you need to run webAI TwIL-8B?
A used GeForce RTX 3060 12 GB ($305). It is the cheapest priced graphics card that holds webAI TwIL-8B (about 6.8 GB at Q4) with room for about 33K tokens of context (63K with 8-bit KV cache), at about 55 tok/s estimate; that is the card price only, and it requires a separate compatible PC. 139 of the 157 GPUs and systems we track fit it.
Math and code reasoning webAI TwIL-8B is a reasoning model for math, code and instruction following, not a general assistant. The fits and speeds below say whether it runs on a machine, not that it is a good everyday chat model, so the site's automatic picks (the largest model a card runs, can-run pages) leave it out. License: webAI Non-Commercial License ver. 1.0.
Parameters8.79B
VRAM needed~6.8 GBfile + headroom
Single GPUs that fit139
Context window128K tokens
8.79B parameters (Hugging Face, accessed 2026-11-10). Q4_K_M GGUF: 5.35 GB (source). Architecture config (Hugging Face config.json): 40 attention layers × 8 KV heads × 128 head dim, so about 160 KB of FP16 KV cache per token of context (how we estimate context).
Non-commercial use only: the webAI Non-Commercial License allows non-commercial research and educational purposes only. TwIL-8B is a reasoning fine-tune of IBM Granite 4.2 8B (same architecture and size) for contest math, code and instruction following, English only; its card says it is not a general assistant. The base Granite 4.2 8B stays Apache 2.0. Context is 131,072 tokens, inherited from the base. Release date is webAI's launch post; the Hugging Face repo's single commit is dated 2026-11-08.
What we checked
Checked Nov 10, 2026. Official is the maker or the weight files. A vendor claim is that project's own documentation. A community claim is someone else's upload. Our check is a file or search we read. Unknown means we have no source, so it is not estimated. None of these rows is a speed we measured.
| Kind | What | Detail | Source |
|---|---|---|---|
| Official | Model card | webAI's card says TwIL-8B is post-trained from ibm-granite/granite-4.2-8b with full-parameter SFT and GRPO/DAPO reinforcement learning. It is a reasoning model that writes a <think> block first, specialised in contest math, code and instruction following, English only, text in and text out. The card says it is not a general assistant, has no safety, preference or tool-use training beyond what Granite 4.2 carries, and is not a knowledge model. Safetensors total 8,791,592,960 parameters (8.79B), dense, GraniteForCausalLM: 40 layers, 32 attention heads, 8 KV heads of size 128, 131,072-token context. The repo is not gated. | Model card, checked Nov 10, 2026 |
| Official | Balanced and verbose variants | Two variants: balanced (the default, in the repo root) and verbose (in the verbose/ folder). GGUF files in the main repo: TwIL-8B-Q4_K_M-balanced.gguf 5,347,917,568 bytes (the card says 4.98 GiB, imatrix), TwIL-8B-Q4_K_M-verbose.gguf 5,347,917,248, TwIL-8B-Q8_0-verbose.gguf 9,345,613,248 and TwIL-8B-F16-verbose.gguf 17,587,420,608. The bf16 safetensors are 17,583,228,544 bytes per variant. This page's estimates use the balanced Q4_K_M file. The card's llama.cpp example is llama-cli -hf webAI-Official/TwIL-8B --hf-file TwIL-8B-Q4_K_M-balanced.gguf -cnv -c 40960 -n 32768, and it says answers need a large generation budget (webAI says balanced averages about 19.5k output tokens per AIME 2026 problem). | Hugging Face file tree, checked Nov 10, 2026 |
| Official | License: non-commercial only | webAI Non-Commercial License ver. 1.0. Section 3.3: the work and any derivative works may be used or intended for use non-commercially only, and non-commercially means for non-commercial research and educational purposes only. So no commercial use. The base model, IBM Granite 4.2 8B, stays Apache 2.0. This is a summary of the license, not legal advice. | LICENSE.md, checked Nov 10, 2026 |
| Vendor claim | webAI's benchmark claims | webAI's own figures (internal, high-effort pass@1, thinking on): MATH-500 98.2, AIME 2025 88.3, AIME 2026 88.3, GPQA Diamond 70.2, IFEval 91.8, IFBench 74.4 and LiveCodeBench 81.6 (the last two on the verbose variant; the card does not record the LiveCodeBench version). All were measured on bf16 weights, not on the GGUF files. The competitor figures in webAI's table are copied from other sources and were not re-run. These are webAI's claims; we have not reproduced any of them. | Model card, checked Nov 10, 2026 |
| Community claim | Community conversions | Community conversions exist and are not official, for example mradermacher/TwIL-8B-GGUF and pratikgx/TwIL-8B-MLX-4bit (4,945,853,960 bytes of safetensors). webAI publishes no MLX build. On a Mac, the official GGUF runs in llama.cpp, whose tagged releases support the Granite architecture; MLX only through a community conversion, which we have not run. | Hugging Face search, checked Nov 10, 2026 |
| Unknown | Not tested by us | We have not run TwIL-8B. No speed on this page is measured: tok/s figures are the site's bandwidth estimates, and a reasoning model's long thinking output makes each answer take longer than the per-token speed suggests. | Model card does not give this, checked Nov 10, 2026 |
GGUF files and sizes
Exact sizes from Hugging Face, accessed 2026-11-10. The fit and context tables on this page use the Q4_K_M file (5.35 GB); a smaller file needs less memory by the difference. VRAM needed is the file + the same headroom.
| File | Published by | Bytes | Size | VRAM needed | Source |
|---|---|---|---|---|---|
| Q4_K_M | webAI | 5,347,917,248 | 5.35 GB | ~6.8 GB | TwIL-8B-Q4_K_M-verbose.gguf |
| Q4_K_M | webAI | 5,347,917,568 | 5.35 GB | ~6.8 GB | TwIL-8B-Q4_K_M-balanced.gguf |
| Q8_0 | webAI | 9,345,613,248 | 9.35 GB | ~10.8 GB | TwIL-8B-Q8_0-verbose.gguf |
| F16 | webAI | 17,587,420,608 | 17.59 GB | ~19.1 GB | TwIL-8B-F16-verbose.gguf |
Other places to get it
Checked Nov 14, 2026. We list only copies we opened and checked. Check file hashes for copies from other uploaders.
- ModelScope: Other uploaderSame account name as on Hugging Face; we found no link from the lab to it
Single GPUs that fit, cheapest first
| Single GPU that fits | VRAM | Speed estimate | Max context estimate | Price |
|---|---|---|---|---|
| NVIDIA GeForce RTX 3060 12 GB | 12 GB | ~55 tok/s | 33K · 63K (8-bit) | low $305 |
| Intel Arc B580 | 12 GB | ~50 tok/s | 33K · 63K (8-bit) | low $330 |
| NVIDIA GeForce RTX 5060 | 8 GB | ~68 tok/s | 9K · 18K (8-bit) | low $410 |
| NVIDIA GeForce RTX 5060 Ti 8 GB | 8 GB | ~68 tok/s | 9K · 18K (8-bit) | low $450 |
| AMD Radeon RX 9070 | 16 GB | ~92 tok/s | 57K · 108K (8-bit) | low $570 |
| AMD Radeon RX 7800 XT | 16 GB | ~90 tok/s | 57K · 108K (8-bit) | low $599 |
| NVIDIA GeForce RTX 4070 SUPER | 12 GB | ~75 tok/s | 33K · 63K (8-bit) | low $620 |
| AMD Radeon RX 9060 XT 16 GB | 16 GB | ~46 tok/s | 57K · 108K (8-bit) | low $620 |
| NVIDIA GeForce RTX 5060 Ti 16 GB | 16 GB | ~68 tok/s | 57K · 108K (8-bit) | low $780 |
| NVIDIA GeForce RTX 5070 | 12 GB | ~98 tok/s | 33K · 63K (8-bit) | low $800 |
| AMD Radeon RX 9070 XT | 16 GB | ~92 tok/s | 57K · 108K (8-bit) | low $810 |
| NVIDIA GeForce RTX 5070 Ti | 16 GB | ~125 tok/s | 57K · 108K (8-bit) | low $1,120 |
| AMD Radeon RX 7900 XTX | 24 GB | ~138 tok/s | 105K · 128K (8-bit) | low $1,350 |
| NVIDIA GeForce RTX 3090 | 24 GB | ~130 tok/s | 105K · 128K (8-bit) | low $1,499 |
| NVIDIA GeForce RTX 5080 | 16 GB | ~133 tok/s | 57K · 108K (8-bit) | low $1,600 |
| Apple Mac Studio M5 Max (32-core GPU, 36 GB) | 36 GB (~24.1 usable) | ~67 tok/s | 105K · 128K (8-bit) | low $2,499 |
| NVIDIA GeForce RTX 4090 | 24 GB | ~138 tok/s | 105K · 128K (8-bit) | low $2,910 |
| NVIDIA DGX Spark (64GB) | 64 GB | ~43 tok/s | 128K (full) | low $4,999 |
| Apple Mac Studio M5 Ultra (64-core GPU, 96 GB) | 96 GB (~72 usable) | ~145 tok/s | 128K (full) | low $5,499 |
| NVIDIA GeForce RTX 5090 | 32 GB | ~218 tok/s | 128K (full) | low $6,500 |
| Framework Desktop, AMD Ryzen AI Max+ PRO 495 (Strix Halo), 192 GB | 192 GB | ~39 tok/s | 128K (full) | low $6,799 |
| NVIDIA DGX Spark (128GB) | 128 GB | ~43 tok/s | 128K (full) | low $6,949.99 |
| NVIDIA RTX A6000 | 48 GB | ~110 tok/s | 128K (full) | low $8,363 |
| AMD Radeon PRO W6600 | 8 GB | ~32 tok/s | 9K · 18K (8-bit) | No price yet |
| AMD Radeon PRO W7400 | 8 GB | ~25 tok/s | 9K · 18K (8-bit) | No price yet |
| AMD Radeon PRO W7500 | 8 GB | ~37 tok/s | 9K · 18K (8-bit) | No price yet |
| AMD Radeon PRO W7600 | 8 GB | ~41 tok/s | 9K · 18K (8-bit) | No price yet |
| AMD Radeon RX 6500 XT 8GB | 8 GB | ~21 tok/s | 9K · 18K (8-bit) | No price yet |
| AMD Radeon RX 6600 | 8 GB | ~32 tok/s | 9K · 18K (8-bit) | No price yet |
| AMD Radeon RX 6600 XT | 8 GB | ~37 tok/s | 9K · 18K (8-bit) | No price yet |
| AMD Radeon RX 6650 XT | 8 GB | ~40 tok/s | 9K · 18K (8-bit) | No price yet |
| AMD Radeon RX 7400 | 8 GB | ~25 tok/s | 9K · 18K (8-bit) | No price yet |
| AMD Radeon RX 7600 | 8 GB | ~41 tok/s | 9K · 18K (8-bit) | No price yet |
| AMD Radeon RX 9050 | 8 GB | ~41 tok/s | 9K · 18K (8-bit) | No price yet |
| AMD Radeon RX 9060 | 8 GB | ~41 tok/s | 9K · 18K (8-bit) | No price yet |
| AMD Radeon RX 9060 XT 8GB | 8 GB | ~46 tok/s | 9K · 18K (8-bit) | No price yet |
| Intel Arc A580 | 8 GB | ~34 tok/s | 9K · 18K (8-bit) | No price yet |
| Intel Arc A750 | 8 GB | ~34 tok/s | 9K · 18K (8-bit) | No price yet |
| Intel Arc A770 8GB | 8 GB | ~34 tok/s | 9K · 18K (8-bit) | No price yet |
| NVIDIA GeForce GTX 1070 | 8 GB | ~30 tok/s | 9K · 18K (8-bit) | No price yet |
| NVIDIA GeForce GTX 1070 Ti | 8 GB | ~30 tok/s | 9K · 18K (8-bit) | No price yet |
| NVIDIA GeForce GTX 1080 | 8 GB | ~37 tok/s | 9K · 18K (8-bit) | No price yet |
| NVIDIA GeForce RTX 2060 SUPER | 8 GB | ~68 tok/s | 9K · 18K (8-bit) | No price yet |
| NVIDIA GeForce RTX 2070 | 8 GB | ~68 tok/s | 9K · 18K (8-bit) | No price yet |
| NVIDIA GeForce RTX 2070 SUPER | 8 GB | ~68 tok/s | 9K · 18K (8-bit) | No price yet |
| NVIDIA GeForce RTX 2080 | 8 GB | ~68 tok/s | 9K · 18K (8-bit) | No price yet |
| NVIDIA GeForce RTX 2080 SUPER | 8 GB | ~74 tok/s | 9K · 18K (8-bit) | No price yet |
| NVIDIA GeForce RTX 3070 | 8 GB | ~68 tok/s | 9K · 18K (8-bit) | No price yet |
| NVIDIA GeForce RTX 3070 Ti | 8 GB | ~89 tok/s | 9K · 18K (8-bit) | No price yet |
| NVIDIA GeForce RTX 4060 | 8 GB | ~42 tok/s | 9K · 18K (8-bit) | No price yet |
| NVIDIA GeForce RTX 4060 Ti 8GB | 8 GB | ~45 tok/s | 9K · 18K (8-bit) | No price yet |
| NVIDIA GeForce RTX 5050 | 8 GB | ~50 tok/s | 9K · 18K (8-bit) | No price yet |
| NVIDIA RTX A1000 | 8 GB | ~30 tok/s | 9K · 18K (8-bit) | No price yet |
| NVIDIA T1000 8GB | 8 GB | ~26 tok/s | 9K · 18K (8-bit) | No price yet |
| AMD Radeon RX 6700 | 10 GB | ~46 tok/s | 21K · 40K (8-bit) | No price yet |
| Intel Arc B570 | 10 GB | ~42 tok/s | 21K · 40K (8-bit) | No price yet |
| NVIDIA GeForce RTX 3080 | 10 GB | ~109 tok/s | 21K · 40K (8-bit) | No price yet |
| NVIDIA GeForce GTX 1080 Ti | 11 GB | ~55 tok/s | 27K · 52K (8-bit) | No price yet |
| NVIDIA GeForce RTX 2080 Ti | 11 GB | ~90 tok/s | 27K · 52K (8-bit) | No price yet |
| AMD Radeon RX 6700 XT | 12 GB | ~55 tok/s | 33K · 63K (8-bit) | No price yet |
| AMD Radeon RX 6750 XT | 12 GB | ~62 tok/s | 33K · 63K (8-bit) | No price yet |
| AMD Radeon RX 7700 XT | 12 GB | ~62 tok/s | 33K · 63K (8-bit) | No price yet |
| AMD Radeon RX 9070 GRE | 12 GB | ~62 tok/s | 33K · 63K (8-bit) | No price yet |
| Intel Arc Pro A60 | 12 GB | ~26 tok/s | 33K · 63K (8-bit) | No price yet |
| NVIDIA GeForce RTX 3080 Ti | 12 GB | ~127 tok/s | 33K · 63K (8-bit) | No price yet |
| NVIDIA GeForce RTX 4070 | 12 GB | ~75 tok/s | 33K · 63K (8-bit) | No price yet |
| NVIDIA GeForce RTX 4070 Ti | 12 GB | ~75 tok/s | 33K · 63K (8-bit) | No price yet |
| NVIDIA RTX A2000 12GB | 12 GB | ~45 tok/s | 33K · 63K (8-bit) | No price yet |
| NVIDIA Tesla P100 PCIe 12GB | 12 GB | ~62 tok/s | 33K · 63K (8-bit) | No price yet |
| NVIDIA TITAN Xp | 12 GB | ~62 tok/s | 33K · 63K (8-bit) | No price yet |
| AMD Radeon PRO W7700 | 16 GB | ~83 tok/s | 57K · 108K (8-bit) | No price yet |
| AMD Radeon RX 6800 | 16 GB | ~74 tok/s | 57K · 108K (8-bit) | No price yet |
| AMD Radeon RX 6800 XT | 16 GB | ~74 tok/s | 57K · 108K (8-bit) | No price yet |
| AMD Radeon RX 6900 XT | 16 GB | ~74 tok/s | 57K · 108K (8-bit) | No price yet |
| AMD Radeon RX 6950 XT | 16 GB | ~83 tok/s | 57K · 108K (8-bit) | No price yet |
| AMD Radeon RX 7600 XT | 16 GB | ~41 tok/s | 57K · 108K (8-bit) | No price yet |
| AMD Radeon RX 7700 | 16 GB | ~90 tok/s | 57K · 108K (8-bit) | No price yet |
| AMD Radeon RX 7900 GRE | 16 GB | ~83 tok/s | 57K · 108K (8-bit) | No price yet |
| AMD Radeon RX 9060 XT LP | 16 GB | ~46 tok/s | 57K · 108K (8-bit) | No price yet |
| Intel Arc A770 16GB | 16 GB | ~38 tok/s | 57K · 108K (8-bit) | No price yet |
| Intel Arc Pro B50 | 16 GB | ~25 tok/s | 57K · 108K (8-bit) | No price yet |
| NVIDIA GeForce RTX 4060 Ti 16 GB | 16 GB | ~45 tok/s | 57K · 108K (8-bit) | No price yet |
| NVIDIA GeForce RTX 4070 Ti SUPER | 16 GB | ~98 tok/s | 57K · 108K (8-bit) | No price yet |
| NVIDIA GeForce RTX 4080 | 16 GB | ~103 tok/s | 57K · 108K (8-bit) | No price yet |
| NVIDIA GeForce RTX 4080 SUPER | 16 GB | ~106 tok/s | 57K · 108K (8-bit) | No price yet |
| NVIDIA RTX 2000 Ada Generation | 16 GB | ~35 tok/s | 57K · 108K (8-bit) | No price yet |
| NVIDIA RTX A4000 | 16 GB | ~68 tok/s | 57K · 108K (8-bit) | No price yet |
| NVIDIA RTX PRO 2000 Blackwell | 16 GB | ~45 tok/s | 57K · 108K (8-bit) | No price yet |
| NVIDIA T4 | 16 GB | ~47 tok/s | 57K · 108K (8-bit) | No price yet |
| NVIDIA Tesla P100 PCIe 16GB | 16 GB | ~80 tok/s | 57K · 108K (8-bit) | No price yet |
| NVIDIA V100 PCIe 16GB | 16 GB | ~126 tok/s | 57K · 108K (8-bit) | No price yet |
| AMD Radeon RX 7900 XT | 20 GB | ~115 tok/s | 81K · 128K (8-bit) | No price yet |
| NVIDIA RTX 4000 Ada Generation | 20 GB | ~55 tok/s | 81K · 128K (8-bit) | No price yet |
| NVIDIA RTX 4000 SFF Ada Generation | 20 GB | ~44 tok/s | 81K · 128K (8-bit) | No price yet |
| NVIDIA RTX A4500 | 20 GB | ~93 tok/s | 81K · 128K (8-bit) | No price yet |
| Intel Arc Pro B60 | 24 GB | ~50 tok/s | 105K · 128K (8-bit) | No price yet |
| NVIDIA A10 | 24 GB | ~88 tok/s | 105K · 128K (8-bit) | No price yet |
| NVIDIA GeForce RTX 3090 Ti | 24 GB | ~138 tok/s | 105K · 128K (8-bit) | No price yet |
| NVIDIA L4 | 24 GB | ~47 tok/s | 105K · 128K (8-bit) | No price yet |
| NVIDIA Quadro RTX 6000 | 24 GB | ~98 tok/s | 105K · 128K (8-bit) | No price yet |
| NVIDIA RTX 4500 Ada Generation | 24 GB | ~65 tok/s | 105K · 128K (8-bit) | No price yet |
| NVIDIA RTX A5000 | 24 GB | ~110 tok/s | 105K · 128K (8-bit) | No price yet |
| NVIDIA RTX A5500 | 24 GB | ~110 tok/s | 105K · 128K (8-bit) | No price yet |
| NVIDIA RTX PRO 4000 Blackwell | 24 GB | ~98 tok/s | 105K · 128K (8-bit) | No price yet |
| NVIDIA RTX PRO 4000 SFF Blackwell | 24 GB | ~65 tok/s | 105K · 128K (8-bit) | No price yet |
| NVIDIA Tesla P40 | 24 GB | ~40 tok/s | 105K · 128K (8-bit) | No price yet |
| NVIDIA TITAN RTX | 24 GB | ~98 tok/s | 105K · 128K (8-bit) | No price yet |
| AMD Radeon PRO V710 | 28 GB | ~64 tok/s | 128K (full) | No price yet |
| AMD Radeon AI PRO R9600 | 32 GB | ~92 tok/s | 128K (full) | No price yet |
| AMD Radeon AI PRO R9600D | 32 GB | ~92 tok/s | 128K (full) | No price yet |
| AMD Radeon AI PRO R9700 | 32 GB | ~92 tok/s | 128K (full) | No price yet |
| AMD Radeon AI PRO R9700S | 32 GB | ~92 tok/s | 128K (full) | No price yet |
| AMD Radeon PRO V620 | 32 GB | ~74 tok/s | 128K (full) | No price yet |
| AMD Radeon PRO W6800 | 32 GB | ~74 tok/s | 128K (full) | No price yet |
| AMD Radeon PRO W7800 32GB | 32 GB | ~83 tok/s | 128K (full) | No price yet |
| Intel Arc Pro B70 | 32 GB | ~67 tok/s | 128K (full) | No price yet |
| NVIDIA Quadro GV100 | 32 GB | ~122 tok/s | 128K (full) | No price yet |
| NVIDIA RTX 5000 Ada Generation | 32 GB | ~85 tok/s | 128K (full) | No price yet |
| NVIDIA RTX PRO 4500 Blackwell | 32 GB | ~125 tok/s | 128K (full) | No price yet |
| NVIDIA V100 PCIe 32GB | 32 GB | ~126 tok/s | 128K (full) | No price yet |
| NVIDIA V100S PCIe 32GB | 32 GB | ~152 tok/s | 128K (full) | No price yet |
| Apple M5 Pro 48GB | 48 GB (~36 usable) | ~47 tok/s | 128K (full) | No price yet |
| NVIDIA A100 40GB PCIe | 40 GB | ~196 tok/s | 128K (full) | No price yet |
| NVIDIA A800 40GB Active | 40 GB | ~196 tok/s | 128K (full) | No price yet |
| AMD Radeon PRO W7800 48GB | 48 GB | ~124 tok/s | 128K (full) | No price yet |
| AMD Radeon PRO W7900 | 48 GB | ~124 tok/s | 128K (full) | No price yet |
| AMD Radeon PRO W7900 Dual Slot | 48 GB | ~124 tok/s | 128K (full) | No price yet |
| Apple Mac Studio M5 Max (40-core GPU, 64 GB) | 64 GB (~48 usable) | ~86 tok/s | 128K (full) | No price yet |
| NVIDIA A40 | 48 GB | ~101 tok/s | 128K (full) | No price yet |
| NVIDIA L40 | 48 GB | ~121 tok/s | 128K (full) | No price yet |
| NVIDIA L40S | 48 GB | ~121 tok/s | 128K (full) | No price yet |
| NVIDIA Quadro RTX 8000 | 48 GB | ~98 tok/s | 128K (full) | No price yet |
| NVIDIA RTX 6000 Ada Generation | 48 GB | ~133 tok/s | 128K (full) | No price yet |
| NVIDIA RTX PRO 5000 Blackwell 48GB | 48 GB | ~175 tok/s | 128K (full) | No price yet |
| NVIDIA RTX PRO 5000 Blackwell 72GB | 72 GB | ~175 tok/s | 128K (full) | No price yet |
| NVIDIA A100 80GB PCIe | 80 GB | ~230 tok/s | 128K (full) | No price yet |
| NVIDIA RTX PRO 6000 Blackwell Max-Q | 96 GB | ~218 tok/s | 128K (full) | No price yet |
| NVIDIA RTX PRO 6000 Blackwell Server Edition | 96 GB | ~201 tok/s | 128K (full) | No price yet |
| NVIDIA RTX PRO 6000 Blackwell Workstation Edition | 96 GB | ~218 tok/s | 128K (full) | No price yet |
Every device that runs it
Graphics cards, Macs and mini PCs from the wider hardware list behind My rig, smallest usable memory first. Not sure about yours? Check your exact rig.
Show all 204 devices that run it
| Device | Type | Memory | Speed estimate | Max context estimate | Price |
|---|---|---|---|---|---|
| AMD Radeon PRO W6600 | Workstation GPU | 8 GB | ~32 tok/s | 9K · 18K (8-bit) | No price yet |
| AMD Radeon PRO W7400 | Workstation GPU | 8 GB | ~25 tok/s | 9K · 18K (8-bit) | No price yet |
| AMD Radeon PRO W7500 | Workstation GPU | 8 GB | ~37 tok/s | 9K · 18K (8-bit) | No price yet |
| AMD Radeon PRO W7600 | Workstation GPU | 8 GB | ~41 tok/s | 9K · 18K (8-bit) | No price yet |
| AMD Radeon RX 6500 XT 8GB | GPU | 8 GB | ~21 tok/s | 9K · 18K (8-bit) | No price yet |
| AMD Radeon RX 6600 | GPU | 8 GB | ~32 tok/s | 9K · 18K (8-bit) | No price yet |
| AMD Radeon RX 6600 XT | GPU | 8 GB | ~37 tok/s | 9K · 18K (8-bit) | No price yet |
| AMD Radeon RX 6650 XT | GPU | 8 GB | ~40 tok/s | 9K · 18K (8-bit) | No price yet |
| AMD Radeon RX 7400 | GPU | 8 GB | ~25 tok/s | 9K · 18K (8-bit) | No price yet |
| AMD Radeon RX 7600 | GPU | 8 GB | ~41 tok/s | 9K · 18K (8-bit) | No price yet |
| AMD Radeon RX 9050 | GPU | 8 GB | ~41 tok/s | 9K · 18K (8-bit) | No price yet |
| AMD Radeon RX 9060 | GPU | 8 GB | ~41 tok/s | 9K · 18K (8-bit) | No price yet |
| AMD Radeon RX 9060 XT 8GB | GPU | 8 GB | ~46 tok/s | 9K · 18K (8-bit) | No price yet |
| Intel Arc A580 | GPU | 8 GB | ~34 tok/s | 9K · 18K (8-bit) | No price yet |
| Intel Arc A750 | GPU | 8 GB | ~34 tok/s | 9K · 18K (8-bit) | No price yet |
| Intel Arc A770 8GB | GPU | 8 GB | ~34 tok/s | 9K · 18K (8-bit) | No price yet |
| NVIDIA GeForce GTX 1070 | GPU | 8 GB | ~30 tok/s | 9K · 18K (8-bit) | No price yet |
| NVIDIA GeForce GTX 1070 Ti | GPU | 8 GB | ~30 tok/s | 9K · 18K (8-bit) | No price yet |
| NVIDIA GeForce GTX 1080 | GPU | 8 GB | ~37 tok/s | 9K · 18K (8-bit) | No price yet |
| NVIDIA GeForce RTX 2060 SUPER | GPU | 8 GB | ~68 tok/s | 9K · 18K (8-bit) | No price yet |
| NVIDIA GeForce RTX 2070 | GPU | 8 GB | ~68 tok/s | 9K · 18K (8-bit) | No price yet |
| NVIDIA GeForce RTX 2070 SUPER | GPU | 8 GB | ~68 tok/s | 9K · 18K (8-bit) | No price yet |
| NVIDIA GeForce RTX 2080 | GPU | 8 GB | ~68 tok/s | 9K · 18K (8-bit) | No price yet |
| NVIDIA GeForce RTX 2080 SUPER | GPU | 8 GB | ~74 tok/s | 9K · 18K (8-bit) | No price yet |
| NVIDIA GeForce RTX 3070 | GPU | 8 GB | ~68 tok/s | 9K · 18K (8-bit) | No price yet |
| NVIDIA GeForce RTX 3070 Ti | GPU | 8 GB | ~89 tok/s | 9K · 18K (8-bit) | No price yet |
| NVIDIA GeForce RTX 4060 | GPU | 8 GB | ~42 tok/s | 9K · 18K (8-bit) | No price yet |
| NVIDIA GeForce RTX 4060 Ti 8GB | GPU | 8 GB | ~45 tok/s | 9K · 18K (8-bit) | No price yet |
| NVIDIA GeForce RTX 5050 | GPU | 8 GB | ~50 tok/s | 9K · 18K (8-bit) | No price yet |
| NVIDIA GeForce RTX 5060 | GPU | 8 GB | ~68 tok/s | 9K · 18K (8-bit) | low $410 |
| NVIDIA GeForce RTX 5060 Ti 8 GB | GPU | 8 GB | ~68 tok/s | 9K · 18K (8-bit) | low $450 |
| NVIDIA RTX A1000 | Workstation GPU | 8 GB | ~30 tok/s | 9K · 18K (8-bit) | No price yet |
| NVIDIA T1000 8GB | Workstation GPU | 8 GB | ~26 tok/s | 9K · 18K (8-bit) | No price yet |
| AMD Radeon RX 6700 | GPU | 10 GB | ~46 tok/s | 21K · 40K (8-bit) | No price yet |
| Intel Arc B570 | GPU | 10 GB | ~42 tok/s | 21K · 40K (8-bit) | No price yet |
| NVIDIA GeForce RTX 3080 | GPU | 10 GB | ~109 tok/s | 21K · 40K (8-bit) | No price yet |
| Apple M1 Pro 16GB | Mac | 16 GB (~10.7 usable) | ~31 tok/s | 25K · 48K (8-bit) | No price yet |
| Apple M2 16GB | Mac | 16 GB (~10.7 usable) | ~16 tok/s | 25K · 48K (8-bit) | No price yet |
| Apple M2 Pro 16GB | Mac | 16 GB (~10.7 usable) | ~31 tok/s | 25K · 48K (8-bit) | No price yet |
| Apple M3 16GB | Mac | 16 GB (~10.7 usable) | ~16 tok/s | 25K · 48K (8-bit) | No price yet |
| Apple M4 16GB | Mac | 16 GB (~10.7 usable) | ~19 tok/s | 25K · 48K (8-bit) | No price yet |
| Apple M5 16GB | Mac | 16 GB (~10.7 usable) | ~24 tok/s | 25K · 48K (8-bit) | No price yet |
| Apple M6 16GB | Mac | 16 GB (~10.7 usable) | ~24 tok/s | 25K · 48K (8-bit) | No price yet |
| NVIDIA GeForce GTX 1080 Ti | GPU | 11 GB | ~55 tok/s | 27K · 52K (8-bit) | No price yet |
| NVIDIA GeForce RTX 2080 Ti | GPU | 11 GB | ~90 tok/s | 27K · 52K (8-bit) | No price yet |
| AMD Radeon RX 6700 XT | GPU | 12 GB | ~55 tok/s | 33K · 63K (8-bit) | No price yet |
| AMD Radeon RX 6750 XT | GPU | 12 GB | ~62 tok/s | 33K · 63K (8-bit) | No price yet |
| AMD Radeon RX 7700 XT | GPU | 12 GB | ~62 tok/s | 33K · 63K (8-bit) | No price yet |
| AMD Radeon RX 9070 GRE | GPU | 12 GB | ~62 tok/s | 33K · 63K (8-bit) | No price yet |
| Intel Arc B580 | GPU | 12 GB | ~50 tok/s | 33K · 63K (8-bit) | low $330 |
| Intel Arc Pro A60 | Workstation GPU | 12 GB | ~26 tok/s | 33K · 63K (8-bit) | No price yet |
| NVIDIA GeForce RTX 3060 12 GB | GPU | 12 GB | ~55 tok/s | 33K · 63K (8-bit) | low $305 |
| NVIDIA GeForce RTX 3080 Ti | GPU | 12 GB | ~127 tok/s | 33K · 63K (8-bit) | No price yet |
| NVIDIA GeForce RTX 4070 | GPU | 12 GB | ~75 tok/s | 33K · 63K (8-bit) | No price yet |
| NVIDIA GeForce RTX 4070 SUPER | GPU | 12 GB | ~75 tok/s | 33K · 63K (8-bit) | low $620 |
| NVIDIA GeForce RTX 4070 Ti | GPU | 12 GB | ~75 tok/s | 33K · 63K (8-bit) | No price yet |
| NVIDIA GeForce RTX 5070 | GPU | 12 GB | ~98 tok/s | 33K · 63K (8-bit) | low $800 |
| NVIDIA RTX A2000 12GB | Workstation GPU | 12 GB | ~45 tok/s | 33K · 63K (8-bit) | No price yet |
| NVIDIA Tesla P100 PCIe 12GB | Data center GPU | 12 GB | ~62 tok/s | 33K · 63K (8-bit) | No price yet |
| NVIDIA TITAN Xp | GPU | 12 GB | ~62 tok/s | 33K · 63K (8-bit) | No price yet |
| Apple M3 Pro 18GB | Mac | 18 GB (~12.1 usable) | ~24 tok/s | 34K · 64K (8-bit) | No price yet |
| AMD Radeon PRO W7700 | Workstation GPU | 16 GB | ~83 tok/s | 57K · 108K (8-bit) | No price yet |
| AMD Radeon RX 6800 | GPU | 16 GB | ~74 tok/s | 57K · 108K (8-bit) | No price yet |
| AMD Radeon RX 6800 XT | GPU | 16 GB | ~74 tok/s | 57K · 108K (8-bit) | No price yet |
| AMD Radeon RX 6900 XT | GPU | 16 GB | ~74 tok/s | 57K · 108K (8-bit) | No price yet |
| AMD Radeon RX 6950 XT | GPU | 16 GB | ~83 tok/s | 57K · 108K (8-bit) | No price yet |
| AMD Radeon RX 7600 XT | GPU | 16 GB | ~41 tok/s | 57K · 108K (8-bit) | No price yet |
| AMD Radeon RX 7700 | GPU | 16 GB | ~90 tok/s | 57K · 108K (8-bit) | No price yet |
| AMD Radeon RX 7800 XT | GPU | 16 GB | ~90 tok/s | 57K · 108K (8-bit) | low $599 |
| AMD Radeon RX 7900 GRE | GPU | 16 GB | ~83 tok/s | 57K · 108K (8-bit) | No price yet |
| AMD Radeon RX 9060 XT 16 GB | GPU | 16 GB | ~46 tok/s | 57K · 108K (8-bit) | low $620 |
| AMD Radeon RX 9060 XT LP | GPU | 16 GB | ~46 tok/s | 57K · 108K (8-bit) | No price yet |
| AMD Radeon RX 9070 | GPU | 16 GB | ~92 tok/s | 57K · 108K (8-bit) | low $570 |
| AMD Radeon RX 9070 XT | GPU | 16 GB | ~92 tok/s | 57K · 108K (8-bit) | low $810 |
| Intel Arc A770 16GB | GPU | 16 GB | ~38 tok/s | 57K · 108K (8-bit) | No price yet |
| Intel Arc Pro B50 | Workstation GPU | 16 GB | ~25 tok/s | 57K · 108K (8-bit) | No price yet |
| NVIDIA GeForce RTX 4060 Ti 16 GB | GPU | 16 GB | ~45 tok/s | 57K · 108K (8-bit) | No price yet |
| NVIDIA GeForce RTX 4070 Ti SUPER | GPU | 16 GB | ~98 tok/s | 57K · 108K (8-bit) | No price yet |
| NVIDIA GeForce RTX 4080 | GPU | 16 GB | ~103 tok/s | 57K · 108K (8-bit) | No price yet |
| NVIDIA GeForce RTX 4080 SUPER | GPU | 16 GB | ~106 tok/s | 57K · 108K (8-bit) | No price yet |
| NVIDIA GeForce RTX 5060 Ti 16 GB | GPU | 16 GB | ~68 tok/s | 57K · 108K (8-bit) | low $780 |
| NVIDIA GeForce RTX 5070 Ti | GPU | 16 GB | ~125 tok/s | 57K · 108K (8-bit) | low $1,120 |
| NVIDIA GeForce RTX 5080 | GPU | 16 GB | ~133 tok/s | 57K · 108K (8-bit) | low $1,600 |
| NVIDIA RTX 2000 Ada Generation | Workstation GPU | 16 GB | ~35 tok/s | 57K · 108K (8-bit) | No price yet |
| NVIDIA RTX A4000 | Workstation GPU | 16 GB | ~68 tok/s | 57K · 108K (8-bit) | No price yet |
| NVIDIA RTX PRO 2000 Blackwell | Workstation GPU | 16 GB | ~45 tok/s | 57K · 108K (8-bit) | No price yet |
| NVIDIA T4 | Data center GPU | 16 GB | ~47 tok/s | 57K · 108K (8-bit) | No price yet |
| NVIDIA Tesla P100 PCIe 16GB | Data center GPU | 16 GB | ~80 tok/s | 57K · 108K (8-bit) | No price yet |
| NVIDIA V100 PCIe 16GB | Data center GPU | 16 GB | ~126 tok/s | 57K · 108K (8-bit) | No price yet |
| Apple M2 24GB | Mac | 24 GB (~16.1 usable) | ~16 tok/s | 58K · 109K (8-bit) | No price yet |
| Apple M3 24GB | Mac | 24 GB (~16.1 usable) | ~16 tok/s | 58K · 109K (8-bit) | No price yet |
| Apple M4 24GB | Mac | 24 GB (~16.1 usable) | ~19 tok/s | 58K · 109K (8-bit) | No price yet |
| Apple M4 Pro 24GB | Mac | 24 GB (~16.1 usable) | ~42 tok/s | 58K · 109K (8-bit) | No price yet |
| Apple M5 24GB | Mac | 24 GB (~16.1 usable) | ~24 tok/s | 58K · 109K (8-bit) | No price yet |
| Apple M5 Pro 24GB | Mac | 24 GB (~16.1 usable) | ~47 tok/s | 58K · 109K (8-bit) | No price yet |
| Apple M6 24GB | Mac | 24 GB (~16.1 usable) | ~27 tok/s | 58K · 109K (8-bit) | No price yet |
| AMD Radeon RX 7900 XT | GPU | 20 GB | ~115 tok/s | 81K · 128K (8-bit) | No price yet |
| NVIDIA RTX 4000 Ada Generation | Workstation GPU | 20 GB | ~55 tok/s | 81K · 128K (8-bit) | No price yet |
| NVIDIA RTX 4000 SFF Ada Generation | Workstation GPU | 20 GB | ~44 tok/s | 81K · 128K (8-bit) | No price yet |
| NVIDIA RTX A4500 | Workstation GPU | 20 GB | ~93 tok/s | 81K · 128K (8-bit) | No price yet |
| Apple M1 Max 32GB | Mac | 32 GB (~21.4 usable) | ~59 tok/s | 89K · 128K (8-bit) | No price yet |
| Apple M1 Pro 32GB | Mac | 32 GB (~21.4 usable) | ~31 tok/s | 89K · 128K (8-bit) | No price yet |
| Apple M2 Max (30-core GPU) 32GB | Mac | 32 GB (~21.4 usable) | ~59 tok/s | 89K · 128K (8-bit) | No price yet |
| Apple M2 Max (38-core GPU) 32GB | Mac | 32 GB (~21.4 usable) | ~59 tok/s | 89K · 128K (8-bit) | No price yet |
| Apple M2 Pro 32GB | Mac | 32 GB (~21.4 usable) | ~31 tok/s | 89K · 128K (8-bit) | No price yet |
| Apple M4 32GB | Mac | 32 GB (~21.4 usable) | ~19 tok/s | 89K · 128K (8-bit) | No price yet |
| Apple M5 32GB | Mac | 32 GB (~21.4 usable) | ~24 tok/s | 89K · 128K (8-bit) | No price yet |
| Apple M6 32GB | Mac | 32 GB (~21.4 usable) | ~27 tok/s | 89K · 128K (8-bit) | No price yet |
| AMD Radeon RX 7900 XTX | GPU | 24 GB | ~138 tok/s | 105K · 128K (8-bit) | low $1,350 |
| Intel Arc Pro B60 | Workstation GPU | 24 GB | ~50 tok/s | 105K · 128K (8-bit) | No price yet |
| NVIDIA A10 | Data center GPU | 24 GB | ~88 tok/s | 105K · 128K (8-bit) | No price yet |
| NVIDIA GeForce RTX 3090 | GPU | 24 GB | ~130 tok/s | 105K · 128K (8-bit) | low $1,499 |
| NVIDIA GeForce RTX 3090 Ti | GPU | 24 GB | ~138 tok/s | 105K · 128K (8-bit) | No price yet |
| NVIDIA GeForce RTX 4090 | GPU | 24 GB | ~138 tok/s | 105K · 128K (8-bit) | low $2,910 |
| NVIDIA L4 | Data center GPU | 24 GB | ~47 tok/s | 105K · 128K (8-bit) | No price yet |
| NVIDIA Quadro RTX 6000 | Workstation GPU | 24 GB | ~98 tok/s | 105K · 128K (8-bit) | No price yet |
| NVIDIA RTX 4500 Ada Generation | Workstation GPU | 24 GB | ~65 tok/s | 105K · 128K (8-bit) | No price yet |
| NVIDIA RTX A5000 | Workstation GPU | 24 GB | ~110 tok/s | 105K · 128K (8-bit) | No price yet |
| NVIDIA RTX A5500 | Workstation GPU | 24 GB | ~110 tok/s | 105K · 128K (8-bit) | No price yet |
| NVIDIA RTX PRO 4000 Blackwell | Workstation GPU | 24 GB | ~98 tok/s | 105K · 128K (8-bit) | No price yet |
| NVIDIA RTX PRO 4000 SFF Blackwell | Workstation GPU | 24 GB | ~65 tok/s | 105K · 128K (8-bit) | No price yet |
| NVIDIA RTX Spark N1X laptop (6144-core GPU, 24GB) | Laptop | 24 GB | ~47 tok/s | 105K · 128K (8-bit) | No price yet |
| NVIDIA Tesla P40 | Data center GPU | 24 GB | ~40 tok/s | 105K · 128K (8-bit) | No price yet |
| NVIDIA TITAN RTX | GPU | 24 GB | ~98 tok/s | 105K · 128K (8-bit) | No price yet |
| Apple M3 Max (30-core GPU) 36GB | Mac | 36 GB (~24.1 usable) | ~46 tok/s | 105K · 128K (8-bit) | No price yet |
| Apple M3 Pro 36GB | Mac | 36 GB (~24.1 usable) | ~24 tok/s | 105K · 128K (8-bit) | No price yet |
| Apple M4 Max (32-core GPU) 36GB | Mac | 36 GB (~24.1 usable) | ~60 tok/s | 105K · 128K (8-bit) | No price yet |
| Apple Mac Studio M5 Max (32-core GPU, 36 GB) | Mac | 36 GB (~24.1 usable) | ~67 tok/s | 105K · 128K (8-bit) | low $2,499 |
| AMD Radeon PRO V710 | Data center GPU | 28 GB | ~64 tok/s | 128K (full) | No price yet |
| AMD Radeon AI PRO R9600 | Workstation GPU | 32 GB | ~92 tok/s | 128K (full) | No price yet |
| AMD Radeon AI PRO R9600D | Workstation GPU | 32 GB | ~92 tok/s | 128K (full) | No price yet |
| AMD Radeon AI PRO R9700 | Workstation GPU | 32 GB | ~92 tok/s | 128K (full) | No price yet |
| AMD Radeon AI PRO R9700S | Workstation GPU | 32 GB | ~92 tok/s | 128K (full) | No price yet |
| AMD Radeon PRO V620 | Data center GPU | 32 GB | ~74 tok/s | 128K (full) | No price yet |
| AMD Radeon PRO W6800 | Workstation GPU | 32 GB | ~74 tok/s | 128K (full) | No price yet |
| AMD Radeon PRO W7800 32GB | Workstation GPU | 32 GB | ~83 tok/s | 128K (full) | No price yet |
| AMD Ryzen AI Max 385 (32GB) | Mini PC | 32 GB | ~37 tok/s | 128K (full) | No price yet |
| AMD Ryzen AI Max 390 (32GB) | Mini PC | 32 GB | ~37 tok/s | 128K (full) | No price yet |
| AMD Ryzen AI Max+ 395 (32GB) | Mini PC | 32 GB | ~37 tok/s | 128K (full) | No price yet |
| Intel Arc Pro B70 | Workstation GPU | 32 GB | ~67 tok/s | 128K (full) | No price yet |
| NVIDIA GeForce RTX 5090 | GPU | 32 GB | ~218 tok/s | 128K (full) | low $6,500 |
| NVIDIA Quadro GV100 | Workstation GPU | 32 GB | ~122 tok/s | 128K (full) | No price yet |
| NVIDIA RTX 5000 Ada Generation | Workstation GPU | 32 GB | ~85 tok/s | 128K (full) | No price yet |
| NVIDIA RTX PRO 4500 Blackwell | Workstation GPU | 32 GB | ~125 tok/s | 128K (full) | No price yet |
| NVIDIA RTX Spark N1X laptop (6144-core GPU, 32GB) | Laptop | 32 GB | ~47 tok/s | 128K (full) | No price yet |
| NVIDIA V100 PCIe 32GB | Data center GPU | 32 GB | ~126 tok/s | 128K (full) | No price yet |
| NVIDIA V100S PCIe 32GB | Data center GPU | 32 GB | ~152 tok/s | 128K (full) | No price yet |
| Apple M3 Max (40-core GPU) 48GB | Mac | 48 GB (~36 usable) | ~59 tok/s | 128K (full) | No price yet |
| Apple M4 Max (40-core GPU) 48GB | Mac | 48 GB (~36 usable) | ~77 tok/s | 128K (full) | No price yet |
| Apple M4 Pro 48GB | Mac | 48 GB (~36 usable) | ~42 tok/s | 128K (full) | No price yet |
| Apple M5 Max (40-core GPU) 48GB | Mac | 48 GB (~36 usable) | ~86 tok/s | 128K (full) | No price yet |
| Apple M5 Pro 48GB | Mac | 48 GB (~36 usable) | ~47 tok/s | 128K (full) | No price yet |
| NVIDIA A100 40GB PCIe | Data center GPU | 40 GB | ~196 tok/s | 128K (full) | No price yet |
| NVIDIA A800 40GB Active | Workstation GPU | 40 GB | ~196 tok/s | 128K (full) | No price yet |
| AMD Radeon PRO W7800 48GB | Workstation GPU | 48 GB | ~124 tok/s | 128K (full) | No price yet |
| AMD Radeon PRO W7900 | Workstation GPU | 48 GB | ~124 tok/s | 128K (full) | No price yet |
| AMD Radeon PRO W7900 Dual Slot | Workstation GPU | 48 GB | ~124 tok/s | 128K (full) | No price yet |
| Apple M1 Max 64GB | Mac | 64 GB (~48 usable) | ~59 tok/s | 128K (full) | No price yet |
| Apple M1 Ultra 64GB | Mac | 64 GB (~48 usable) | ~106 tok/s | 128K (full) | No price yet |
| Apple M2 Max (30-core GPU) 64GB | Mac | 64 GB (~48 usable) | ~59 tok/s | 128K (full) | No price yet |
| Apple M2 Max (38-core GPU) 64GB | Mac | 64 GB (~48 usable) | ~59 tok/s | 128K (full) | No price yet |
| Apple M2 Ultra 64GB | Mac | 64 GB (~48 usable) | ~106 tok/s | 128K (full) | No price yet |
| Apple M3 Max (40-core GPU) 64GB | Mac | 64 GB (~48 usable) | ~59 tok/s | 128K (full) | No price yet |
| Apple M4 Max (40-core GPU) 64GB | Mac | 64 GB (~48 usable) | ~77 tok/s | 128K (full) | No price yet |
| Apple M4 Pro 64GB | Mac | 64 GB (~48 usable) | ~42 tok/s | 128K (full) | No price yet |
| Apple M5 Pro 64GB | Mac | 64 GB (~48 usable) | ~47 tok/s | 128K (full) | No price yet |
| Apple Mac Studio M5 Max (40-core GPU, 64 GB) | Mac | 64 GB (~48 usable) | ~86 tok/s | 128K (full) | No price yet |
| NVIDIA A40 | Data center GPU | 48 GB | ~101 tok/s | 128K (full) | No price yet |
| NVIDIA L40 | Data center GPU | 48 GB | ~121 tok/s | 128K (full) | No price yet |
| NVIDIA L40S | Data center GPU | 48 GB | ~121 tok/s | 128K (full) | No price yet |
| NVIDIA Quadro RTX 8000 | Workstation GPU | 48 GB | ~98 tok/s | 128K (full) | No price yet |
| NVIDIA RTX 6000 Ada Generation | Workstation GPU | 48 GB | ~133 tok/s | 128K (full) | No price yet |
| NVIDIA RTX A6000 | Workstation GPU | 48 GB | ~110 tok/s | 128K (full) | low $8,363 |
| NVIDIA RTX PRO 5000 Blackwell 48GB | Workstation GPU | 48 GB | ~175 tok/s | 128K (full) | No price yet |
| NVIDIA RTX Spark N1X laptop (6144-core GPU, 48GB) | Laptop | 48 GB | ~47 tok/s | 128K (full) | No price yet |
| AMD Ryzen AI Max 385 (64GB) | Mini PC | 64 GB | ~37 tok/s | 128K (full) | No price yet |
| AMD Ryzen AI Max 390 (64GB) | Mini PC | 64 GB | ~37 tok/s | 128K (full) | No price yet |
| AMD Ryzen AI Max+ 395 (64GB) | Mini PC | 64 GB | ~37 tok/s | 128K (full) | low $2,199.99 |
| NVIDIA DGX Spark (64GB) | Mini PC | 64 GB | ~43 tok/s | 128K (full) | low $4,999 |
| NVIDIA RTX Spark N1X laptop (6144-core GPU, 64GB) | Laptop | 64 GB | ~47 tok/s | 128K (full) | No price yet |
| Apple M2 Max (38-core GPU) 96GB | Mac | 96 GB (~72 usable) | ~59 tok/s | 128K (full) | No price yet |
| Apple M3 Max (30-core GPU) 96GB | Mac | 96 GB (~72 usable) | ~46 tok/s | 128K (full) | No price yet |
| Apple M3 Ultra 96GB | Mac | 96 GB (~72 usable) | ~108 tok/s | 128K (full) | No price yet |
| Apple Mac Studio M5 Ultra (64-core GPU, 96 GB) | Mac | 96 GB (~72 usable) | ~145 tok/s | 128K (full) | low $5,499 |
| NVIDIA RTX PRO 5000 Blackwell 72GB | Workstation GPU | 72 GB | ~175 tok/s | 128K (full) | No price yet |
| NVIDIA A100 80GB PCIe | Data center GPU | 80 GB | ~230 tok/s | 128K (full) | No price yet |
| Apple M1 Ultra 128GB | Mac | 128 GB (~96 usable) | ~106 tok/s | 128K (full) | No price yet |
| Apple M2 Ultra 128GB | Mac | 128 GB (~96 usable) | ~106 tok/s | 128K (full) | No price yet |
| Apple M3 Max (40-core GPU) 128GB | Mac | 128 GB (~96 usable) | ~59 tok/s | 128K (full) | No price yet |
| Apple M4 Max (40-core GPU) 128GB | Mac | 128 GB (~96 usable) | ~77 tok/s | 128K (full) | No price yet |
| Apple M5 Max (40-core GPU) 128GB | Mac | 128 GB (~96 usable) | ~86 tok/s | 128K (full) | No price yet |
| NVIDIA RTX PRO 6000 Blackwell Max-Q | Workstation GPU | 96 GB | ~218 tok/s | 128K (full) | No price yet |
| NVIDIA RTX PRO 6000 Blackwell Server Edition | Data center GPU | 96 GB | ~201 tok/s | 128K (full) | No price yet |
| NVIDIA RTX PRO 6000 Blackwell Workstation Edition | Workstation GPU | 96 GB | ~218 tok/s | 128K (full) | No price yet |
| AMD Ryzen AI Max 385 (128GB) | Mini PC | 128 GB | ~37 tok/s | 128K (full) | No price yet |
| AMD Ryzen AI Max 390 (128GB) | Mini PC | 128 GB | ~37 tok/s | 128K (full) | No price yet |
| AMD Ryzen AI Max+ 395 (128GB) | Mini PC | 128 GB | ~37 tok/s | 128K (full) | low $3,499.99 |
| NVIDIA DGX Spark (128GB) | Mini PC | 128 GB | ~43 tok/s | 128K (full) | low $6,949.99 |
| NVIDIA RTX Spark N1X laptop (6144-core GPU, 128GB) | Laptop | 128 GB | ~47 tok/s | 128K (full) | No price yet |
| Apple M2 Ultra 192GB | Mac | 192 GB (~144 usable) | ~106 tok/s | 128K (full) | No price yet |
| Apple M3 Ultra 256GB | Mac | 256 GB (~192 usable) | ~108 tok/s | 128K (full) | No price yet |
| Apple M5 Ultra 256GB | Mac | 256 GB (~192 usable) | ~145 tok/s | 128K (full) | No price yet |
| Framework Desktop, AMD Ryzen AI Max+ PRO 495 (Strix Halo), 192 GB | Mini PC | 192 GB | ~39 tok/s | 128K (full) | low $6,799 |
| Apple M5 Ultra 512GB | Mac | 512 GB (~384 usable) | ~145 tok/s | 128K (full) | No price yet |
Speed calculator
estimateEstimate = bandwidth ÷ GB read per token (the model's listed GGUF file: Q4_K_M for most models, otherwise the quant named in the list; scaled by active/total parameters for MoE), scaled by a per-platform efficiency and fixed per-token costs calibrated to 45 measured llama.cpp benchmarks (2025–26). Fit = file size + 1.5 GB headroom ≤ total VRAM (on Macs, the part of unified memory macOS lets the GPU use by default), and room for at least 4K tokens of context with an 8-bit KV cache. For a model with no KV cache data, only the file and headroom are checked, so a fit says nothing about context. Multi-GPU: VRAM adds, speed stays one card's (a layer split runs the cards in turn); max context uses the total. Speed is for short context. Max context = what is left after the file and a 1 GB runtime reserve ÷ KV cache bytes per token, default FP16 and 8-bit (q8_0) cache. How speed works · how context works
Related
- BuildBuild that fits it: 16 GB starter: Radeon RX 9070
- BuildBuild that fits it: 16 GB NVIDIA: RTX 5060 Ti 16 GB
- BuildBuild that fits it: 24 GB: one used RTX 3090
- BuildBuild that fits it: 32 GB: two Radeon RX 9070
- BuildBuild that fits it: 48 GB: dual used RTX 3090
- Base modelIBM Granite 4.2 8B, the Apache 2.0 base model TwIL-8B is trained from
- CatalogEvery model in the catalog