What GPU do you need to run Xing4.0-29B-A4B?
A new Radeon RX 7900 XTX ($1,350). It is the cheapest priced graphics card that holds Xing4.0-29B-A4B (about 21.6 GB at Q4) with room for about 62K tokens of context (117K with 8-bit KV cache), at about 170 tok/s estimate; that is the card price only, and it requires a separate compatible PC. 54 of the 157 GPUs and systems we track fit it.
Parameters31.22B4B active (mixture-of-experts)
VRAM needed~21.6 GBfile + headroom
Single GPUs that fit54
Context window256K tokens
31.22B parameters, 4B active (mixture-of-experts) (Hugging Face, accessed 2026-11-13). Q4_K_M GGUF: 20.06 GB (source). Architecture config (Hugging Face config.json): 40 attention layers caching one 576-wide compressed latent (MLA), so about 45 KB of FP16 KV cache per token of context (how we estimate context).
China Telecom's agentic mixture-of-experts model, from its XingChen AGI team (the Xing series was formerly TeleChat): 29B total and 4B active per the card, with multi-head latent attention (MLA) and one MTP layer. The checkpoint holds 31.2B parameters; the card's 29B is its own rounding or count. It is new on this site because mainline llama.cpp added support on Nov 5, 2026, and tagged release v1.0.0 includes it. The official GGUF repo has no Q4_K_M, so the size is bartowski's Q4_K_M (XingChen-AGI_Xing4.0-29B-A4B-Q4_K_M.gguf, 20,056,603,584 bytes); every size is in the table below. Context is the card's 256K native window. Release date is the Hugging Face repo creation date.
What we checked
Checked Nov 13, 2026. Official is the maker or the weight files. A vendor claim is that project's own documentation. A community claim is someone else's upload. Our check is a file or search we read. Unknown means we have no source, so it is not estimated. None of these rows is a speed we measured.
| Kind | What | Detail | Source |
|---|---|---|---|
| Official | Model card | The card says the model was "developed by China Telecom Artificial Intelligence Technology Co., Ltd." (the Xing series was formerly TeleChat) and gives "29B total / 4B active" and a 256K native context, "extensible to 512K". config.json (model_type xing4_0): 40 layers with MLA (kv_lora_rank 512, qk_rope_head_dim 64), 64 routed experts with 4 per token plus 1 shared expert, the first 2 layers dense, 1 MTP (nextn) layer, and mHC (hc_mult 4). Hugging Face's safetensors total is 31,215,031,088 parameters (31,215,028,592 BF16 and 2,496 F32). The card's 29B is its own rounding or count; the checkpoint holds 31.2B. Transformers needs the repo's own model code. | Model card, checked Nov 13, 2026 |
| Official | License | Apache 2.0, in the repo's LICENSE file. This is a summary of the license, not legal advice. | LICENSE, checked Nov 13, 2026 |
| Official | Weight files | 12 BF16 safetensors files sum to 62,431,093,616 bytes (62.43 GB) at commit acbdebfe496d117e6fde3ce152d2450fb0d5fa53. These are downloads, not the memory a runtime needs. | Hugging Face API, checked Nov 13, 2026 |
| Official | Official GGUF files | XingChen-AGI/Xing4.0-29B-A4B-GGUF (commit 006904bc1d05ae2c37f938a4abdffee27e488544) has iq2_M-slim, iq3_xxs, iq3_m, iq4_xs, iq4_nl, q5_0, q6_k and q8_0, spelled as the repo spells them, and no Q4_K_M. It also has a second file named IQ4_NL (20,104,012,544 bytes) beside iq4_nl (19,198,485,216 bytes); both are in the table below. | Official GGUF repo, checked Nov 13, 2026 |
| Community claim | bartowski GGUFs | bartowski/XingChen-AGI_Xing4.0-29B-A4B-GGUF (commit d324547035a19a4254afa224913a4287fb0dc0ee) was made with llama.cpp b12174, and the card says the MTP layers are kept for draft-mtp. The fits on this page use its Q4_K_M (XingChen-AGI_Xing4.0-29B-A4B-Q4_K_M.gguf, 20,056,603,584 bytes). Its IQ4_XS and Q4_K_L are in the table below. | bartowski GGUF card, checked Nov 13, 2026 |
| Our check | llama.cpp | Pull request #31589, "Add support for the xing4_0", merged on 2026-11-05 (build b12174 and later), and it is in tagged release v1.0.0 (2026-11-13). The maker's card, as of 2026-11-10, says "Stock build b12174+ on CPU / CUDA" and does not mention Metal or a Mac. We have not verified it on a Mac, and we have not checked Ollama or LM Studio. A loader is not a confirmed setup or a speed on this site. | llama.cpp PR #31589, checked Nov 13, 2026 |
| Vendor claim | MTP drafting | The checkpoint has 1 MTP layer, and the GGUFs keep it, so llama.cpp can draft tokens with --spec-type draft-mtp. That is speculative decoding, not a speed figure, and the estimates on this page do not assume it. | Model card, checked Nov 13, 2026 |
| Vendor claim | SGLang and vLLM | SGLang mainline has supported it since 2026-11-05 (#39793), and vLLM runs it through an official out-of-tree plugin. Those are server setups; we have not run either. | Model card, checked Nov 13, 2026 |
| Vendor claim | Benchmark scores (XingChen's) | The card reports SWE-bench Verified 75.00 and Terminal-Bench 2.1 57.50. These are XingChen's own reported scores, not checked by us. | Model card, checked Nov 13, 2026 |
| Community claim | MLX (unofficial) | Stock mlx-lm has no xing4_0 class. mlx-community/Xing4.0-29B-A4B-OptiQ-4bit (20,579,605,322 bytes) says: "Stock mlx-lm has no xing4_0 class — support is an unmerged pull request". That repo works through the mlx-optiq package's vendored port, so MLX support is unofficial for now. | mlx-community card, checked Nov 13, 2026 |
GGUF files and sizes
Exact sizes from Hugging Face, accessed 2026-11-13. The fit and context tables on this page use the Q4_K_M file (20.06 GB); a smaller file needs less memory by the difference. VRAM needed is the file + the same headroom.
| File | Published by | Bytes | Size | VRAM needed | Source |
|---|---|---|---|---|---|
| iq2_M-slim | XingChen-AGI (official) | 11,800,813,760 | 11.8 GB | ~13.3 GB | Hugging Face |
| iq3_xxs | XingChen-AGI (official) | 13,989,927,648 | 13.99 GB | ~15.5 GB | Hugging Face |
| iq3_m | XingChen-AGI (official) | 15,446,399,712 | 15.45 GB | ~16.9 GB | Hugging Face |
| IQ4_XS | bartowski (community) | 17,982,561,216 | 17.98 GB | ~19.5 GB | Hugging Face |
| iq4_xs | XingChen-AGI (official) | 18,272,862,944 | 18.27 GB | ~19.8 GB | Hugging Face |
| iq4_nl | XingChen-AGI (official) | 19,198,485,216 | 19.2 GB | ~20.7 GB | Hugging Face |
| Q4_K_M | bartowski (community) | 20,056,603,584 | 20.06 GB | ~21.6 GB | XingChen-AGI_Xing4.0-29B-A4B-Q4_K_M.gguf |
| IQ4_NL | XingChen-AGI (official) | 20,104,012,544 | 20.1 GB | ~21.6 GB | Hugging Face |
| q5_0 | XingChen-AGI (official) | 20,416,753,312 | 20.42 GB | ~21.9 GB | Hugging Face |
| Q4_K_L | bartowski (community) | 21,686,090,688 | 21.69 GB | ~23.2 GB | Hugging Face |
| q6_k | XingChen-AGI (official) | 26,876,812,000 | 26.88 GB | ~28.4 GB | Hugging Face |
| q8_0 | XingChen-AGI (official) | 34,164,155,104 | 34.16 GB | ~35.7 GB | Hugging Face |
Other places to get it
Checked Nov 14, 2026. We list only copies we opened and checked. Check file hashes for copies from other uploaders.
- ModelScope: Lab's own account
Single GPUs that fit, cheapest first
| Single GPU that fits | VRAM | Speed estimate | Max context estimate | Price |
|---|---|---|---|---|
| AMD Radeon RX 7900 XTX | 24 GB | ~170 tok/s | 62K · 117K (8-bit) | low $1,350 |
| NVIDIA GeForce RTX 3090 | 24 GB | ~207 tok/s | 62K · 117K (8-bit) | low $1,499 |
| Apple Mac Studio M5 Max (32-core GPU, 36 GB) | 36 GB (~24.1 usable) | ~88 tok/s | 64K · 121K (8-bit) | low $2,499 |
| NVIDIA GeForce RTX 4090 | 24 GB | ~218 tok/s | 62K · 117K (8-bit) | low $2,910 |
| NVIDIA DGX Spark (64GB) | 64 GB | ~81 tok/s | 256K (full) | low $4,999 |
| Apple Mac Studio M5 Ultra (64-core GPU, 96 GB) | 96 GB (~72 usable) | ~133 tok/s | 256K (full) | low $5,499 |
| NVIDIA GeForce RTX 5090 | 32 GB | ~300 tok/s | 231K · 256K (8-bit) | low $6,500 |
| Framework Desktop, AMD Ryzen AI Max+ PRO 495 (Strix Halo), 192 GB | 192 GB | ~68 tok/s | 256K (full) | low $6,799 |
| NVIDIA DGX Spark (128GB) | 128 GB | ~81 tok/s | 256K (full) | low $6,949.99 |
| NVIDIA RTX A6000 | 48 GB | ~182 tok/s | 256K (full) | low $8,363 |
| Intel Arc Pro B60 | 24 GB | ~84 tok/s | 62K · 117K (8-bit) | No price yet |
| NVIDIA A10 | 24 GB | ~152 tok/s | 62K · 117K (8-bit) | No price yet |
| NVIDIA GeForce RTX 3090 Ti | 24 GB | ~218 tok/s | 62K · 117K (8-bit) | No price yet |
| NVIDIA L4 | 24 GB | ~87 tok/s | 62K · 117K (8-bit) | No price yet |
| NVIDIA Quadro RTX 6000 | 24 GB | ~165 tok/s | 62K · 117K (8-bit) | No price yet |
| NVIDIA RTX 4500 Ada Generation | 24 GB | ~118 tok/s | 62K · 117K (8-bit) | No price yet |
| NVIDIA RTX A5000 | 24 GB | ~182 tok/s | 62K · 117K (8-bit) | No price yet |
| NVIDIA RTX A5500 | 24 GB | ~182 tok/s | 62K · 117K (8-bit) | No price yet |
| NVIDIA RTX PRO 4000 Blackwell | 24 GB | ~165 tok/s | 62K · 117K (8-bit) | No price yet |
| NVIDIA RTX PRO 4000 SFF Blackwell | 24 GB | ~118 tok/s | 62K · 117K (8-bit) | No price yet |
| NVIDIA Tesla P40 | 24 GB | ~76 tok/s | 62K · 117K (8-bit) | No price yet |
| NVIDIA TITAN RTX | 24 GB | ~165 tok/s | 62K · 117K (8-bit) | No price yet |
| AMD Radeon PRO V710 | 28 GB | ~102 tok/s | 147K · 256K (8-bit) | No price yet |
| AMD Radeon AI PRO R9600 | 32 GB | ~131 tok/s | 231K · 256K (8-bit) | No price yet |
| AMD Radeon AI PRO R9600D | 32 GB | ~131 tok/s | 231K · 256K (8-bit) | No price yet |
| AMD Radeon AI PRO R9700 | 32 GB | ~131 tok/s | 231K · 256K (8-bit) | No price yet |
| AMD Radeon AI PRO R9700S | 32 GB | ~131 tok/s | 231K · 256K (8-bit) | No price yet |
| AMD Radeon PRO V620 | 32 GB | ~112 tok/s | 231K · 256K (8-bit) | No price yet |
| AMD Radeon PRO W6800 | 32 GB | ~112 tok/s | 231K · 256K (8-bit) | No price yet |
| AMD Radeon PRO W7800 32GB | 32 GB | ~122 tok/s | 231K · 256K (8-bit) | No price yet |
| Intel Arc Pro B70 | 32 GB | ~105 tok/s | 231K · 256K (8-bit) | No price yet |
| NVIDIA Quadro GV100 | 32 GB | ~198 tok/s | 231K · 256K (8-bit) | No price yet |
| NVIDIA RTX 5000 Ada Generation | 32 GB | ~148 tok/s | 231K · 256K (8-bit) | No price yet |
| NVIDIA RTX PRO 4500 Blackwell | 32 GB | ~202 tok/s | 231K · 256K (8-bit) | No price yet |
| NVIDIA V100 PCIe 32GB | 32 GB | ~202 tok/s | 231K · 256K (8-bit) | No price yet |
| NVIDIA V100S PCIe 32GB | 32 GB | ~234 tok/s | 231K · 256K (8-bit) | No price yet |
| Apple M5 Pro 48GB | 48 GB (~36 usable) | ~69 tok/s | 256K (full) | No price yet |
| NVIDIA A100 40GB PCIe | 40 GB | ~279 tok/s | 256K (full) | No price yet |
| NVIDIA A800 40GB Active | 40 GB | ~279 tok/s | 256K (full) | No price yet |
| AMD Radeon PRO W7800 48GB | 48 GB | ~160 tok/s | 256K (full) | No price yet |
| AMD Radeon PRO W7900 | 48 GB | ~160 tok/s | 256K (full) | No price yet |
| AMD Radeon PRO W7900 Dual Slot | 48 GB | ~160 tok/s | 256K (full) | No price yet |
| Apple Mac Studio M5 Max (40-core GPU, 64 GB) | 64 GB (~48 usable) | ~102 tok/s | 256K (full) | No price yet |
| NVIDIA A40 | 48 GB | ~170 tok/s | 256K (full) | No price yet |
| NVIDIA L40 | 48 GB | ~197 tok/s | 256K (full) | No price yet |
| NVIDIA L40S | 48 GB | ~197 tok/s | 256K (full) | No price yet |
| NVIDIA Quadro RTX 8000 | 48 GB | ~165 tok/s | 256K (full) | No price yet |
| NVIDIA RTX 6000 Ada Generation | 48 GB | ~211 tok/s | 256K (full) | No price yet |
| NVIDIA RTX PRO 5000 Blackwell 48GB | 48 GB | ~258 tok/s | 256K (full) | No price yet |
| NVIDIA RTX PRO 5000 Blackwell 72GB | 72 GB | ~258 tok/s | 256K (full) | No price yet |
| NVIDIA A100 80GB PCIe | 80 GB | ~312 tok/s | 256K (full) | No price yet |
| NVIDIA RTX PRO 6000 Blackwell Max-Q | 96 GB | ~300 tok/s | 256K (full) | No price yet |
| NVIDIA RTX PRO 6000 Blackwell Server Edition | 96 GB | ~285 tok/s | 256K (full) | No price yet |
| NVIDIA RTX PRO 6000 Blackwell Workstation Edition | 96 GB | ~300 tok/s | 256K (full) | No price yet |
Every device that runs it
Graphics cards, Macs and mini PCs from the wider hardware list behind My rig, smallest usable memory first. Not sure about yours? Check your exact rig.
Show all 96 devices that run it
| Device | Type | Memory | Speed estimate | Max context estimate | Price |
|---|---|---|---|---|---|
| AMD Radeon RX 7900 XTX | GPU | 24 GB | ~170 tok/s | 62K · 117K (8-bit) | low $1,350 |
| Intel Arc Pro B60 | Workstation GPU | 24 GB | ~84 tok/s | 62K · 117K (8-bit) | No price yet |
| NVIDIA A10 | Data center GPU | 24 GB | ~152 tok/s | 62K · 117K (8-bit) | No price yet |
| NVIDIA GeForce RTX 3090 | GPU | 24 GB | ~207 tok/s | 62K · 117K (8-bit) | low $1,499 |
| NVIDIA GeForce RTX 3090 Ti | GPU | 24 GB | ~218 tok/s | 62K · 117K (8-bit) | No price yet |
| NVIDIA GeForce RTX 4090 | GPU | 24 GB | ~218 tok/s | 62K · 117K (8-bit) | low $2,910 |
| NVIDIA L4 | Data center GPU | 24 GB | ~87 tok/s | 62K · 117K (8-bit) | No price yet |
| NVIDIA Quadro RTX 6000 | Workstation GPU | 24 GB | ~165 tok/s | 62K · 117K (8-bit) | No price yet |
| NVIDIA RTX 4500 Ada Generation | Workstation GPU | 24 GB | ~118 tok/s | 62K · 117K (8-bit) | No price yet |
| NVIDIA RTX A5000 | Workstation GPU | 24 GB | ~182 tok/s | 62K · 117K (8-bit) | No price yet |
| NVIDIA RTX A5500 | Workstation GPU | 24 GB | ~182 tok/s | 62K · 117K (8-bit) | No price yet |
| NVIDIA RTX PRO 4000 Blackwell | Workstation GPU | 24 GB | ~165 tok/s | 62K · 117K (8-bit) | No price yet |
| NVIDIA RTX PRO 4000 SFF Blackwell | Workstation GPU | 24 GB | ~118 tok/s | 62K · 117K (8-bit) | No price yet |
| NVIDIA RTX Spark N1X laptop (6144-core GPU, 24GB) | Laptop | 24 GB | ~87 tok/s | 62K · 117K (8-bit) | No price yet |
| NVIDIA Tesla P40 | Data center GPU | 24 GB | ~76 tok/s | 62K · 117K (8-bit) | No price yet |
| NVIDIA TITAN RTX | GPU | 24 GB | ~165 tok/s | 62K · 117K (8-bit) | No price yet |
| Apple M3 Max (30-core GPU) 36GB | Mac | 36 GB (~24.1 usable) | ~68 tok/s | 64K · 121K (8-bit) | No price yet |
| Apple M3 Pro 36GB | Mac | 36 GB (~24.1 usable) | ~41 tok/s | 64K · 121K (8-bit) | No price yet |
| Apple M4 Max (32-core GPU) 36GB | Mac | 36 GB (~24.1 usable) | ~82 tok/s | 64K · 121K (8-bit) | No price yet |
| Apple Mac Studio M5 Max (32-core GPU, 36 GB) | Mac | 36 GB (~24.1 usable) | ~88 tok/s | 64K · 121K (8-bit) | low $2,499 |
| AMD Radeon PRO V710 | Data center GPU | 28 GB | ~102 tok/s | 147K · 256K (8-bit) | No price yet |
| AMD Radeon AI PRO R9600 | Workstation GPU | 32 GB | ~131 tok/s | 231K · 256K (8-bit) | No price yet |
| AMD Radeon AI PRO R9600D | Workstation GPU | 32 GB | ~131 tok/s | 231K · 256K (8-bit) | No price yet |
| AMD Radeon AI PRO R9700 | Workstation GPU | 32 GB | ~131 tok/s | 231K · 256K (8-bit) | No price yet |
| AMD Radeon AI PRO R9700S | Workstation GPU | 32 GB | ~131 tok/s | 231K · 256K (8-bit) | No price yet |
| AMD Radeon PRO V620 | Data center GPU | 32 GB | ~112 tok/s | 231K · 256K (8-bit) | No price yet |
| AMD Radeon PRO W6800 | Workstation GPU | 32 GB | ~112 tok/s | 231K · 256K (8-bit) | No price yet |
| AMD Radeon PRO W7800 32GB | Workstation GPU | 32 GB | ~122 tok/s | 231K · 256K (8-bit) | No price yet |
| AMD Ryzen AI Max 385 (32GB) | Mini PC | 32 GB | ~65 tok/s | 231K · 256K (8-bit) | No price yet |
| AMD Ryzen AI Max 390 (32GB) | Mini PC | 32 GB | ~65 tok/s | 231K · 256K (8-bit) | No price yet |
| AMD Ryzen AI Max+ 395 (32GB) | Mini PC | 32 GB | ~65 tok/s | 231K · 256K (8-bit) | No price yet |
| Intel Arc Pro B70 | Workstation GPU | 32 GB | ~105 tok/s | 231K · 256K (8-bit) | No price yet |
| NVIDIA GeForce RTX 5090 | GPU | 32 GB | ~300 tok/s | 231K · 256K (8-bit) | low $6,500 |
| NVIDIA Quadro GV100 | Workstation GPU | 32 GB | ~198 tok/s | 231K · 256K (8-bit) | No price yet |
| NVIDIA RTX 5000 Ada Generation | Workstation GPU | 32 GB | ~148 tok/s | 231K · 256K (8-bit) | No price yet |
| NVIDIA RTX PRO 4500 Blackwell | Workstation GPU | 32 GB | ~202 tok/s | 231K · 256K (8-bit) | No price yet |
| NVIDIA RTX Spark N1X laptop (6144-core GPU, 32GB) | Laptop | 32 GB | ~87 tok/s | 231K · 256K (8-bit) | No price yet |
| NVIDIA V100 PCIe 32GB | Data center GPU | 32 GB | ~202 tok/s | 231K · 256K (8-bit) | No price yet |
| NVIDIA V100S PCIe 32GB | Data center GPU | 32 GB | ~234 tok/s | 231K · 256K (8-bit) | No price yet |
| Apple M3 Max (40-core GPU) 48GB | Mac | 48 GB (~36 usable) | ~81 tok/s | 256K (full) | No price yet |
| Apple M4 Max (40-core GPU) 48GB | Mac | 48 GB (~36 usable) | ~96 tok/s | 256K (full) | No price yet |
| Apple M4 Pro 48GB | Mac | 48 GB (~36 usable) | ~64 tok/s | 256K (full) | No price yet |
| Apple M5 Max (40-core GPU) 48GB | Mac | 48 GB (~36 usable) | ~102 tok/s | 256K (full) | No price yet |
| Apple M5 Pro 48GB | Mac | 48 GB (~36 usable) | ~69 tok/s | 256K (full) | No price yet |
| NVIDIA A100 40GB PCIe | Data center GPU | 40 GB | ~279 tok/s | 256K (full) | No price yet |
| NVIDIA A800 40GB Active | Workstation GPU | 40 GB | ~279 tok/s | 256K (full) | No price yet |
| AMD Radeon PRO W7800 48GB | Workstation GPU | 48 GB | ~160 tok/s | 256K (full) | No price yet |
| AMD Radeon PRO W7900 | Workstation GPU | 48 GB | ~160 tok/s | 256K (full) | No price yet |
| AMD Radeon PRO W7900 Dual Slot | Workstation GPU | 48 GB | ~160 tok/s | 256K (full) | No price yet |
| Apple M1 Max 64GB | Mac | 64 GB (~48 usable) | ~81 tok/s | 256K (full) | No price yet |
| Apple M1 Ultra 64GB | Mac | 64 GB (~48 usable) | ~115 tok/s | 256K (full) | No price yet |
| Apple M2 Max (30-core GPU) 64GB | Mac | 64 GB (~48 usable) | ~81 tok/s | 256K (full) | No price yet |
| Apple M2 Max (38-core GPU) 64GB | Mac | 64 GB (~48 usable) | ~81 tok/s | 256K (full) | No price yet |
| Apple M2 Ultra 64GB | Mac | 64 GB (~48 usable) | ~115 tok/s | 256K (full) | No price yet |
| Apple M3 Max (40-core GPU) 64GB | Mac | 64 GB (~48 usable) | ~81 tok/s | 256K (full) | No price yet |
| Apple M4 Max (40-core GPU) 64GB | Mac | 64 GB (~48 usable) | ~96 tok/s | 256K (full) | No price yet |
| Apple M4 Pro 64GB | Mac | 64 GB (~48 usable) | ~64 tok/s | 256K (full) | No price yet |
| Apple M5 Pro 64GB | Mac | 64 GB (~48 usable) | ~69 tok/s | 256K (full) | No price yet |
| Apple Mac Studio M5 Max (40-core GPU, 64 GB) | Mac | 64 GB (~48 usable) | ~102 tok/s | 256K (full) | No price yet |
| NVIDIA A40 | Data center GPU | 48 GB | ~170 tok/s | 256K (full) | No price yet |
| NVIDIA L40 | Data center GPU | 48 GB | ~197 tok/s | 256K (full) | No price yet |
| NVIDIA L40S | Data center GPU | 48 GB | ~197 tok/s | 256K (full) | No price yet |
| NVIDIA Quadro RTX 8000 | Workstation GPU | 48 GB | ~165 tok/s | 256K (full) | No price yet |
| NVIDIA RTX 6000 Ada Generation | Workstation GPU | 48 GB | ~211 tok/s | 256K (full) | No price yet |
| NVIDIA RTX A6000 | Workstation GPU | 48 GB | ~182 tok/s | 256K (full) | low $8,363 |
| NVIDIA RTX PRO 5000 Blackwell 48GB | Workstation GPU | 48 GB | ~258 tok/s | 256K (full) | No price yet |
| NVIDIA RTX Spark N1X laptop (6144-core GPU, 48GB) | Laptop | 48 GB | ~87 tok/s | 256K (full) | No price yet |
| AMD Ryzen AI Max 385 (64GB) | Mini PC | 64 GB | ~65 tok/s | 256K (full) | No price yet |
| AMD Ryzen AI Max 390 (64GB) | Mini PC | 64 GB | ~65 tok/s | 256K (full) | No price yet |
| AMD Ryzen AI Max+ 395 (64GB) | Mini PC | 64 GB | ~65 tok/s | 256K (full) | low $2,199.99 |
| NVIDIA DGX Spark (64GB) | Mini PC | 64 GB | ~81 tok/s | 256K (full) | low $4,999 |
| NVIDIA RTX Spark N1X laptop (6144-core GPU, 64GB) | Laptop | 64 GB | ~87 tok/s | 256K (full) | No price yet |
| Apple M2 Max (38-core GPU) 96GB | Mac | 96 GB (~72 usable) | ~81 tok/s | 256K (full) | No price yet |
| Apple M3 Max (30-core GPU) 96GB | Mac | 96 GB (~72 usable) | ~68 tok/s | 256K (full) | No price yet |
| Apple M3 Ultra 96GB | Mac | 96 GB (~72 usable) | ~116 tok/s | 256K (full) | No price yet |
| Apple Mac Studio M5 Ultra (64-core GPU, 96 GB) | Mac | 96 GB (~72 usable) | ~133 tok/s | 256K (full) | low $5,499 |
| NVIDIA RTX PRO 5000 Blackwell 72GB | Workstation GPU | 72 GB | ~258 tok/s | 256K (full) | No price yet |
| NVIDIA A100 80GB PCIe | Data center GPU | 80 GB | ~312 tok/s | 256K (full) | No price yet |
| Apple M1 Ultra 128GB | Mac | 128 GB (~96 usable) | ~115 tok/s | 256K (full) | No price yet |
| Apple M2 Ultra 128GB | Mac | 128 GB (~96 usable) | ~115 tok/s | 256K (full) | No price yet |
| Apple M3 Max (40-core GPU) 128GB | Mac | 128 GB (~96 usable) | ~81 tok/s | 256K (full) | No price yet |
| Apple M4 Max (40-core GPU) 128GB | Mac | 128 GB (~96 usable) | ~96 tok/s | 256K (full) | No price yet |
| Apple M5 Max (40-core GPU) 128GB | Mac | 128 GB (~96 usable) | ~102 tok/s | 256K (full) | No price yet |
| NVIDIA RTX PRO 6000 Blackwell Max-Q | Workstation GPU | 96 GB | ~300 tok/s | 256K (full) | No price yet |
| NVIDIA RTX PRO 6000 Blackwell Server Edition | Data center GPU | 96 GB | ~285 tok/s | 256K (full) | No price yet |
| NVIDIA RTX PRO 6000 Blackwell Workstation Edition | Workstation GPU | 96 GB | ~300 tok/s | 256K (full) | No price yet |
| AMD Ryzen AI Max 385 (128GB) | Mini PC | 128 GB | ~65 tok/s | 256K (full) | No price yet |
| AMD Ryzen AI Max 390 (128GB) | Mini PC | 128 GB | ~65 tok/s | 256K (full) | No price yet |
| AMD Ryzen AI Max+ 395 (128GB) | Mini PC | 128 GB | ~65 tok/s | 256K (full) | low $3,499.99 |
| NVIDIA DGX Spark (128GB) | Mini PC | 128 GB | ~81 tok/s | 256K (full) | low $6,949.99 |
| NVIDIA RTX Spark N1X laptop (6144-core GPU, 128GB) | Laptop | 128 GB | ~87 tok/s | 256K (full) | No price yet |
| Apple M2 Ultra 192GB | Mac | 192 GB (~144 usable) | ~115 tok/s | 256K (full) | No price yet |
| Apple M3 Ultra 256GB | Mac | 256 GB (~192 usable) | ~116 tok/s | 256K (full) | No price yet |
| Apple M5 Ultra 256GB | Mac | 256 GB (~192 usable) | ~133 tok/s | 256K (full) | No price yet |
| Framework Desktop, AMD Ryzen AI Max+ PRO 495 (Strix Halo), 192 GB | Mini PC | 192 GB | ~68 tok/s | 256K (full) | low $6,799 |
| Apple M5 Ultra 512GB | Mac | 512 GB (~384 usable) | ~133 tok/s | 256K (full) | No price yet |
Multi-GPU: cards that need two or more
| Cards | Total VRAM | Speed estimate | Max context estimate |
|---|---|---|---|
| 2 × Intel Arc B580 | 24 GB | ~84 tok/s | 62K · 117K (8-bit) |
| 2 × NVIDIA GeForce RTX 3060 12 GB | 24 GB | ~102 tok/s | 62K · 117K (8-bit) |
| 2 × NVIDIA GeForce RTX 4070 SUPER | 24 GB | ~133 tok/s | 62K · 117K (8-bit) |
| 3 × NVIDIA GeForce RTX 5060 | 24 GB | ~122 tok/s | 62K · 117K (8-bit) |
| 3 × NVIDIA GeForce RTX 5060 Ti 8 GB | 24 GB | ~122 tok/s | 62K · 117K (8-bit) |
| 2 × NVIDIA GeForce RTX 5070 | 24 GB | ~165 tok/s | 62K · 117K (8-bit) |
| 2 × AMD Radeon RX 7800 XT | 32 GB | ~129 tok/s | 231K · 256K (8-bit) |
| 2 × AMD Radeon RX 9060 XT 16 GB | 32 GB | ~78 tok/s | 231K · 256K (8-bit) |
| 2 × AMD Radeon RX 9070 | 32 GB | ~131 tok/s | 231K · 256K (8-bit) |
| 2 × AMD Radeon RX 9070 XT | 32 GB | ~131 tok/s | 231K · 256K (8-bit) |
| 2 × NVIDIA GeForce RTX 5060 Ti 16 GB | 32 GB | ~122 tok/s | 231K · 256K (8-bit) |
| 2 × NVIDIA GeForce RTX 5070 Ti | 32 GB | ~202 tok/s | 231K · 256K (8-bit) |
| 2 × NVIDIA GeForce RTX 5080 | 32 GB | ~211 tok/s | 231K · 256K (8-bit) |
Speed calculator
estimateEstimate = bandwidth ÷ GB read per token (the model's listed GGUF file: Q4_K_M for most models, otherwise the quant named in the list; scaled by active/total parameters for MoE), scaled by a per-platform efficiency and fixed per-token costs calibrated to 45 measured llama.cpp benchmarks (2025–26). Fit = file size + 1.5 GB headroom ≤ total VRAM (on Macs, the part of unified memory macOS lets the GPU use by default), and room for at least 4K tokens of context with an 8-bit KV cache. For a model with no KV cache data, only the file and headroom are checked, so a fit says nothing about context. Multi-GPU: VRAM adds, speed stays one card's (a layer split runs the cards in turn); max context uses the total. Speed is for short context. Max context = what is left after the file and a 1 GB runtime reserve ÷ KV cache bytes per token, default FP16 and 8-bit (q8_0) cache. How speed works · how context works