RigPerDollar

What GPU do you need to run inclusionAI Ling-3.1-flash 560B-A25B?

No single GPU. inclusionAI Ling-3.1-flash 560B-A25B needs about 359.5 GB at Q4, more than any GPU, system or build we price.

Parameters572.12B25B active (mixture-of-experts)
Q4_K_M GGUF357.97 GBsource
VRAM needed~359.5 GBfile + headroom
Single GPUs that fit0
ReleasedOct 13, 2026source
LicenseMIT
Context window1M tokens

572.12B parameters, 25B active (mixture-of-experts) (Hugging Face, accessed 2026-10-19). Q4_K_M GGUF: 357.97 GB (source). Architecture config (Hugging Face config.json; 72 layers in groups of 8 (layer_group_size): every 8th layer is gated MLA, the other 63 are KDA linear attention with a fixed-size state (arch hybrid, so the speed estimate adds the recurrent-state cost); each MLA layer caches one latent of kv_lora_rank 512 + qk_rope_head_dim 64 (llama.cpp convert: conversion/bailingmoe3.py)): 9 attention layers (the rest are recurrent) × 1 KV heads × 576 head dim, so about 10 KB of FP16 KV cache per token of context (how we estimate context).

Card: 560B total, 25B active, hybrid-linear MoE (63 KDA linear-attention + 9 gated MLA layers), 1M-token context. Parameter count is the safetensors total of the checkpoint (bartowski's GGUF card also says 572B), which includes the MTP layer. config.json's max_position_embeddings is 262,144, below the card's 1M. The BailingMoeV3 architecture is in stock llama.cpp (bartowski quantized with release b11627). Release date is the Hugging Face repo creation date.

Other places to get it

Checked Nov 14, 2026. We list only copies we opened and checked. Check file hashes for copies from other uploaders.

No single GPU in our database fits it. Use multiple GPUs (see calculator).

Every device that runs it

Graphics cards, Macs and mini PCs from the wider hardware list behind My rig, smallest usable memory first. Not sure about yours? Check your exact rig.

Show all 1 devices that run it
DeviceTypeMemorySpeed estimateMax context estimatePrice
Apple M5 Ultra 512GBMac512 GB (~384 usable)~45 tok/s256K (full)No price yet

Speed calculator

estimate

Estimate = bandwidth ÷ GB read per token (the model's listed GGUF file: Q4_K_M for most models, otherwise the quant named in the list; scaled by active/total parameters for MoE), scaled by a per-platform efficiency and fixed per-token costs calibrated to 45 measured llama.cpp benchmarks (2025–26). Fit = file size + 1.5 GB headroom ≤ total VRAM (on Macs, the part of unified memory macOS lets the GPU use by default), and room for at least 4K tokens of context with an 8-bit KV cache. For a model with no KV cache data, only the file and headroom are checked, so a fit says nothing about context. Multi-GPU: VRAM adds, speed stays one card's (a layer split runs the cards in turn); max context uses the total. Speed is for short context. Max context = what is left after the file and a 1 GB runtime reserve ÷ KV cache bytes per token, default FP16 and 8-bit (q8_0) cache. How speed works · how context works

Reported by others

These are speed reports posted by individual users or reviewers. We haven't checked or reproduced them. They are not our estimates, they don't feed the estimator or My Rig, and they can't be compared with the tok/s figures elsewhere on this site or with each other, because each used a different model, quant, engine and test.

What this does and doesn't show

For coding agents, how much total output a machine keeps up with several requests at once, and how long each waits for its first token, matter as much as single-request tok/s. These reports show that for specific setups only; they don't show what you'd get with a different engine, quant or workload.

Ling-3.1-flash on two DGX Sparks linked (LingSpark recipe, TensorFold)

Reported by @andersjw_ (ajensenwaud on GitHub)
How this run was set up
Model & revisioninclusionAI Ling-3.1-flash-1019 (per the README)
QuantThe author's own EXL3 pack, 2.71 bits per weight on average (197 GB): https://huggingface.co/ajensenwaud/Ling-3.1-flash-1019-exl3-2.71bpw
Runtime & versionTensorFold v1.2.2 (Zig engine) plus LingSpark's 26 patches
Topology2x DGX Spark joined by their CX7 ports, tensor parallel (model split across both)This is two machines; it is not a hardware entry on this site.
Input / output / context512-token replies, thinking off, one conversation, 1,048,576-token context window (release configuration)
Concurrency1 conversation
Speculative decodingOn: Ling's own MTP layer drafts tokens (speculative decoding). No run without drafting is given.
How speed was timedMeasured with spark-bench (https://github.com/ajensenwaud/spark-bench), both GPUs free, no other client
Date measuredUnknown (README as of 2026-11-05)
Sourceajensenwaud/lingspark-2dgx README (GitHub, commit 33b3c9c) · @plotarmordev on X, 2026-11-07
Evidence statusSelf-reported, not independently reproduced

With speculative decoding (Ling MTP drafting), greedy

WhatReportedNote
Decode, chat / code / agent / report32.2 / 33.6 / 35.2 / 33.1 tok/s—
Decode after a 128K-token prompt25.9 tok/s—
First token after a 128K-token prompt268 s—

Every figure has MTP drafting on; the README gives no plain-decoding figure.

Follow-up turns reuse the cached conversation and start in 7–8 s, per the README.

Caveats

  • Needs two Sparks linked, a patched community engine and a custom 2.71 bpw pack.
  • The README asks for about 200 GB of free disk space for the build on each Spark (for the pack), plus room for images and the engine build.
  • Self-reported in the project README, not independently reproduced.