RigPerDollar

What local AI models can the NVIDIA DGX Spark (128GB) run?

97 of our 115 models, up to StepFun Step 3.7 Flash. That is the largest current model that fits the DGX Spark's 128 GB at UD-Q4_K_M with room for about 94K tokens of context (180K with 8-bit KV cache), at about 34 tok/s estimate. It costs from $6,949.99 new.

Memory128 GB LPDDR5xsource
Bandwidth273 GB/ssource
Models that fit97 of 115
Pricelow $6,949.99new · source, 2026-10-23

Memory type: LPDDR5x (source) · Released: 2025 (source)

  • Speed: GB10 is not calibrated; uses the discrete-card CUDA constants, which may overstate it.

Specs from first-party pages (NVIDIA), accessed 2026-10-18. Speeds use the NVIDIA (CUDA) constants (how).

Every model, largest that fits first

ModelQuantFileFits?Speed estimateMax context estimate
StepFun Step 3.7 FlashUD-Q4_K_M122.09 GBFits~34 tok/s94K · 180K (8-bit)
Swift 1.5 Qwen3.8-Flash-NextQ4_K_M119.6 GBFits~48 tok/s256K (full)
Qwen3.8-Flash-NextQ4_K_M119.6 GBFits~48 tok/s256K (full)
Ornith-1.5-122BQ4_K_M78.92 GBFits~33 tok/s256K (full)
Mistral Medium 3.5 128BQ4_K_M78.41 GBFits~3 tok/s131K · 247K (8-bit)
Qwen3.5-122B-A10BQ4_K_M77.62 GBFits~33 tok/s256K (full)
Mistral Small 4 119BQ4_K_M72.64 GBFits~55 tok/s256K (full)
NVIDIA Nemotron 3.5 Super VL 120B-A12BQ4_K_M70.1 GBWeights fit context unknown~34 tok/sUnknown (no KV data)
gpt-oss-120bQ4_K_M62.84 GBFits~76 tok/s128K (full)
NVIDIA Nemotron 3.5 Bolt 74B-A9BUD-Q4_K_XL50.22 GBFits~34 tok/s256K (full)
Aleph Alpha Kolibri-1 78B-A3.5BQ4_K_M49.76 GBFits~92 tok/s256K (full)
Qwen3-Coder-NextQ4_K_M48.53 GBFits~85 tok/s256K (full)
Ripple-65B PreviewQ4_K_M38.8 GBFits~6 tok/s1M (full)
Recursive Flint 40B-A2.5BQ4_125.99 GBFits~118 tok/s256K (full)
NVIDIA Nemotron 3.5 Lightning 30B-A3BQ4_K_M25.48 GBFits~71 tok/s256K (full)
K2-Horizon-MoVA-36B-A4BQ4_K_M22.37 GBFits~86 tok/s512K (full)
Qwen3.6-35B-A3BQ4_K_M22.29 GBFits~84 tok/s256K (full)
Qwen3.5-35B-A3BQ4_K_M22.29 GBFits~84 tok/s256K (full)
NVIDIA Nemotron 3.5 Bolt 74B-A9B 24GB (2-bit)native 2-bit22.02 GBFits~66 tok/s256K (full)
MiniCPM-V 4.7 35B-A3BQ4_K_M21.17 GBFits~86 tok/s256K (full)
K2-Horizon-32BQ4_K_M21.08 GBFits~11 tok/s394K · 512K (8-bit)
Nanbeige4.3-33BQ4_K_M20.79 GBFits~11 tok/s486K · 512K (8-bit)
Xing4.0-29B-A4BQ4_K_M20.06 GBFits~81 tok/s256K (full)
Raon-1 mini preview Early supportQ4_K_M19.69 GBWeights fit context unknown~61 tok/sUnknown (no KV data)
PLaMo 3 Translate 31B Translation onlyQ4_K_M19.53 GBFits~12 tok/s32K (full)
Meta Muse Glimmer 30BQ4_K_M18.39 GBFits~13 tok/s128K (full)
Gemma 4 31BQ4_K_M18.32 GBFits~13 tok/s256K (full)
GLM-4.7-FlashQ4_K_M18.31 GBFits~111 tok/s198K (full)
Qwen3.6-27BQ4_K_M17.98 GBFits~13 tok/s256K (full)
Qwen3.5-27BQ4_K_M17.98 GBFits~13 tok/s256K (full)
IBM Granite 4.2 30BQ4_K_M17.72 GBFits~13 tok/s128K (full)
Microsoft Fara1.5 27BQ4_K_M17.53 GBFits~13 tok/s256K (full)
Qwen3.8-27BQ4_K_M17.44 GBFits~13 tok/s256K (full)
Gemma 4 26B A4BQ4_K_M17.04 GBFits~82 tok/s256K (full)
Kvark Sparkle 1.1 27BQ4_K_M16.81 GBFits~14 tok/s256K (full)
LFM2.5-24B-A2BQ4_K_M14.53 GBFits~109 tok/s128K (full)
Mistral-Small-3.2-24B-Instruct-2506Q4_K_M14.33 GBFits~16 tok/s128K (full)
Devstral-Small-2505Q4_K_M14.33 GBFits~16 tok/s128K (full)
gpt-oss-20bQ4_K_M11.67 GBFits~99 tok/s128K (full)
phi-4Q4_K_M9.05 GBFits~26 tok/s16K (full)
JetBrains Mellum2.1 12B-A2.5B ThinkingQ4_K_M8.07 GBFits~116 tok/s128K (full)
Gemma 4 12BQ4_K_M7.12 GBFits~32 tok/s256K (full)
NVIDIA Nemotron 3.5 Bolt E4BQ4_K_M6.18 GBFits~37 tok/s64K (full)
Qwen3.5-9BQ4_K_M6.17 GBFits~34 tok/s256K (full)
Ternary Bonsai 2 27BPTQ1_0 ternary5.95 GBFits~35 tok/s256K (full)
Microsoft Fara1.5 9BQ4_K_M5.91 GBFits~35 tok/s256K (full)
Xiaomi MiMo-V2.6-Distill-Qwen-9BQ4_K_M5.84 GBFits~36 tok/s256K (full)
ZoomGain-9BQ4_K_M5.63 GBFits~37 tok/s256K (full)
K2-Horizon-7BQ4_K_M5.59 GBFits~41 tok/s512K (full)
webAI TwIL-8B Math and code reasoningQ4_K_M5.35 GBFits~43 tok/s128K (full)
IBM Granite 4.2 8BQ4_K_M5.35 GBFits~43 tok/s128K (full)
PLaMo 3 Translate 8B Translation onlyQ4_K_M5.28 GBFits~43 tok/s32K (full)
LFM2.5-8B-A1BQ4_K_M5.16 GBFits~127 tok/s128K (full)
Gemma 4 E4BQ4_K_M4.98 GBFits~46 tok/s128K (full)
Spark-X2.6-4BQ4_K_M3.54 GBFits~63 tok/s1M (full)
Gemma 4 E2BQ4_K_M3.46 GBFits~64 tok/s128K (full)
Qwen3.5-4BQ4_K_M3.01 GBFits~62 tok/s256K (full)
NVIDIA Nemotron 3 Nano 4BQ4_K_M2.9 GBFits~64 tok/s256K (full)
Microsoft Fara1.5 4BQ4_K_M2.88 GBFits~64 tok/s256K (full)
ZoomGain-4BQ4_K_M2.71 GBFits~67 tok/s256K (full)
IBM Granite 4.2 3BQ4_K_M2.24 GBFits~95 tok/s128K (full)
Raon-1 nano Early supportQ4_K_M2.08 GBWeights fit context unknown~82 tok/sUnknown (no KV data)
Ternary Bonsai 2 9BPTQ1_0 ternary2 GBFits~84 tok/s256K (full)
PLaMo 3 Translate 2B Translation onlyQ4_K_M1.81 GBFits~114 tok/s32K (full)
LFM3-1.7BQ4_K_M1.1 GBFits~122 tok/s256K (full)
LFM3-VL-2BQ4_K_M1.1 GBFits~122 tok/s128K (full)
Ternary Bonsai 2 4BPTQ1_0 ternary0.95 GBFits~132 tok/s256K (full)
MiniCPM-V 4.7 1BQ4_K_M0.67 GBFits~157 tok/s256K (full)
PLaMo 3 Translate 0.8B Translation onlyQ4_K_M0.57 GBFits~280 tok/s32K (full)
BotPsy-0.8BQ4_K_M0.53 GBFits~172 tok/s256K (full)
Ternary Bonsai 2 2BPTQ1_0 ternary0.43 GBFits~186 tok/s256K (full)
NVIDIA Nemotron 3 Super 120B-A12B OlderUD-Q4_K_M82.54 GBFits~27 tok/s256K (full)
GLM-4.5-Air OlderQ4_K_M73.5 GBFits~29 tok/s128K (full)
Qwen2.5-72B-Instruct OlderQ4_K_M47.42 GBFits~5 tok/s32K (full)
Llama-3.3-70B-Instruct OlderQ4_K_M42.52 GBFits~6 tok/s128K (full)
DeepSeek-R1-Distill-Llama-70B OlderQ4_K_M42.52 GBFits~6 tok/s128K (full)
Qwen2.5-Coder-32B-Instruct OlderQ4_K_M19.85 GBFits~12 tok/s32K (full)
DeepSeek-R1-Distill-Qwen-32B OlderQ4_K_M19.85 GBFits~12 tok/s128K (full)
Qwen3-32B OlderQ4_K_M19.76 GBFits~12 tok/s40K (full)
Qwen3-30B-A3B OlderQ4_K_M18.63 GBFits~99 tok/s40K (full)
Qwen3-Coder-30B-A3B-Instruct OlderQ4_K_M18.56 GBFits~99 tok/s256K (full)
gemma-3-27b-it OlderQ4_K_M16.55 GBFits~14 tok/s128K (full)
LFM2-24B-A2B OlderQ4_K_M14.42 GBFits~109 tok/s128K (full)
Qwen3-14B OlderQ4_K_M9 GBFits~26 tok/s40K (full)
Qwen2.5-Coder-14B-Instruct OlderQ4_K_M8.99 GBFits~26 tok/s32K (full)
DeepSeek-R1-Distill-Qwen-14B OlderQ4_K_M8.99 GBFits~26 tok/s128K (full)
Mistral-Nemo-Instruct-2407 OlderQ4_K_M7.48 GBFits~31 tok/s128K (full)
gemma-3-12b-it OlderQ4_K_M7.3 GBFits~32 tok/s128K (full)
NVIDIA-Nemotron-Nano-9B-v2 OlderQ4_K_M6.53 GBFits~32 tok/s128K (full)
Qwen3-8B OlderQ4_K_M5.03 GBFits~45 tok/s40K (full)
Llama-3.1-8B-Instruct OlderQ4_K_M4.92 GBFits~46 tok/s128K (full)
Qwen2.5-Coder-7B-Instruct OlderQ4_K_M4.68 GBFits~48 tok/s32K (full)
Mistral-7B-Instruct-v0.3 OlderQ4_K_M4.37 GBFits~52 tok/s32K (full)
Qwen3-4B OlderQ4_K_M2.5 GBFits~86 tok/s40K (full)
gemma-3-4b-it OlderQ4_K_M2.49 GBFits~86 tok/s128K (full)
Llama-3.2-3B-Instruct OlderQ4_K_M2.02 GBFits~104 tok/s128K (full)
Qwen3-0.6B OlderQ4_K_M0.48 GBFits~313 tok/s40K (full)
Qwen3-235B-A22BQ4_K_M142.15 GBNo——
MiniMax-M3.1-Flash Unsloth 2-bitUD-Q2_K_XL143.91 GBNo——
DeepSeek-V4-FlashUD-Q4_K_XL155.1 GBNo——
MiniMax-M3.1-Flash Q2_KQ2_K155.91 GBNo——
Thinking Machines Inkling-Small OlderQ4_K_M159.34 GBNo——
Xiaomi MiMo-V2.6-Flash 309B-A15BMXFP4167.37 GBNo——
GLM-5.3-FlashUD-Q4_K_XL199.71 GBNo——
MiniMax-M3.1-FlashQ4_K_M258.01 GBNo——
Reflection Beam 1.0 501B-A23BQ4_K_M302.41 GBNo——
Thinking Machines Inkling 2 Small Early supportUD-Q4_K_XL354.15 GBNo——
inclusionAI Ling-3.1-flash 560B-A25BQ4_K_M357.97 GBNo——
DeepSeek-V4.1-Flash (1017)UD-Q4_K_XL500.71 GBNo——
Meta Muse Spark Open 816B-A42BUD-Q4_K_XL505.5 GBNo——
Mistral Large 4UD-Q4_K_XL605 GBNo——

Q4_K_M unless the row says otherwise; UD- sizes are unsloth dynamic quants, the only Q4 build published for those models. Max context is an estimate of the longest context whose KV cache fits beside the model: default FP16 cache, then 8-bit (q8_0).

Speed calculator

estimate

Estimate = bandwidth ÷ GB read per token (the model's listed GGUF file: Q4_K_M for most models, otherwise the quant named in the list; scaled by active/total parameters for MoE), scaled by a per-platform efficiency and fixed per-token costs calibrated to 45 measured llama.cpp benchmarks (2025–26). Fit = file size + 1.5 GB headroom ≤ total VRAM (on Macs, the part of unified memory macOS lets the GPU use by default), and room for at least 4K tokens of context with an 8-bit KV cache. For a model with no KV cache data, only the file and headroom are checked, so a fit says nothing about context. Multi-GPU: VRAM adds, speed stays one card's (a layer split runs the cards in turn); max context uses the total. Speed is for short context. Max context = what is left after the file and a 1 GB runtime reserve ÷ KV cache bytes per token, default FP16 and 8-bit (q8_0) cache. How speed works · how context works

Reported by others

These are speed reports posted by individual users or reviewers. We haven't checked or reproduced them. They are not our estimates, they don't feed the estimator or My Rig, and they can't be compared with the tok/s figures elsewhere on this site or with each other, because each used a different model, quant, engine and test.

What this does and doesn't show

For coding agents, how much total output a machine keeps up with several requests at once, and how long each waits for its first token, matter as much as single-request tok/s. These reports show that for specific setups only; they don't show what you'd get with a different engine, quant or workload.

Qwen3.8-Flash-Next, one DGX Spark, 8 coding agents at once (NVIDIA Flare)

Reported by @filicroval
How this run was set up
Model & revisionnvidia/Qwen3.8-Flash-Next-FP4-Local @ b9ce18a6
QuantNVIDIA's own NVFP4 checkpoint (NVFP4 experts, MXFP8 attention), not re-quantized
Runtime & versionNVIDIA Flare 0.1.0 + Ignition, NVIDIA/flare-local-inference @ ca1f960, built from source with patches (greedy top-k 1, mixed-prefill fix, oldest-first prefix eviction, prompt priority 16)
Topology1x DGX Spark (GB10, 128 GB unified memory)
Input / output / contextSix-turn code-review sessions: a shared ~5K-token agent prompt with tool definitions, a new source file each turn, replies up to 256 tokens, thinking on, greedy. Context 262,144 tokens.
Concurrency1 and 8 agents
Speculative decodingMTP, 3 fixed drafts per round
How speed was timedTotal output tok/s across all agents; the 8-agent figure is the mean of five runs
Date measuredOct 28–29, 2026
Sourcesxuff/qwen38-flash-next-flare README (GitHub) · @filicroval on X
Evidence statusSelf-reported, not independently reproduced

Reported results

WhatReportedNote
1 agent38.0 tok/s—
8 agents, total across all 883.0 tok/s TOTALmean of 5 runs, single runs 79.7–85.8
Per agent, with 8 agentsabout 10 tok/sderived by us: 83.0 / 8, not reported per agent
48 agent turns finished in143 s—
Median time to first token, 8 agents2.7 s—

Author's reference row: TensorFold, EXL3 4.05 bpw recipe

Author's comparison on the same test, measured on a different date (Oct 22–24, 2026) with an earlier recipe pin.

WhatReportedNote
1 agent29.9 tok/s—
8 agents, total23.9 tok/s total—
48 agent turns finished in480 s—
Median time to first token, 8 agents63 s—

The author notes TensorFold serves one request at a time, so with eight agents the others wait in line.

Caveats

  • Needs a from-source build with patches, and about 230 GB of free disk space for the build (checkpoint + package + kernel caches).
  • Self-reported by one person, not independently reproduced.
  • The repo's quality scores were taken on an earlier recipe.

DeepSeek-V4.1 Flash, two DGX Sparks linked (TensorFold)

Reported by @jayleaton
How this run was set up
Model & revisionDeepSeek-V4.1-Flash-1017 (per author; build "1017 update")
QuantUnknown in the latest post
Runtime & versionTensorFold, "latest" at the time of the post; exact version Unknown
Topology2x DGX Spark, TP=2 (model split across both)This is two machines; it is not a hardware entry on this site.
Input / output / contextUnknown in the latest post (prompts and output length not given)
ConcurrencyUnknown for the coding-task figure; 4 streams for the total
Speculative decodingMTP, settings Unknown
How speed was timedUnknown in the latest post
Date measuredUnknown (posted 2026-10-31)
Source@jayleaton on X, 2026-10-31 · @jayleaton on X, 2026-10-20 · @jayleaton on X, 2026-10-10 · @jayleaton on X, 2026-10-04
Evidence statusSelf-reported, not independently reproduced

Latest post, 2026-10-31

Post

WhatReportedNote
Real coding task, with MTP100.9 tok/s—
4 streams, total137.4 tok/s—
Prefill at 128K tokens2,203 tok/s—

Earlier post, 2026-10-20

Different builds; do not compare with the latest numbers.

Post

WhatReportedNote
QuantAuthor's flash17 EXL3 quant, 2.8 bpw203.8 GB of weights across the two Sparks
Code decode86.6 tok/s—
4 code streams, total191 tok/s—
32K-token prompt16.0 s—

In the author's words, there are "no shared, agreed benchmarks for speed" across these recipes.

Earlier post, 2026-10-10

Different builds; do not compare with the latest numbers.

Post

WhatReportedNote
Engine and quantZig rebuild of the engine, running the author's own 2.8 bpw EXL3 quant—
Code, 1 stream90.3 tok/s—
Code, 4 streams, total156.4 tok/s—
KV cache1,790 bytes per token (FP8 index keys), 2.33 GiB per Spark—
ContextAuthor says one request can use up to 1,048,576 tokens of contextclaim, not checked

Earlier post, 2026-10-04

Post

The author reported a memory leak and paused speed work for hour-long soak tests.

The 4-stream total reads 156.4 (Oct 10), 191 (Oct 20) and 137.4 tok/s (Oct 31). Those came from different builds and tests, so the change doesn't show a speedup or slowdown.

Caveats

  • Needs two Sparks linked, a community engine and a custom quant.
  • Self-reported, not independently reproduced.

M5 Ultra 256 GB vs two DGX Sparks: DeepSeek V4 Flash and Qwen 3.8 Flash Next

Reported by Alex Ziskind (@AZisk, YouTube)
How this run was set up
Model & revisionDeepSeek V4 Flash and Qwen 3.8 Flash Next
Quant4-bit, a different file on each platform; exact files Unknown
Runtime & versionvLLM; version Unknown
Topology2x DGX Spark, TP=2 over one 200 Gb QSFP cable with RoCE; the reviewer measured the link at about 111 (unit unclear in the video); about 111 GB used per SparkThis is two machines; it is not a hardware entry on this site.
Input / output / contextContext settings Unknown; prompt sizes as listed in the results
Concurrency1 and 8 users
Speculative decodingUnknown
How speed was timedUnknown
Date measuredUnknown (video published 2026-10-01)
SourceAlex Ziskind on YouTube, 2026-10-01
Evidence statusThird-party, from video transcript; on-screen charts not reviewed

Mac and 2x Spark: one user

WhatMac2x SparkNote
DeepSeek writing, short promptabout 38 tok/sabout 38 tok/s—
Qwen writing45 tok/s38 tok/s—
Opening run, 800 tokens38.7 tok/s34.3 tok/s—

Mac and 2x Spark: prompt reading (time to first token)

WhatMac2x SparkNote
Qwen, 2K tokens1.5 s0.85 s—
DeepSeek, 2K tokens2.5 s1.25 s—
DeepSeek, 8K tokensabout 10 sabout 4 s—
DeepSeek, 16K tokens21 sUnknowngiven for the Mac only
DeepSeek, 32K-token real codebase (14 Python files)50 s17 s—
128K tokensNot run72 s—
Cached follow-up, 16K context: first ask21 s8.3 s—
Cached follow-up, 16K context: follow-up3 s1.4 s—

Mac and 2x Spark: several users

WhatMac2x SparkNote
DeepSeek, 4 users, total66 tok/s totalNot runthe Mac's peak; everyone's tokens added up
DeepSeek, 8 users, total46 tok/s total70 tok/s totaleveryone's tokens added up
DeepSeek, 8 users with 1,000 tokens of chat history, totalabout 11 tok/s total25 tok/s totaleveryone's tokens added up
Same test: each user's wait for the first wordover a minuteabout 24 s—
Same test: each user after thatabout 5 tok/sabout 7 tok/s—
Qwen, 8 users, total66 tok/s (llama.cpp), 70 tok/s (oMLX)120 tok/s totaleveryone's tokens added up

The Mac column is the reviewer's comparison machine from the same video, a Mac Studio M5 Ultra 256 GB running llama.cpp and MLX (oMLX for several users), versions Unknown.

The reviewer used a different 4-bit file on each platform (his words: same model, not the same file). Exact files, runtime versions and context settings: Unknown.

These are two different setups; the table shows what the reviewer measured, not which machine to buy.

Caveats

  • Different 4-bit file on each platform.
  • The MLX figures were run in-process, not served over HTTP like the other runs.
  • Totals are everyone's tokens added up.
  • Numbers spoken in the video; on-screen charts not reviewed.
  • The video has a sponsor segment.

DeepSeek-V4.1-Flash-1017 NVFP4 on four DGX Spark class boxes, tensor parallel 4 (vLLM), with draft tokens on

Reported by Niklas Lenz (@niklaslenz_ai, niklaslenz.com)
How this run was set up
Model & revisionnvidia/DeepSeek-V4.1-Flash-1017-NVFP4 (revision 480550d)
QuantNVIDIA's NVFP4 checkpoint
Runtime & versionvLLM nightly 0.30.1rc1.dev558+g0cbac6cd1 with ondigo-winder recipe e7b0d33 patches 0001–0003, NCCL 2.30.7
Topology4x GB10 boxes (2x ASUS Ascent GX10, 1x Gigabyte AI TOP ATOM, 1x Lenovo ThinkStation PGX), TP4 over a switchless CX7 ring. One box ran an older driver and kernel than the other three.These figures are for the whole four-box cluster. They are not per box and this is not a hardware entry on this site.
Input / output / context512 output tokens, thinking off, temperature 1.0, top-p 1.0, prompt cache off, seed 42. Context 262,144 tokens.
Concurrency1, 2, 4 and 8 requests (prose prompt)
Speculative decodingOn: DSpark, 6 draft tokens (greedy). The raw file shows about 2.5 accepted tokens per step on these prompts. No run with drafting off was published.
How speed was timedMedian of 2 runs per prompt. Decode = tokens after the first divided by time after the first token. At once = total tokens of all requests divided by wall time.
Date measured2026-10-31
SourceAuthor's page (niklaslenz.com) · Raw output file (speed2.json) · @niklaslenz_ai on X · ondigo-winder recipe (GitHub) · nvidia/DeepSeek-V4.1-Flash-1017-NVFP4 checkpoint (Hugging Face)
Evidence statusSelf-reported; numbers checked by us against the author's published raw output file; not independently reproduced

With draft tokens on (DSpark, 6 drafts), whole cluster

WhatReportedNote
Decode, code prompt33.6 tok/s—
Decode, prose prompt30.2 tok/s—
1 request at once, total27.6 tok/s total—
2 requests at once, total46.6 tok/s total—
4 requests at once, total73.5 tok/s total—
8 requests at once, total114.8 tok/s total—
Prefill at a 2K prompt3,183 tok/s—

The author's charts include rankings that we do not reproduce.

Caveats

  • Draft tokens were on for every number; no run with drafting off was published.
  • Mixed drivers: one box ran an older driver and kernel than the other three.
  • These are whole-cluster figures; do not divide or multiply them by the number of boxes.
  • One person's setup; the numbers come from the author's own test tool, not ours.

Ling-3.0-flash-fp4 on one DGX Spark (SGLang): baseline and with three switches

Reported by Niklas Lenz (@niklaslenz_ai, niklaslenz.com)
How this run was set up
Model & revisionLing-3.0-flash-fp4 (inclusionAI); revision Unknown
QuantMXFP4 experts, FP8 rest, some BF16, about 4.42 bpw (author's figure)
Runtime & versionSGLang with the NVIDIA/Ant Ling recipe; version Unknown
Topology1x DGX Spark
Input / output / context512 output tokens, thinking off, temperature 0.6, every prompt unique. Context Unknown.
Concurrency1 request at a time
Speculative decodingOff in the baseline row. On (MTP) only in the last row, labelled.
How speed was timedMedian of 2 runs; how decode was timed is not stated in the post
Date measuredUnknown (posted 2026-10-30)
Source@niklaslenz_ai on X, 2026-10-30
Evidence statusSelf-reported in an X post and its image; no raw output file found on the author's site; not independently reproduced

Baseline (no drafting)

WhatReportedNote
Code decode36.2 tok/s—
Prefill at a 16K prompt4,121 tok/s—
First token at a 128K prompt45.5 s—

With all three switches, including MTP drafting (labelled separately, not a baseline)

WhatReportedNote
Code decode42.4 tok/s—
First token at a 128K prompt49.9 s—

The three switches are Humming MoE kernels, an FP8 lm_head and MTP, added one by one; this row has all three, so it is not MTP alone.

The author's charts include rankings that we do not reproduce.

Caveats

  • The author reports stray Chinese characters in some English answers with the switches on (7 of 52 saved answers with all three, 0 of 51 on the baseline).
  • One person's setup; the numbers come from the author's own test, not ours.

DeepSeek V4.1 Flash: one Mac Studio M5 Ultra 512 GB and four ASUS GX10 boxes (vLLM TP4), same bench

Reported by @seapy on X
How this run was set up
Model & revisionDeepSeek V4.1 Flash, original weights (chart title: "original weights, no quantization"). seapy doesn't name the snapshot; this site's model entry is the 1017 checkpoint
QuantNone per the chart. The spark-ds41f README says it uses DeepSeek's native FP8/FP4/BF16 weights: "Nothing is re-quantized". Mac file: Unknown
Runtime & versionMac: mlx-serve 26.11.2 by @ddalcu (an MLX-based server for Apple Silicon). GX10: the spark-ds41f recipe by @christopherowen, vLLM with tensor parallel 4 (seapy's reply: "It's vLLM (TP4)"); vLLM version Unknown
TopologyMac: 1x Mac Studio M5 Ultra 512 GB (8 TB). GX10: 4x ASUS GX10 (GB10, 128 GB each), vLLM TP4; seapy says the recipe works without a switchThe GX10 figures are for the whole four-box cluster, not per box. Neither the cluster nor the 512 GB Mac is a hardware entry on this site.
Input / output / contextText and code prompts at concurrency 1 and 16, plus 128K-token and 980K-token prompts for prefill; other prompt and output lengths Unknown
Concurrency1 and 16
Speculative decodingNot stated by seapy for either side. The spark-ds41f README lists "DSpark and CopyDrafter speculative decoding" under Speed and "DSpark speculative decoding with five draft tokens" in its production configuration, so the recipe's documented configuration includes DSpark drafting; seapy doesn't state it. Mac side: Unknown.
How speed was timed"total tok/s" per the chart; the prefill rows are prompt tokens per second. Method otherwise Unknown
Date measuredUnknown (posted 2026-11-11)
Source@seapy on X, 2026-11-11 (post and chart) · seapy's reply: vLLM (TP4) · seapy's reply: no switch · spark-ds41f recipe (GitHub) · mlx-serve (GitHub)
Evidence statusSelf-reported, not independently reproduced

Mac and 4x GX10, total tok/s as charted (drafting not stated; the GX10 recipe's documented configuration includes DSpark)

WhatMac4x GX10Note
Concurrency 1, text52.6 tok/s total73.4 tok/s total—
Concurrency 1, code83.7 tok/s total122 tok/s total—
Concurrency 16, text137 tok/s total330 tok/s totaleveryone's tokens added up
Concurrency 16, code148 tok/s total847 tok/s totaleveryone's tokens added up

Mac and 4x GX10: prompt reading (prefill; drafting not stated)

WhatMac4x GX10Note
128K-token prompt3,325 prompt tok/s7,522 prompt tok/s—
980K-token prompt1,787 prompt tok/s5,505 prompt tok/s—

The Mac column is the chart's other machine: one Mac Studio M5 Ultra 512 GB running mlx-serve 26.11.2.

Prices on the chart are seapy's, dated 11 Nov 2026: Mac $25,799 (apple.com, 512 GB / 8 TB); GX10 $5,999.99 each at Micro Center, $24,000 for four.

seapy's post headline gives speed ratios; those are his, and we don't repeat them.

These are two different setups; the table shows what seapy measured, not which machine to buy.

Caveats

  • seapy doesn't say whether drafting was on for either side; the GX10 recipe's documented configuration includes DSpark drafting with five draft tokens.
  • These are whole-cluster figures for the GX10 side; do not divide or multiply them by the number of boxes.
  • seapy says only "DeepSeek V4.1 Flash"; this site's model entry is the 1017 checkpoint.
  • Self-reported, not independently reproduced.

Owner tips

Setup tips for people who own this machine. Each one says where it comes from. They don't change the fit or speed estimates on this page.

Running headless? Set Display Reserved Memory to Auto

Confirmed in NVIDIA's documentation

DGX Spark reserves 2 GB of its unified memory for the display by default. Since the October 2026 update (OTA2610) a UEFI setting lets you change it: Auto, 0.5 GB, 1 GB, 2 GB (default) or 4 GB.

With no display connected, Auto reserves 0.5 GB. NVIDIA recommends 0.5 GB for headless systems that only use remote access, and says this frees up to 1.5 GB for the OS and apps.

Steps (NVIDIA's user guide)

  1. Run free -h and note the total.
  2. Restart and hold Esc or Del to open UEFI setup.
  3. Go to Advanced → Advanced Menu → Display Reserved Memory (the UEFI manual lists it under Platform Configuration).
  4. Pick Auto or 0.5 GB.
  5. Save & Exit → Save Changes and Reset.
  6. Run free -h again.

Auto only detects displays at boot, so if you plug a monitor in later, pick a fixed size.

NVIDIA's porting guide also documents setting it from the OS through a UEFI variable (linked below).

Sources: NVIDIA DGX Spark User Guide, UEFI settings · DGX Spark UEFI manual, Advanced tab · NVIDIA DGX Spark porting guide (PDF) · Tip shared by @plotarmordev · Steps (@plotarmordev)

Watch CPU temperature separately

Owner-reported; not in NVIDIA's documentation

Owners report that the Spark's fan follows GPU load, so CPU-heavy work (agent loops, tokenization, compiles, CPU offload) can run the CPU hot while the GPU is idle and the fan stays slow.

If your Spark runs agents around the clock, set an alert on CPU temperature, not just GPU temperature; cat /sys/class/thermal/thermal_zone*/temp reads every thermal zone.

Community fan-override tools exist. They're unofficial and built on a reverse-engineered fan override, and we haven't tested them.

Sources: @plotarmordev on X · @plotarmordev on X (follow-up) · Owner report on NVIDIA's developer forum