DGX Spark Cluster Guide: What Two Linked Sparks Buy You

A DGX Spark cluster of two units keeps more of a big model in memory. Each DGX Spark has 128 GB of memory, so two of them have 256 GB of physical memory between them. You can’t use all of it for model weights. The operating system, the inference software and the KV cache (the model’s saved context) all take a share. In one NVIDIA forum thread, Ray reported about 218 GiB of the 256 GB, because Ray holds back memory for its own object store. (Ray is a tool that runs one job across several machines.)

A cluster is one way to run a model bigger than 128 GB. Some single-Spark tools read the weights that don’t fit from the SSD instead. We cover one of them in SSD-streamed setups on one Spark.

Inside each Spark, memory moves at 273 GB/s. The link between two Sparks runs at 200 Gbit/s, which works out to 25 GB/s. That is a nominal rate, and apps may see less. A community guide measured 109 Gb/s with one ib_write_bw stream and about 17.4 GB/s with NCCL (NVIDIA’s library for moving data between GPUs) over two links. None of the sources below tested one user’s reply speed on one Spark against two with everything else held the same.

Our My rig calculator and the Spark page model one Spark only. They don’t estimate a cluster or SSD streaming.

Four ways to run a model bigger than one box

People use these terms loosely. This guide keeps them apart.

1. In-memory distributed cluster. The model is split across two or more Sparks (or Macs), and every weight sits in some machine’s memory. The split can be by layers (pipeline) or within layers (tensor). exo documents both. Most of this guide is about this case.
2. Layer or expert offload on one machine. Some layers or experts sit in GPU memory and the rest sit in system RAM on the same machine.
3. SSD-streamed runtime. A runtime built for it keeps part of the model in memory and reads the rest from the SSD as each token is made. Strata is one example. This is the runtime’s own design, and it is different from the operating system’s swap. Having enough disk does not make any model runnable, and the download size is not the amount of memory a model needs.
4. Prefill on a Spark, decode on a Mac. One machine reads the prompt and the other writes the answer. Our Mac Studio clusters and the Spark + Mac setup guide covers this one and Mac clusters.

What NVIDIA supports

NVIDIA documents this in the Cluster Assistant page and the connect multiple Sparks playbook (cabling page). Here are the layouts they describe.

In the four-device ring, Spark 1 and Spark 3 have no direct cable, and neither do Spark 2 and Spark 4. You can’t mix direct cables and a switch in one cluster.

Use one cable per link. NVIDIA’s page says: “Connecting two devices with two cables will not improve performance.” The playbook says the same for two devices.

A switch has to give each Spark port 200 Gbit/s. A 400G switch port may need to be split into two 200G ports. All Spark ports must be on the same local network segment (L2), with the default MTU (the largest packet size). The playbook points to MikroTik compatibility docs for the CRS804 and CRS812 switches.

What Cluster Assistant does

Cluster Assistant is NVIDIA’s setup helper for 2 to 4 DGX Spark and GB10 devices. GB10 is the chip inside the Spark, and other GB10 boxes exist. The helper does two jobs. It sets up the network, and it sets up SSH logins between the devices.

It does not set up workloads. In NVIDIA’s words, it “does not set up workloads, such as inference or fine-tuning.” Tools like Slurm or Kubernetes are yours to install. Other things to know:

• Every node needs April 2026 system software or later.
• The link check expects a 200 Gbit/s link. Its speed test passes at 184 Gbit/s or more.
• Supported cables are QSFP112 DAC (direct attach copper) cables for 400 GbE, in Ethernet mode only: Amphenol NJAAKK-N911 and Luxshare LMTQF022-SD-R.
• To change the layout, you delete the cluster and create it again.
• For the next steps it points to NVIDIA playbooks for NCCL, vLLM and PyTorch fine-tuning.

NVIDIA playbooks

• Connect Multiple DGX Sparks for Distributed Workloads. It covers 2 to 4 Sparks. You can use Sync (listed as beginner, about 10 minutes) or set it up by hand (about 1 hour). It needs April 2026 DGX OS or later. A switch needs one 200 Gbit/s link per device. The playbook says the ConnectX-7 network card enables RDMA (direct memory access over the network) between devices. Last updated 09/24/2026.
• NCCL playbook. It covers 2, 3 or 4 nodes, with tabs for two nodes direct, three nodes in a ring and four nodes on a switch. You build NCCL v2.31.2-1 for sm_121 and nccl-tests v2.19.7. It recommends Cluster Assistant for the network setup and the same username on every node. Last updated 10/14/2026.

What you need to buy

These are the prices the retailers showed on Oct 28, 2026.

For two Sparks with a direct cable, the prices add up to $14,079.97 on Oct 28, 2026. That is two Sparks at $6,949.99 each plus one $179.99 cable. Two GX10 boxes alone add up to $11,998.

Other GB10 boxes are on sale too. We did not verify their prices, so they are not in the table.

The 64 GB DGX Spark option

NVIDIA now offers the DGX Spark with 64 GB of memory. Its spec page lists “64 GB LPDDR5X* or 128 GB LPDDR5x”. Both have a 256-bit memory interface and 273 GB/s of memory bandwidth. The footnote says the “64 GB memory configuration is available exclusively through participating OEM partners.” In its Oct 2, 2026 post, NVIDIA said Acer, ASUS, Dell, Gigabyte, HP and MSI would sell it from Friday, Oct 23, starting at $4,999.

NVIDIA makes these claims in that post. One 64 GB unit “supports up to 100-billion-parameter models.” Two units connect with a QSFP cable and pool their memory to 128 GB, for models of “up to 200 billion parameters.” In NVIDIA’s Qwen 3.8 27B test, two clustered 64 GB systems delivered “up to 1.7x performance compared with a single system.” We have not tested any of this. NVIDIA’s post is the only source we have for the 1.7x figure.

The $4,999 is NVIDIA’s announced starting price for partner systems. We have not checked a store listing for any 64 GB model. The pair also needs a cable, such as the stacking cable in the table above. Two boxes give the same 128 GB in total as one 128 GB Spark, split across two machines. Each box reads its own memory at 273 GB/s. Our DGX Spark 64GB page estimates fit and speed for one box only.

Software paths

The cable and Cluster Assistant get you a network. You still need software that splits one model across both machines. There are two ways to split it. Pipeline splitting gives each machine a share of the layers. Tensor splitting divides each layer across the machines. The exo project documents both modes. The two-Spark examples below (GLM, botAGI, Ata and LingSpark) use tensor parallel across two nodes, written TP=2.

• vLLM with Ray, from NVIDIA’s vLLM playbook. vLLM is an inference server. In the NVIDIA forum thread, NVIDIA staff said to use the IP address assigned to the ConnectX-7 network card (the Spark’s fast network port). You may need --node-ip-address in run_cluster.sh. The script was updated on Dec 22, 2025 with a fix. Run the serve command on the head node only. Ray’s object store size can be changed with --object-store-memory.
• NVIDIA’s NCCL playbook, described above.
• The community GLM guide. It runs vLLM 0.31.0 with TP=2 on two Sparks. It uses two QSFP cables between the Sparks. That is a community recipe, and it differs from NVIDIA’s one-cable guidance.
• llama.cpp RPC and TensorRT-LLM are other options for running across machines. We have not tested either one.
• EXL3 and TensorFold are community tools used in Ata’s and LingSpark’s runs below. EXL3 is a quantized weight format, sized in bits per weight (bpw).

Third-party cluster benchmark reports

These are other people’s runs on two Sparks, as their authors reported them. Each source used a different model, quant, runtime and setting, so rows are not comparable with each other. We don’t rank them, and a lower-bit quant or a draft model can change output as well as speed. “Unknown” means the source didn’t say. Fields shared by every row in a table are in the list above it.

Four kinds of speed appear below, in separate columns:

• Prefill: how fast the prompt is read, before the first output token.
• Single-user decode: how fast one user’s answer is written.
• Single-stream output throughput: output tokens for one user divided by the whole request time, prefill included. Only the GLM table uses it.
• Aggregate: total tokens per second across several users at once. One user gets less.

Speculative decoding shows up in several rows. A small draft model (or the model’s own MTP head, short for multi-token prediction) guesses tokens ahead, and the main model checks them. It changes speed, so each row says whether it was on.

autotrust GLM guide

From the GLM guide.

• Hardware: 2 Sparks, TP=2, two QSFP cables.
• Model: autotrust/GLM5.3-Flash-E224-DGX-Spark, NVFP4. The model file is 158 GB, with 71.2 GiB of weights on each node.
• Engine: vLLM 0.31.0, fp8 KV cache, configured max model length 1,048,576 tokens.
• Prompt: a fixed English writing prompt of about 40 tokens, temperature 1.0, top_p 0.95, reasoning effort low.
• Timing: output tokens ÷ total request time, including prefill (clock starts before the HTTP request); not decode-only. Output tokens come from usage.completion_tokens, reasoning tokens included.
• Warmup: one warmup request before timing.
• Output cap: max_tokens 1,024 for a single stream and 512 for concurrent streams. Actual output length: Unknown.
• Evidence: author-reported, single setup.

The guide found MTP slower on this setup: 11.1 against 15.5 tok/s for a single stream, both measured as output tokens ÷ total request time with prefill included.

botAGI DeepSeek-V4-Flash-0731 context series

From botAGI/dspark-0731-gb10. The repo says its earlier client-side numbers were “biased upward” and “superseded in part.” It says “The engine-side series is the one to quote.” We show only the engine-side series.

• Hardware: 2 nodes, TP=2 over 200G RoCE (a fast way to move memory between machines over Ethernet).
• Model: DeepSeek-V4-Flash-0731, a 284B mixture-of-experts (MoE) model with about 13B active parameters. Checkpoint apetersson/DeepSeek-V4-Flash-0731-Abliterated-FP8, revision 7d02640c. That is an abliterated community checkpoint with native MXFP4 experts. The official checkpoint was not benchmarked.
• Engine: vLLM image ghcr.io/anemll/dspark-vllm-gx10@sha256:a83948492cf13df455170fb42885f5ef4db54fefe0feff0f841ecbff464ac9d8, version Unknown, flashinfer_b12x MoE backend.
• Draft: DSpark draft model on (speculative), 39% to 56% acceptance.
• Timing: engine-side /metrics, single stream, median of 5.
• Output length and date: Unknown.
• Evidence: author-reported. botAGI’s summary repo lists it as “lab_single_run”.

Ata’s overview chart

From Ata’s post on 2026-10-28, labelled “Measured on my DGX Sparks.”

• Speed type: single-user decode on real prompts, tok/s on a JSON prompt and on a code prompt.
• Output: 300 tokens per reply. Temperature 0.7, except the Nemotron and 27B rows at temperature 0.
• Input lengths, exact prompts, context or KV limits, software versions: Unknown.
• Warm or cold: Unknown.
• Evidence: author-reported, single setup.

Two Sparks, TP=2:

For the DeepSeek row the author notes it “fits, but only 7-9 GiB free on each Spark.”

The same chart has single-Spark rows. We list them here for context only.

The Qwen3.8-Flash-Next rows can look like a one-Spark versus two-Spark test. The single Spark ran MTP 5 and the pair ran MTP 4. Two things changed at once, so this pair of rows does not show how speed scales with a second node.

Ata’s original DeepSeek test

From an earlier post by Ata on 2026-10-23. It came out before the overview chart, so we keep it in its own table.

• Hardware: 2 Sparks, TP=2, QSFP link, one rank per Spark.
• Model: deepseek-v4.1-flash-1017, 763B MoE, Mia’s 2.9 bpw EXL3 build (v1).
• Engine: TensorFold, version Unknown.
• Draft: DSpark draft model on (speculative).
• Single user, temperature 0.7, 300 output tokens, median of 5. Input length, context and warm or cold: Unknown.
• Evidence: author-reported. The author says: “Speed only. Quality against the full model not measured.”

The post also reports a weight load of 27 s. The 8K prefill figure doesn’t tell you the input length of the decode trials.

LingSpark Ling-3.1-flash recipe

From the LingSpark README by @andersjw_ (ajensenwaud on GitHub), commit 33b3c9c of 2026-11-05, relayed in a post by @plotarmordev on 2026-11-07.

• Hardware: 2 Sparks joined by their CX7 ports, tensor parallel (model split across both).
• Model: inclusionAI Ling-3.1-flash-1019 (per the README), in the author’s own EXL3 pack, 2.71 bits per weight on average (197 GB).
• Engine: TensorFold v1.2.2 (Zig engine) plus LingSpark’s 26 patches.
• Draft: Ling’s own MTP layer on (speculative), greedy. The README gives no run without drafting.
• One conversation, thinking off, 512-token replies, 1,048,576-token context window (release configuration).
• Timing: spark-bench, both GPUs free, no other client.
• Warm or cold and date measured: Unknown (README as of 2026-11-05).
• Evidence: author-reported, single setup.

Every figure has MTP drafting on. The README says follow-up turns reuse the cached conversation and start in 7 to 8 s. It asks for about 200 GB of free disk space for the build on each Spark for the pack, plus room for the images and the engine build. The Ling-3.1-flash page lists this run too.

SSD-streamed setups on one Spark

This section is about one Spark with no cluster. It is the third case in the list near the top. Strata keeps part of the model in memory and reads the rest from the SSD for each token. That is Strata’s own design, and it is different from OS swap. Nothing here means that enough disk lets any model run. The download size is not the memory need either. In the recorded Faster-mode 8K run on the Spark, Strata held 11,187 of DeepSeek’s 15,744 experts in the GPU’s share of memory and streamed the rest. The same doc’s general description gives 12,163 of 15,744.

The records below come from Strata 0.2.7.1 (release tag v0.2.7, published 2026-10-27) and its DGX Spark notes. The app version stays 0.2.7. Control is 0.1.53. The new Spark engines and the new DeepSeek engine for the Mac are 0.1.53. The release says Windows, Linux x86 and Mac users running Qwen only get “no new engine, no speed change and no app change.” Other hardware named in the release notes is announced plans. Those are not results, so we leave them out.

Tested setup and limits:

• One GB10 unit with 128 GB (one DGX Spark), Ubuntu 24.04, driver 580.173, CUDA 13.0. The release notes also ask for glibc 2.39 or later and driver 580.65 or later.
• Terminal only. There is no desktop app for ARM Linux.
• In the doc’s words: “Nothing here is known to work on another ARM machine.”
• Not tested: contexts over 128K, more than 8 chat slots, long soak runs.
• All figures are self-reported by the project’s author. We have not checked them.

DeepSeek-V4.1-Flash on one Spark

Model DeepSeek-V4.1-Flash at 2.00 bpw, a 257 GB download. The default “Faster” mode runs 5 of the model’s 6 experts per token. (In a mixture-of-experts model, each token runs through a few small expert blocks.) Faster is a reduced-expert mode, and it can affect output quality. The doc reports no measured quality figure for it on the Spark. The 6-expert mode is what the project calls full quality.

The two rows used different prompts and runs, so they don’t compare the two modes directly. Prefill and aggregate figures for DeepSeek on the Spark: Unknown.

Qwen3.8-Flash-Next on one Spark

These measure four default speed settings in a release-candidate engine of 0.2.7.1, turned on against turned off (STRATA_SM121_PERF=0). Setup: tools/dgx_bench.py, fresh 4K, 16K and 32K prompts, temperature 0, 256-token answers, 64K context, medians of 4, run on 2026-10-27. Model revision and warm or cold: Unknown. Decode used the MTP drafter (speculative decoding on).

The gain from the settings is in prefill. The +36% figure is IQ3_S prefill at 32K.

Across all sizes the doc gives 1.20x for IQ3_S and 1.24x for NVFP4. Decode did not change in any material way (0.996x for IQ3_S, 1.02x for NVFP4).

Several chats at once, IQ3_S, 2K prompts, answers capped at 128 tokens. These are aggregate totals across all clients. Each chat gets less.

The project’s DeepSeek records on a Mac are in our Mac Studio clusters and the Spark + Mac setup guide.

Who a Spark cluster makes sense for

It fits people who want more of a large model held in memory. The GLM model above is 158 GB, and it ran with 71.2 GiB of weights on each node. Ata’s 763B DeepSeek build fit on two Sparks with 7 to 9 GiB free on each. Some single-Spark tools stream the overflow from the SSD instead, as in the Strata records above, with their own limits and settings.

It also fits people who serve several requests at once. In the GLM guide, aggregate throughput rose from 15.6 tok/s with one stream to 102.4 tok/s with 16 streams. The 15.6 figure counts total request time, prefill included. Each of the 16 streams got 6.4 tok/s.

Think twice if your model already fits on one Spark and you want faster replies for one user. No source here measured one user’s speed on one Spark against two with everything else the same. Check what fits on one Spark first with My rig. Keep in mind that our Spark speed estimates use constants fitted to discrete NVIDIA cards and may run high, as the methodology explains. For why tok/s alone can mislead, see tok/s vs agent performance.

Expect setup work. The GLM guide needed 64 GB of swap and MAX_JOBS=2 to avoid running out of memory, and it set GPU memory use to 0.79. The forum thread shows that the NIC’s IP address matters for vLLM with Ray. Cluster Assistant stops at the network, so the inference stack is yours to build. If you are still choosing a first machine, the budget local AI PC guide covers single boxes by price.

What we haven’t verified

We have not run a Spark cluster or Strata ourselves. These items are open:

• Every benchmark above. Each one is the author’s own report from a single setup, and we have not reproduced any of them.
• How one user’s speed changes from one Spark to two with everything else held the same. No source here ran that test.
• NVIDIA’s forum performance threads. The forum thread above is the only one we cite.
• Prices for other GB10 boxes.
• Store prices for any 64 GB DGX Spark system, and NVIDIA’s 1.7x claim for two clustered 64 GB units. The $4,999 is NVIDIA’s announced starting price.
• The price of the Luxshare cable and of the breakout cables a 400G switch port needs.
• Whether the ASUS GX10 price on NVIDIA’s marketplace page matches the live page. We read it from a search result.
• The output length, date and vLLM version for botAGI’s context series.
• Input lengths, prompts, context limits, software versions and speculative settings for most rows in Ata’s overview chart.
• LingSpark’s speed with MTP drafting off. The README gives no such figure.
• Warm or cold state for nearly every row. Most sources don’t say.
• Model quality for any of the quantized builds or for Strata’s 5-of-6-expert mode. Ata says quality against the full model was not measured.
• Strata on any other ARM machine, at contexts over 128K, with more than 8 chat slots, or over a long run. Its own doc lists these as untested.
• The speed chart the Strata author posted on X. We did not open it, so we quote none of its numbers.