Mac Studio Cluster for Local AI: exo, RDMA and Spark + Mac

You can build a Mac Studio cluster for local AI with exo, a tool that splits one model across several Macs. The Macs talk over Thunderbolt 5 cables using RDMA, which needs macOS 26.2 or later. RDMA (remote direct memory access) lets one Mac read another Mac’s memory with much less waiting. The main reason to do this is to fit a model that is too big for one Mac.

Apple’s own clustering app, Claris Harbor, is in private preview and can’t be downloaded yet. Our Claris Harbor guide covers what is confirmed and which Macs have Thunderbolt 5.

Keep two limits in mind. First, the memory of all the Macs adds up on paper, but you can’t use all of it as one block for one model. Each Mac shares its memory with macOS, and the model is split across machines. Second, the cluster results below come from a few setups that Apple loaned or supplied to reviewers. We have not tested a cluster ourselves.

For a single Mac, My Rig shows which models fit and our speed estimates.

Four ways to run a model bigger than one machine

People use four different setups when a model is too big or too slow for one box. They work in different ways, so their results don’t compare directly.

1. In-memory cluster. Several machines hold the model in their combined memory. The model can be split two ways. Pipeline splitting gives each machine a share of the layers. Tensor splitting divides the work inside each layer across the machines. The exo README documents both modes. Examples: exo or MLX across Macs, or vLLM across two DGX Sparks. The two-Spark examples in that guide use a tensor split across both machines (TP=2). Most of this guide covers this case.
2. Offload on one machine. Part of the model sits in GPU memory and the rest sits in system RAM, by layer or by expert. This guide does not cover it.
3. SSD-streamed runtime on one machine. The runtime keeps part of the model in memory and reads the rest from the SSD as it runs. Strata does this. It is that runtime’s own design. It is a different thing from the operating system’s swap file. A big enough disk does not make any model runnable, and the download size is not the amount of memory a model needs. See One Mac with SSD streaming.
4. Split prefill and decode. One machine reads the prompt and another writes the reply. The exo team showed this with a DGX Spark and a Mac Studio. See Experimental: DGX Spark + Mac.

What you need

These rules come from the exo README, checked October 28, 2026, plus notes from Jeff Geerling’s review.

• A Mac with Thunderbolt 5. The README says RDMA works on any Mac with Thunderbolt 5. It lists the M4 Pro Mac mini, M4 Max Mac Studio, M4 Max MacBook Pro and M3 Ultra Mac Studio.
• macOS 26.2 or later on every Mac. The exo macOS app needs 26.2 or later too.
• RDMA turned on. You boot each Mac into Recovery and run rdma_ctl enable.
• Every Mac cabled to every other Mac. This layout is called a full mesh. Geerling found no Thunderbolt 5 switches, which is why each pair needs its own cable.
• On a Mac Studio, skip the Thunderbolt 5 port next to the Ethernet port. The README says it can’t be used.
• The exact same macOS version on every Mac, down to the beta number.

Geerling wrote that, right now, you can only cross-connect up to four Macs. He also wrote that Apple told him all five Thunderbolt 5 ports are RDMA-enabled. He measured Thunderbolt at about 50 to 60 Gbps in real use.

The exo README does not list M5 Macs as RDMA-capable. Its only M5 mention is a note that the Homebrew build of macmon 0.6.1 crashes on Apple M5. RDMA on an M5 Mac Studio is unverified.

MLX distributed

exo uses MLX, Apple’s machine learning framework, to run models on the Macs. The MLX distributed docs (MLX 0.32.4) describe how MLX itself runs across several machines:

• MLX has four backends: MPI, ring (TCP, including Thunderbolt rings), JACCL and NCCL (for CUDA).
• JACCL is RDMA over Thunderbolt. It needs macOS 26.2 or later and Thunderbolt 5. The docs say it is needed for things like tensor parallelism.
• JACCL only works when every pair of machines has its own cable (a full mesh).
• Two Macs can use more than one Thunderbolt cable. JACCL sends over all of them at once.
• You turn on RDMA in Recovery with rdma_ctl enable. It can’t be done remotely.
• mlx.distributed_config helps you set up the machines. mlx.launch runs the job, for example an mlx_lm chat over JACCL.

Published cluster results

Two reviewers tested the same kind of cluster: four Mac Studios with the M3 Ultra chip. Two had 512 GB of memory and two had 256 GB, for 1.5 TB in total. That 1.5 TB is the physical total across four machines.

Tok/s means tokens per second, and a token is a short piece of a word. TTFT (time to first token) is how long you wait before the reply starts. Quant is how much the model’s numbers are compressed, such as 8-bit or 4-bit. “Unknown” means the source did not state it. Each source gets its own table because the setups differ. Don’t use these tables to rank models, quants or tools, or to predict your own speed. Our methodology page explains how we estimate speed on a single machine.

Jeff Geerling

Jeff Geerling (December 18, 2025) used four Mac Studios that Apple loaned him. He put the cost at about $40,000 as configured: $11,699 for each 512 GB Mac and $8,099 for each 256 GB Mac. He ran prerelease software and hit stability issues. At the time, exo was the only tool he found that supported RDMA. He found that RDMA cut latency (the wait for each message between Macs) from about 300 µs to under 50 µs. He also tested llama.cpp’s RPC mode, another way to split a model across machines over a network. Per Geerling, it spreads the model’s layers across the machines. It got slower as he added Macs.

Test conditions for the table below:

• Hardware: up to four Mac Studio M3 Ultra, Apple-loaned. The cluster mixed 256 GB and 512 GB Macs. Which Macs ran the one-node and two-node tests is Unknown.
• Models and quants, per the chart captions in the exo repo: Qwen3 235B in 8-bit, DeepSeek V3.1 671B in 8-bit, Kimi K2 Thinking in native 4-bit.
• Tools: llama.cpp RPC over TCP, and exo with tensor parallel over RDMA. Versions Unknown, prerelease software.
• Prompt length, output length, context, warm or cold start, and draft model use: Unknown.
• Metric: tok/s as charted. Whether it is decode speed, and how it was timed: Unknown.
• Evidence status: third-party review on loaned hardware. We read the values from Geerling’s chart images, which the exo repo reproduces.

The chart has no one-Mac value for Kimi K2 Thinking.

Creative Strategies

Creative Strategies (Max Weinbach, December 18, 2025) used Apple-supplied review systems, per the author. Each Mac had a 32-core CPU and 80-core GPU. The author put the total at about $39,596.

Test conditions: model Kimi K2 Thinking in native int4. Tool: exo, which the author says he used. exo version: Unknown. Prompt length, output length, context, warm or cold start and draft model use are Unknown. The metric type behind the tok/s figure is Unknown. Evidence status: third-party review on Apple-supplied hardware.

exo project claim

The exo README claims tensor parallel gives “up to 1.8x speedup on 2 devices and 3.2x on 4”. It does not state the hardware, model or test conditions. This is a project claim. We have no measured result behind it.

One Mac with SSD streaming

This is the third setup in the list above. Strata is a project on GitHub. Its Mac engine runs DeepSeek-V4.1-Flash on one Mac by keeping part of the model in memory and reading the rest from the SSD for each token. On a 128 GB Mac it keeps about 68 GB of the model’s experts in memory. It reads 48 MB (2.00 bpw) or 111 MB (2.25 bpw) from the SSD per token. Bpw (bits per weight) is how compressed the model is.

Version: Strata 0.2.7.1 (release tag v0.2.7, published October 27, 2026). The app stays at 0.2.7. Control is 0.1.53. The new DeepSeek engine for Macs is 0.1.53. Mac users who only run Qwen get no new engine and no speed change. Plans in the release notes for other hardware are announcements, so we don’t treat them as results.

The Strata DeepSeek on Mac doc lists these needs: Apple Silicon M1 or newer, macOS 15 or later, and 257 GB (2.00 bpw) or 274 GB (2.25 bpw) of space on an internal SSD or NVMe drive. USB and network drives are not supported. It lists 64 GB of memory as the minimum and says less still starts but runs much slower. Only a 128 GB Mac was measured, so smaller Macs have no results.

Test conditions for both tables:

• Hardware: one M5 Max Mac with 128 GB. Nothing under 128 GB was run.
• Model: DeepSeek-V4.1-Flash at 2.00 bpw and 2.25 bpw. Text only.
• Engine: Strata 0.2.7.1 build, Metal engine, shipped defaults.
• Expert mode: full quality, all six experts per token. The DSpark draft model was off.
• Runs: two fresh-process runs per test. Warm or cold start: Unknown. Output length: Unknown.
• Machine state: another person’s idle 60 GB Qwen server was loaded on the same Mac and shared its file cache. This is not a clean-machine benchmark.
• Context: prompts up to about 32K tokens. Longer contexts were not measured.
• Evidence status: measured by the Strata maintainers. No outside check.

Answer speed (single-user decode, tok/s, range over the two runs):

Prompt reading speed (prefill, tok/s):

The 720 tok/s figure is prompt reading at 32K with 2.25 bpw. It is not answer speed. The doc also reports KL divergence from the original model of 0.170 (2.00 bpw) and 0.111 (2.25 bpw). KL divergence measures how far a quant’s output drifts from the original. Lower means closer.

The same release added a DeepSeek engine for the DGX Spark. Our DGX Spark cluster guide covers it in its SSD-streamed section.

Mac Studio prices today

We checked these two Mac Studio configs on apple.com. They match the prices in our data.

The clusters above used M3 Ultra Mac Studios with 256 GB and 512 GB of memory. We have not seen published RDMA cluster results on M5 Mac Studios. We also have not verified prices for higher-memory M5 configs, so we don’t list them and don’t suggest buying them for a cluster.

Experimental: DGX Spark + Mac

This is the fourth setup in the list above. The NVIDIA DGX Spark is a small NVIDIA box with 128 GB of memory and 273 GB/s of memory bandwidth, per our data. The exo team showed a way to pair it with a Mac Studio. They call it disaggregated prefill and decode, which means the two machines split the two stages of a reply:

• Prefill is reading your prompt. The Spark does this part.
• Decode is writing the reply one token at a time. The Mac does this part.

While the Spark reads the prompt, it sends the KV cache to the Mac layer by layer. The KV cache is the model’s saved working data about the prompt. The Mac then writes the reply from it.

This setup splits the work by stage. It does not give you combined memory capacity for a bigger model.

The exo demo

The numbers below come from the exo blog. The team used early-access DGX Sparks, EXO 1.0, and a Mac Studio with the M3 Ultra. The model was Llama-3.1 8B in FP16, which means 16-bit weights with no compression. The prompt was 8,192 tokens and the reply was 32 tokens. Concurrency, speculative decoding settings and the test date are Unknown.

The exo team reports this as 2.8x faster. That figure compares total time with the M3 Ultra alone. Generation took 0.85 s on the Mac in both cases, so the decode speed itself did not change. The gain came from the Spark reading the prompt faster.

Their math assumes a 10 GbE link (10 gigabit Ethernet) between the machines. It says the gains need long prompts and enough network bandwidth. Depending on the model’s attention design, full overlap needs prompts of roughly 5K to 40K tokens. Our tok/s vs agent jobs guide explains why prompt reading time matters for long agent runs.

Why we can’t call it a recipe yet

The Spark runs Linux. The exo README says exo currently runs on CPU on Linux and that GPU support for Linux is under development. Its install notes also mention a CUDA option (--extra mlx-cuda13). Those two statements don’t line up, so we can’t confirm a Spark + Mac setup you can reproduce today.

There are community reports of people trying Spark + Mac over a 2.5 GbE link. We could not open the original source, so we don’t quote any numbers from them.

We also found no documented RDMA link between a Spark and a Mac. The Spark’s fast network ports are ConnectX-7 QSFP Ethernet. The Mac’s are Thunderbolt 5.

What we haven’t verified