Distributed Inference Infrastructure Built on Apple Silicon Unified Memory
The AI industry is capitalizing infrastructure for a narrow slice of its own usage. Hundreds of billions of dollars are being committed to hyperscale data center buildout — much of it debt-financed, much of it behind schedule, and much of it sized for a real-time, frontier-model interaction pattern that represents a minority of how AI is actually being used in production today. Meanwhile, a large and growing share of real-world AI workloads are asynchronous, latency-tolerant, and well within the capability of small, purpose-fit models — the kind of jobs that don't need a $44-million-per-megawatt data center to run.
At the same time, millions of Apple Silicon Macs — machines with genuinely fast unified memory architecture — sit idle overnight, on desks, in labs, and in closets, contributing nothing to this equation.
mlxMesh is a public, open-source mesh network for AI inference, purpose-built for Apple's M-series unified memory architecture and built on the foundation laid by Exo. It reframes the infrastructure question: instead of asking "how do we build more centralized compute," it asks "how do we make the compute that already exists legible, poolable, and trustworthy enough to use." This paper lays out the case for that approach, the architecture behind it, how it compares to the existing decentralized-compute landscape, and why the specific combination of technologies mlxMesh uses — division-order-style contribution accounting, Ed25519 wallet identity, MQTT-based backpressure, true mesh networking, MLX-native unified memory execution, Exo clustering, end-to-end encryption, and iOS devices as a secure coordination layer — represents a materially different approach than the compute marketplaces that came before it.
Building a modern AI data center costs roughly $44 million per megawatt — about $30 million in servers and GPUs plus $14 million or more in construction — and a single facility can run hundreds of megawatts, meaning total costs reach into the billions before a single paying customer signs on. Much of this is now underwritten by debt: $178.5 billion in data center credit deals closed in 2025 alone, with most of that debt rated junk grade.
Despite headline claims of gigawatts of capacity coming online, real buildout is lagging badly behind announcements. Independent tracking suggests only about 3GW of new IT load came online in the US in 2025, against roughly 5GW under construction worldwide — a fraction of what's been publicly promised. Even simple facilities move slowly: a 1MW edge data center in Raleigh, NC — a fraction the size of the gigawatt campuses now being pitched — took eleven months to build, before accounting for land acquisition, permitting, and other delays.
The revenue side compounds the risk. Much of the customer base financing this debt-fueled buildout consists of AI startups that are themselves unprofitable, creating a chain where lenders' repayment depends on companies that can't yet cover their own costs. If announced capacity fails to materialize at the promised scale, hundreds of billions of dollars — much of it channeled through private credit backed by pensions, retirement funds, and insurers — will have been misallocated.
The International Energy Agency (IEA) provides the clearest picture available of the physical footprint behind this buildout:
There is a second, less-discussed cost to centralization: dependency risk at the level of individual human capability. Developers increasingly work in a mode where an LLM generates syntax while the developer supervises logic, architecture, and intent — a legitimate and often more productive way to work. But this creates a quiet single point of failure. When a centralized coding assistant goes down, an affected developer isn't merely inconvenienced — they may be working in a language, framework, or codebase pattern they don't have deep independent fluency in, generated by a tool that is now unavailable. They have the map but not the muscle memory. This is a new category of infrastructure risk: not "the app doesn't work," but "a human capability was quietly outsourced to a service that just went offline." A distributed mesh with local and regional inference doesn't just add redundancy for cost reasons — it's a continuity-of-capability argument. If inference can fail over to a local or regional pod instead of a single hyperscale provider, the person relying on it never hits that cliff.
A common justification for the pace and scale of US AI infrastructure spending is that it is a necessary response to strategic competition with China. The data suggests this framing overstates the closeness of the race, even as it understates a different, more grounded set of concerns.
By capacity, the US is not in a close race. US technology companies' capital expenditure on AI infrastructure surpassed $400 billion in 2025, compared to $63 billion in China, and is projected to exceed $800 billion in 2026 — roughly 12–13 times China's spend. US data center capacity stood at over 50 GW at the end of 2025, versus approximately 31 GW in China and 12 GW in the EU. One industry estimate puts the US's AI-optimized computing power (measured in H100-equivalent GPUs) at roughly 8 times the size of China's. By nearly every infrastructure metric, this is a decisive US lead, not a close contest.
China's own strategy does not attempt to match this scale. China's five-year plan commits approximately $295 billion over five years to a state-led computing buildout — a fraction of what a single year of US hyperscaler capex now represents. China's approach instead leans on cheap domestic electricity, state-directed siting of data centers in its western provinces, and increasing reliance on domestic chips (Huawei, Cambricon, SMIC) in response to US export controls on advanced Nvidia hardware. This is a strategy built around the inputs China already controls, not an attempt to outbuild US hyperscale capacity gigawatt for gigawatt.
The race framing itself has on-the-record critics. Some policy researchers have argued that the "race with China" narrative functions in part as a pretext to justify deregulation and favorable conditions for AI companies, independent of how close the underlying competition actually is. Separately, market analysts — including NYU's Scott Galloway — have compared the current pace of US capex to prior technology overinvestment cycles, warning that the industry may be building five years' worth of infrastructure capacity into a two-year window, a pattern that has historically preceded overcapacity and valuation corrections.
The more defensible synthesis: the US is funding an infrastructure race against a competitor that, by its own stated strategy, is not attempting to match that scale. A distributed, open-source mesh sidesteps this dynamic by design. It does not require winning a capital-expenditure arms race to be useful, and because it is open-source from inception, there is no proprietary advantage at stake to lose in the first place.
Public discourse about AI value — and nearly all current infrastructure investment — is implicitly modeled around real-time, frontier-model, chat-style interaction. That is real value. But it is a minority of what production AI usage actually looks like, or will look like as adoption matures.
A large and growing share of industrial AI usage is background, asynchronous, and does not require frontier capability. A representative example: a scheduled job that polls SQL Server telemetry every few minutes and passes the metrics to a small model (e.g., Qwen 3B) for interpretation — flagging anomalies, recommending query optimizations, or suggesting execution plan changes. This pattern generalizes across log triage, anomaly scoring, batch summarization, and alert enrichment: anything that runs on a schedule rather than in a chat window. None of this needs GPT/Claude/Gemini-class frontier latency or scale. It needs "good enough," cheap, and reliably available — exactly the profile a mesh of distributed, smaller, specialized models is well-suited to serve.
This mismatch has a clean physical analogy. An EV under good conditions runs about 250–300 Wh per mile. Under real-world stress — cold weather, highway speeds, active climate control — that can climb to 500–600 Wh per mile or more; real-world owner data confirms Wh/mi commonly rises from a 250 Wh/mi baseline at 60mph to 325+ Wh/mi at 80mph, with another 25–50 Wh/mi added in extreme heat or cold. An e-bike, by contrast, covers 30–50 miles on a single kWh, largely independent of weather, because it isn't fighting the aerodynamic drag, rolling resistance, and climate-control load of a multi-thousand-pound vehicle. On a hard day, that same 1 kWh moves an EV roughly 1.7–2 miles, while it moves an e-bike rider 40 miles — a 20x-plus efficiency gap between two vehicles solving very different problems with the same unit of energy.
The current AI infrastructure build-out is, in effect, issuing everyone a semi truck because some trips genuinely need one — while most trips could run on a bicycle, and the truck would still be there for genuine freight work. A frontier model in a hyperscale data center is the semi truck: necessary for novel reasoning, huge context windows, and genuinely heavy multi-modal work. A small model on a cron job reading server telemetry is the e-bike: right-sized for a narrow, repeated, latency-tolerant job. The industry is currently capitalizing infrastructure for the semi-truck use case while a large share of real usage is bicycle-shaped.
This workload mismatch pairs with a broader and more contested question: how much of the AI value proposition is genuine displacement versus genuine augmentation? The research picture here is mixed and worth presenting honestly rather than cherry-picked.
MIT Sloan's "The EPOCH of AI: Human-Machine Complementarities at Work" (March 2025) distinguishes automation (task transfer to machines) from augmentation (AI enhancing human productivity), and argues many tasks benefit more from the latter. The research found that human-intensive tasks not only persisted but grew in frequency between 2016 and 2024, and that newly created job tasks in 2024 showed higher levels of human-intensive characteristics than the tasks that disappeared that year.
A related MIT CSAIL study on economic — not just technical — limits to automation found that even where AI is technically capable of a task, it is often not economically viable to deploy: only about 23% of wages tied to vision-based tasks were found to be cost-effective to automate with AI at then-current costs, meaning capability and adoption follow different curves.
This is not the whole picture, and a credible whitepaper should say so. A more recent Stanford analysis (cited via MIT Technology Review, May 2026) found a 16% decline in entry-level jobs in AI-exposed occupations after controlling for other factors, and a separate MIT labor-exposure index estimated that current AI systems could technically already replace roughly 11.7% of the US workforce by task-mapping — though the authors caution this is a capability snapshot, not a prediction of actual job loss. The honest synthesis: the picture is contested, not settled, and the "right tool for the right job" framing (Section 2.1) is a better foundation for infrastructure planning than either "AI replaces everything" or "AI replaces nothing."
Set against this backdrop of overbuilt, debt-financed, energy-hungry centralized infrastructure is a simple and underexploited fact: a very large share of existing consumer and prosumer compute sits idle. One widely cited estimate puts global GPU utilization at under 60% at any given moment — meaning over 40% of global GPU capacity is idle at any point in time, spread across gaming rigs, creative workstations, and leftover mining-era hardware.
Apple Silicon is a particularly strong candidate for this untapped pool, for reasons that go beyond simple idle-cycle harvesting:
mlxMesh's core wager is that this idle, high-bandwidth-memory hardware fleet — activated overnight, coordinated intelligently, and paid fairly — is a better foundation for the inference layer of AI infrastructure than another gigawatt-scale data center campus.
mlxMesh does not start from zero. It builds directly on Exo, which demonstrated that consumer Apple Silicon devices can be clustered to run models too large for any single machine, using peer-to-peer coordination rather than a centralized orchestrator. mlxMesh extends this foundation with WAN-scale mesh coordination, MoE expert-sharding, and a purpose-built incentive and identity layer, described in Section 5.
The idea of pooling idle consumer hardware for meaningful compute is not new, and has a strong, decades-long track record:
This history matters for mlxMesh's credibility: pooling distributed consumer hardware for serious computational work is a proven model with a 25-year track record, not a speculative new idea.
More directly relevant to mlxMesh's architecture, recent academic work has explored the exact pattern of mixture-of-experts execution across decentralized, volunteer-style hardware:
This gives mlxMesh's MoE expert-sharding approach a peer-reviewed academic foundation, not just an engineering hypothesis.
mlxMesh combines several distinct technical layers into a single coherent system. Each layer solves a specific problem that, taken individually, exists in prior art — but the combination is what differentiates the architecture from every existing decentralized compute network (see Section 7).
This hierarchy avoids the two failure modes common to naive peer-to-peer designs: it doesn't require every node to know about every other node (which doesn't scale), and it doesn't rely on a single centralized coordinator (which reintroduces the single-point-of-failure problem this whole project exists to avoid).
Node agents run inference natively against Apple's MLX framework, taking direct advantage of unified memory architecture rather than treating Apple Silicon as a generic accelerator. This is a deliberate scope narrowing rather than a limitation.
mlxMesh uses Exo's clustering approach as the substrate for coordinating multiple physical machines into a single logical inference target, extending it with WAN-aware routing rather than assuming all nodes are on the same local network.
Rather than requiring an entire model to reside on a single machine, mlxMesh shards mixture-of-experts models across nodes, routing individual inference requests to the specific experts needed rather than replicating full models everywhere. This is what allows the network to serve large models using hardware that individually could never hold the full model in memory, and it is grounded in the academic work described in Section 4.2.
Node health, congestion, and availability are communicated across the mesh using MQTT-based backpressure signaling, allowing the network to make real-time routing decisions based on actual network conditions rather than static assumptions about node capacity. This is a structural difference from GPU rental marketplaces (Section 7), which isolate every job to a single host and have no mechanism for network-aware, multi-node request routing.
iOS devices, backed by Secure Enclave, serve as a trusted coordination and identity layer for the mesh — anchoring device identity and authorization in hardware-backed security rather than software-only credentials. This is paired with end-to-end encryption across the mesh, so that inference requests and responses traveling between nodes are not exposed to intermediate coordinators.
Each participating node is identified by an Ed25519 keypair functioning as a portable wallet identity, used both for authentication and as the basis for the settlement mechanism described in Section 6. This is a deliberate departure from the rest of the decentralized-compute field: mlxMesh does not issue a native token and does not settle contribution on-chain. Contribution and settlement are tracked on an off-chain ledger tied to each node's Ed25519 identity, avoiding the failure mode common to token-based compute networks, where network security and participant incentives end up funded by market speculation in the token itself rather than by the value of the compute actually being provided. Node identity also carries forward across sessions and devices as a portable credential, rather than being re-established per job the way a marketplace account typically is.
Additional trust mechanisms reinforce this identity layer: trust-on-first-use certificate pinning (the same model used by SSH) for node-to-node authentication after initial contact, and a Hashcash/Bitcoin-style proof-of-work requirement for bootstrap credit issuance, which raises the cost of Sybil attacks (an attacker creating many fake identities to claim disproportionate rewards) without requiring a token market to do it.
mlxMesh routes requests through two distinct lanes rather than a single undifferentiated queue: a fast/interactive lane for latency-sensitive requests, and a background/batch lane for asynchronous, latency-tolerant jobs — the workload class described in Section 2. This lets the network serve both use cases from the same pool of nodes without one workload type degrading the other.
Not all inference requests carry the same sensitivity, and mlxMesh treats this as a routing input rather than an afterthought: requests can be tagged with a sensitivity tier, and higher-sensitivity tiers are gated to nodes that can attest to Secure Enclave-backed hardware security, rather than being routed to any available node in the mesh.
mlxMesh's incentive layer borrows a structure from real-world revenue-interest accounting: the division order. In traditional industries where multiple parties hold a fractional interest in a single output (a common model in oil and gas royalty accounting, for example), a division order specifies exactly how proceeds from that shared output are split among contributors. mlxMesh applies the same logic to inference: when a single request is served by multiple nodes contributing different experts or pipeline stages, each contributing node's share of the resulting payment is calculated proportionally and auditable — not estimated, not flattened into a generic per-hour or per-token rate that ignores who actually did the work.
This is a meaningfully different incentive primitive than anything currently used in the decentralized compute space. Akash and Golem use straightforward marketplace or task-based billing tied to a single provider per job. Bittensor uses algorithmic reward distribution for subnet-level output quality, not a proportional accounting of multi-party contribution to a single output. None of them are designed for the case mlxMesh is built around: several physically distinct nodes jointly producing a single inference result.
Earlier architectural work surfaced specific trust vulnerabilities common to naive decentralized compute designs: unauthenticated credit minting, self-reported enclave/security status, and in-memory-only deduplication that could be gamed. mlxMesh's design addresses these directly rather than assuming good-faith participation:
The decentralized compute space — much of it blockchain-based — is more mature than it might appear, and mlxMesh should be positioned honestly against it rather than presented as having no competition. Notably, mlxMesh itself is not blockchain-based; it is included here as a decentralized compute network, not a token or crypto project.
| Capability | Akash | Vast.ai | Golem | Bittensor | Surplus Intelligence | mlxMesh |
|---|---|---|---|---|---|---|
| What it actually sells | Container leases on GPUs | Raw GPU instance rental | Task-split compute jobs | Incentivized ML outputs | Resold API credits (not hardware) | Coordinated inference across a live mesh |
| Unit of work | Whole GPU / container | Whole GPU instance | Subtask | Subnet-defined task | API call | Model-shard / expert-routed request |
| Hardware model | Heterogeneous, unvetted | Heterogeneous, reliability-scored | Heterogeneous | Heterogeneous, subnet-specific | N/A (software layer only) | Apple Silicon unified memory, MLX-native |
| Interconnect awareness | None — jobs run in isolation | None | Basic subtask splitting | None | None | MQTT backpressure + mesh networking, live network-aware |
| Cross-node model execution | No | No | Limited | No — separate miners per subnet | No | Yes — MoE expert-sharding across nodes for one request |
| Incentive layer | On-chain token (AKT), escrow | Fiat, per-second billing | Token (GLM), usage-based | Token, algorithmic quality reward | Fiat credits | Off-chain, division-order-style proportional, auditable settlement — no native token |
| Identity layer | Blockchain wallet | Platform account | Blockchain wallet | Blockchain wallet | Platform account | Ed25519 portable wallet identity + iOS Secure Enclave coordination |
| Security posture | Platform SOC2 only, not hosts | Platform SOC2 only, not hosts | Minimal | Minimal | Standard API auth | End-to-end encryption, hardware-anchored identity, PoW Sybil resistance |
| Target workload | Training + general containers | Training, fine-tuning, batch inference | General task computing | ML subnet competitions | API arbitrage | Inference-first, latency-tolerant, MoE-shardable |
Where mlxMesh is genuinely differentiated:
An honest tradeoff worth stating directly: targeting Apple Silicon specifically is simultaneously mlxMesh's core technical differentiator (unified memory bandwidth advantages for MoE-style workloads) and its biggest adoption constraint relative to Akash, Vast.ai, and Golem, all of which are hardware-agnostic and already have large NVIDIA GPU fleets onboarded. This is a deliberate strategic bet, not an oversight: there are many millions of M-series Macs in the world, a large share of them sitting fully idle overnight, and none of the existing networks are built to address that specific pool.
A note on frontier training vs. mesh inference: decentralized networks, including mlxMesh, are not positioned to replace frontier-scale training clusters — tightly-coupled training across 100,000+ top-tier GPUs has interconnect requirements that a WAN-based mesh cannot realistically meet. This sharpens rather than weakens mlxMesh's positioning: the target is the inference layer, and specifically the async, latency-tolerant workload class described in Section 2, not a head-on competition with hyperscale training infrastructure.
mlxMesh is built as open-source infrastructure from the outset, on the premise that a public compute mesh should be auditable by the people whose hardware and data pass through it.
The project is licensed under AGPL-3.0. Free use, modification, and distribution are permitted under those terms — network users (SaaS operators) get access to the source, and derivative works remain AGPL-3.0. Commercial use outside those terms (proprietary SaaS integration, closed-source products, or enterprise deployments without AGPL compliance) requires a separate commercial license. Contact jmelton@americancode.org.
During its early phase, the project operates under a BDFL (Benevolent Dictator For Life) governance model. The project maintainer is Jacob Melton at American Code (jmelton@americancode.org). This is the period where architectural decisions are made quickly and with full accountability — a model appropriate for a project that is still stabilizing its core protocol.
The repository is publicly available at github.com/american-code/mlxMesh. Bug reports, security disclosures, and pull requests are welcome. Security issues should be reported privately to jmelton@americancode.org rather than as public issues — see SECURITY.md for the full disclosure policy and verification instructions.
Any Apple Silicon Mac running Exo can join the mesh. Nodes connect outbound to the coordinator — no NAT traversal, no port forwarding, no firewall rules required:
oim node start --coordinator https://us.mlxmesh.net
Nodes are independently operated. The project maintainer does not control individual node hardware, data, or earnings. Credits are tracked per Ed25519 node identity and earned proportionally to verified compute delivered.
As the network grows and its economics stabilize, governance will evolve toward a more formally participatory model. The intent is that a public compute mesh should remain publicly governed — with the protocol itself serving as the authoritative layer rather than any single operator's word. The specific mechanism (a technical steering committee, an on-chain vote, a foundation structure, or something else) will be determined as the community of node operators and contributors grows large enough to make it meaningful rather than performative.
| # | Status | Description |
|---|---|---|
| M1 | Done | Node agent: manifest assembly, resource governor, bench, Ed25519 identity |
| M2 | Done | Pod coordinator: registry, fast-lane router, background scheduler, job queue, rate limiting |
| M3 | Done | Spot-check verification, tier-claim validation, measurement store |
| M4 | Done | Centralized global directory with gossip sync and cache fallback |
| M5 | Done | Division-order settlement ledger with SQLite persistence, startup grants with PoW |
| M6 | Done | MoE expert-shard planner with proportional assignment and load imbalance detection |
| M7 | Partial | Federated directory: PKI-based pod witnessing and cross-pod signed-ledger-event audit live in production. Full permissionless decentralization (BFT consensus, open pod registration) remains a future milestone. |
| M8 | Done | iOS coordination/security layer: CoreML classifier, P256 ECDH payload encryption, coordination registry, native SwiftUI apps for iOS/tvOS/watchOS |
| M9 | Done | Portable wallet identity: Ed25519 account key, challenge-response auth, iCloud-Keychain sync, Base32 seed recovery |
The industry's current answer to rising AI demand is more concentration: more debt-financed data centers, more centralized dependency, more capital chasing a workload profile that doesn't match how most AI is actually used.
mlxMesh is a bet on the opposite answer — that the compute already sitting idle in millions of homes and offices, coordinated intelligently and paid fairly, is a better foundation for the next phase of AI infrastructure than another gigawatt campus. The project is open source, the architecture is documented, and the mesh needs nodes.
This whitepaper draws on public reporting and research from the IEA, MIT Sloan, MIT CSAIL, Brookings, and independent technology journalism, alongside academic literature on volunteer and decentralized computing. Figures cited reflect publicly available data as of mid-2026 and are subject to revision as the underlying research evolves.