Technical Overview

mlxMesh

A distributed inference protocol turning geographically-spread Apple Silicon into a single, routable AI compute fabric.

AGPL-3.0 Ed25519 Identity No Native Token Apple Silicon Native

What it is

A distributed inference protocol that turns geographically-spread machines into a single, routable AI compute fabric — with strict privacy tiers, measured (not declared) performance accounting, and no native token.

Brand: MeshAI  |  Protocol: Open Inference Mesh  |  CLI: oim

Most distributed inference tools (e.g. Exo) work within a single LAN cluster. oim adds a coordination layer above that: it federates multiple clusters across the internet into a routable mesh, with:

• Dual-lane routing — fast lane for interactive jobs (resolver-routed, low-latency), background lane for recurring/batch jobs (scheduler-routed, sticky-session)
• MoE expert sharding (planner only — not wired into live dispatch) — the only WAN-viable strategy for large models (sequential token passing can't survive 20–150 ms inter-hop latency)
• Division-order accounting — measured resource lines, not declared promises; credits from bootstrap grants decay as earned capacity grows
• Sensitivity tiers — LOW / MODERATE / HIGH_REQUIRES_ATTESTATION (Secure Enclave gate on Apple Silicon)
• Ed25519 node identity — derived from public key, never operator-chosen
• iOS coordination / security layer — iPhone/iPad devices classify on-device and host encrypted payload pointers, adding a privacy layer without becoming compute nodes
• Portable wallet identity — an Ed25519 account key (iCloud-Keychain synced, seed-recoverable) that consolidates credits across a user's devices
• Native clients — SwiftUI apps for iOS/iPadOS, tvOS, and watchOS render live topology, and drive contribution/coordination from Apple hardware

Submitting inference jobs

The mesh is designed for background inference jobs — workloads where you need a response but not within interactive latency. For real-time chat you still want a local Exo cluster; for deferred, cost-sensitive, or burst workloads you route to the mesh.

How a job enters the network

Your application
        |
        | POST /v1/chat/completions   (OpenAI-compatible)
        v
Pod Coordinator (regional)
    |       |
    |       credit check -> dispatch -> node selection
    v
Node Agent (wrapping local Exo)
    |
    |   tokens stream back
    v
Your application

1. Your app checks the caller's credit balance (or the system does it automatically on submit).
2. The coordinator selects a node via the fast-lane router (measured TPS, model availability, sensitivity tier).
3. The node streams the response back through the coordinator.
4. Credits are debited on completion, proportional to tokens delivered.

Submitting a job (OpenAI-compatible API)

cURL:

curl https://<coordinator>/v1/chat/completions \
  -H "Authorization: Bearer <your-api-key>" \
  -H "X-OIM-User-ID: <user-uuid>" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "llama-3.2-3b",
    "messages": [{"role": "user", "content": "Summarize this document: ..."}],
    "max_tokens": 2048,
    "stream": false
  }'

Python (openai SDK):

from openai import OpenAI

client = OpenAI(
    base_url="https://<coordinator>/v1",
    api_key="<your-api-key>",
)

response = client.chat.completions.create(
    model="llama-3.2-3b",
    messages=[{"role": "user", "content": "Summarize this document: ..."}],
)
print(response.choices[0].message.content)

Model parameter reference

Credit / token validator

Before dispatching, the coordinator checks the caller's credit balance:

1. GET /users/{user_id}/balance
2. Estimate job cost = ConsumerCost(lane, sensitivity, max_tokens)
3. If total >= cost -> dispatch. Else -> 402 Payment Required
4. On completion: debit consumer, credit node, book spread to treasury

Cost / reward matrix (the house edge)

All pricing lives in internal/economics so the debit and credit paths can never diverge. The model is a spread, not a transfer: a provider is always paid less than the consumer is charged, and the difference — the house edge (25%) — accrues to the network treasury (oim-treasury, a reserved ledger account). The treasury funds iOS coordination rewards; startup grants and availability rewards are minted directly, not treasury-funded.

consumer_cost    = 1.0 x lane x sensitivity   (credits per 1k output tokens)
provider_reward  = consumer_cost x (1 - 0.25)  <- node earns 75%
network_margin   = consumer_cost - provider_reward <- treasury keeps 25%

lane:        fast x1.0 (interactive premium) . background x0.5 (batch discount)
sensitivity: low x0.5 . moderate x1.0 . high_requires_attestation x3.0

iOS coordination job: a flat 0.02 credits per pointer served, paid to the device's linked wallet account out of the treasury.

Verified availability reward (bootstrap incentive, opt-in)

--availability-reward (off by default) has the coordinator itself act as a tiny, randomly-timed test consumer: every ~10 minutes (jittered), it dispatches one small real inference request through the exact same dispatch path and pricing function real consumer traffic uses, to one of the longest-idle real nodes. No debit, no treasury margin — it's a self-funded subsidy minted directly into the node's account, the same way the startup grant is minted from nothing. The probe throttles against queue backpressure: above 40% saturation, a round is skipped entirely. At low backpressure, the probe budget scales up automatically so idle nodes earn more in a quiet network.

Streaming

Server-side token streaming (stream: true) is implemented on the fast lane — real SSE passthrough at every hop (Exo → node → coordinator → client), with billing read from the trailing usage frame. The background lane intentionally stays buffered/polling.

Quick start

oim wraps Exo as a black-box HTTP API. Apple Silicon (M-series) with unified memory is the reference platform.

git clone https://github.com/american-code/mlxMesh
cd mlxMesh
make install

oim node status
oim node status --exo-url http://localhost:52415 --cap 0.5
oim bench run --model mlx-community/Llama-3.2-3B-Instruct-4bit --prompt medium --samples 3

Configuration

Reachability: outbound work-pull (default)

By default a node receives work the way an ASIC miner receives it from a pool: it opens an outbound connection to the coordinator, long-polls for jobs, runs them via Exo, and posts results back — all outbound. NAT, home routers, port forwarding, UPnP, and firewalls are all irrelevant.

oim node start --coordinator https://us.mlxmesh.net   # pull mode, zero network config

Push mode (opt-in): set an explicit --reachability-endpoint and the coordinator dispatches into the node over HTTP instead.

Running over TLS (HTTPS)

Before exposing anything beyond localhost, turn on TLS — otherwise API keys and job payloads travel in plaintext.

scripts/gen-dev-certs.sh 192.168.1.135
oim-coordinator --listen :9000 --tls-cert certs/server.crt --tls-key certs/server.key ...
oim node start --coordinator https://192.168.1.135:9000 --tls-ca certs/ca.crt

Coordinator to node dispatch over TLS too — nodes are independently operated and self-signed, so instead of chain verification the coordinator pins the exact certificate fingerprint recorded at that node's registration — tamper-evident via the same Ed25519 signature covering the rest of the manifest.

Architecture overview

Global Directory (librarian)    <- M4 (centralized) -> M7 (ledger authority partial; directory federation stubbed)
        |
        v
Pod Coordinators (1 per region)   <- M2
    |           |
    |           Coordination Registry <- M8 (iOS pointer-hosts; never routed to)
    v
Node Agents (wrapping Exo)        <- M1
    ^
    |   live topology + contribution/coordination + wallet
Native clients: iOS . tvOS . watchOS <- M8 / M9

Milestones

Protocol core

Extended scope

• iOS as a coordination/security layer, not a compute node. iOS/iPadOS cannot run Exo, so iOS devices classify on-device and host encrypted payload pointers instead.
• Encrypted-pointer privacy path — client-side P256 ECDH + HKDF-SHA256 + AES-256-GCM; the coordinator sees only a metadata pointer, never plaintext.
• Portable wallet (M9) — credit consolidation + recovery across devices without a blockchain.
• Native Apple apps — live topology, "Try the mesh," network-load/backpressure, coordination layer, wallet, tvOS global-map view.
• Simulation harness — 100+ container multi-region mesh with continuous traffic and live coordination participants.
• Cluster deduplication — prevents double-counting capacity for clustered nodes.
• Warm model support — coordinator can instruct a node to pre-load a model, reducing cold-start latency.
• Observed throughput routing — coordinator tracks actual node performance from completed jobs for routing decisions.

Known gaps

• Node Setup cluster topology is web-only and Exo-driven; schema not yet confirmed against a live non-sim instance.
• Webhook/async callback submission is documented as a target but not implemented.
• M7 federated directory is a stub; FederatedResolver / DHTResolver remain stubs.
• MoE expert sharding is a planner, not wired into any dispatch path. What is wired is query decomposition, background-lane only.

Security model & threat analysis

Because this is (a) open source and (b) a credit-based compute network, the first question is: if anyone can read and fork the code, what stops them from minting credits, spoofing the system, or knocking it over?

Can queries actually flow through? Yes — verified.

The end-to-end path (credit gate → dispatch → node/Exo → response → debit) works today. It is usable as a trusted-coordinator network right now. It is not yet safe as an open, decentralized credit network.

What already holds

• The ledger is server-authoritative. No client or node can write to it directly.
• The write path is Ed25519-signed. You cannot report earnings for a node you don't control, or impersonate another node's identity.
• Grants are Sybil-resistant — proof-of-work nonce (18 bits) plus per-IP rate limiting.
• Wallet auth is challenge-response; device-linking is account-signed.
• Per-IP rate limiting wraps every endpoint; job queue is bounded with backpressure; sensitivity tiers + Secure-Enclave attestation gate sensitive jobs; TLS 1.2+ available; payloads can be client-encrypted.
• Node-to-coordinator calls are resilient — exponential backoff + jitter retry, client-side token-bucket limiter on outbound calls.

Open vulnerabilities (tracked)

1. RESOLVED — Unverified self-reported earnings. Both lanes are now credited from the coordinator's own observed token count. Verified end-to-end (75/25 split, no double-credit).
2. RESOLVED — Earn/debit asymmetry. Debit and credit derived from the same coordinator-observed token count.
3. PARTIALLY ADDRESSED — Federated ledger authority (M7). Coordinator impersonation is now closed (Ed25519 coordinator identity + TOFU pod pinning). Rogue-pod claims are now auditable via signed, sequenced cross-pod ledger-event witnessing. Still open: this is PKI-based trust among a curated/TOFU-pinned set of pods, not Byzantine-fault-tolerant consensus or staking/slashing. Full open, permissionless decentralization remains a stub.
4. DDoS — mostly hardened. Global in-flight concurrency cap, 8 MiB request-body cap, slow-loris guard, SSRF allowlist with per-redirect re-validation and connect-time IP checking (closes DNS-rebinding gap). Remaining: a distributed volumetric flood still needs an upstream WAF/scrubbing layer.
5. RESOLVED — Streaming billing edge case. A stream ending with no usage frame now logs a rejection metric instead of silently billing $0.

Bottom line

Run by a trusted operator (the seed), the credit system is sound against client/node tampering. True open decentralization needs a consensus/staking/proof design that does not yet exist here. A third-party security review of the credit, attestation, and settlement paths has been completed.

Path to release

Security — blocks any public exposure

Safety & correctness — blocks trusting the numbers

Scalability — blocks growth past the seed

Scaling reality check: Fast lane is single-node per job by design. Adding nodes doesn't make individual requests faster — it increases aggregate concurrent capacity. Per-job speed is a property of the selected node, not node count.

Real bottleneck: a single coordinator's SQLite ledger is single-writer. Every job completion does 3+ writes plus signature verification. Realistic sustained ceiling is low hundreds to low thousands of job-completions/sec per coordinator — meaning SQLite write contention, not node compute, is the actual limiter at 10,000+ nodes. Regional sharding (one coordinator per pod) mitigates this, but "Coordinator HA" and "Ledger beyond SQLite" remain the missing pieces for scaling past the seed.

Performance benchmarks — honest positioning vs hosted APIs

Key takeaways:

• mlxMesh per-node throughput is competitive with hosted frontier APIs (Claude Sonnet, GPT-4.1/4o).
• The real gap is latency, not throughput: hosted APIs achieve 30–80ms first-token latency; mlxMesh has extra hop overhead from its multi-region routing path.
• Hosted providers win on aggregate scale via continuous batching across massive GPU fleets.
• mlxMesh trades centralized-batching efficiency for zero per-token cost, no data leaving infrastructure, and no third-party dependency.

Honest positioning: mlxMesh doesn't beat OpenAI/Anthropic on scale or first-token latency, but it achieves tokens-per-second parity with hosted frontier APIs using consumer hardware.

Performance optimizations

Client SDKs

Release engineering

Repository layout

cmd/
  oim/          CLI + node agent entry point
  coordinator/  Pod coordinator server (M2)
  directory/    Global directory server (M4)
  stub-exo/     Fake Exo for simulation
internal/
  protocol/     Wire types, crypto, job specs
  exoadapter/   Thin HTTP client wrapping Exo
  agent/        Node agent HTTP server
  jobrunner/    Executes jobs against local Exo
  governor/     Resource caps and foreground check
  capability/   Live manifest assembly
  bench/        Tier benchmarking
  attestation/  Secure Enclave attestation verification
  identity/     Ed25519 (+ ECDH) keypair persistence
  coordinator/  Registry, routers, queue, coordination registry (M8)
  wallet/       Portable account identity (M9)
  directory/    Resolver interface + PinStore pod pinning
  federation/   Signed cross-pod ledger-event witnessing + audit (M7)
  settlement/   Division-order ledger, startup-grant PoW
  economics/    All pricing: cost/reward matrix, house edge
  payloadcrypto/ ECDH-P256 -> HKDF-SHA256 -> AES-256-GCM
  httptls/      TLS serving helpers, fingerprint-pinned clients
  httpmw/       Shared HTTP middleware
  httpx/        Outbound HTTP resilience
  sse/          Shared SSE relay
  metrics/      Prometheus-format counters/gauges
  nodeconfig/   Node YAML config load + validation
dashboard/      Web dashboard (React + Vite)
landing/        mlxmesh.net landing page
OIMDashboard/   SwiftUI apps — iOS / tvOS / watchOS (M8/M9 clients)
tests/          Protocol-, coordination-, integration-level tests

Privacy model

No prompt content, no raw embeddings, no biometric data ever leaves the device.

Why no token?

Protocol credits are off-chain (stablecoin / fiat payment rails). Issuing a native token risks the Helium-style collapse pattern: token price speculation decouples from actual compute supply, then crashes when market sentiment shifts. Bootstrap grants are per-pod, keyed to verified capacity, and decay as earned revenue grows.

Marketplace feasibility

The current ledger architecture does not support a credit marketplace. Building one would require: a credit transfer mechanism (atomic transfers between accounts), external monetary value (a pricing mechanism), a trust model (centralized escrow or blockchain-based trustless transfer), and regulatory compliance (KYC/AML) if real money is involved. The ledger is intentionally centralized and append-only, with no transfer capability today.

License

This project is licensed under the GNU Affero General Public License v3.0 (AGPL-3.0).

Open source use: Free to use, modify, and distribute under AGPL-3.0. Commercial use: Requires a separate commercial license for proprietary SaaS offerings, integration into commercial products without releasing source, or enterprise use without AGPL compliance.

The AGPL-3.0 requires that source code modifications be made available to users of the software, that network users (SaaS) have access to the source code, and that derivative works also be licensed under AGPL-3.0.

For commercial licensing: jmelton@americancode.org