How the hub and the nodes fit together

Apps know one address and one key. The hub knows your devices, what each chip runs well, and who called what. The nodes do the work and never open a port.

Hub

One static Go binary in Docker (about 17 MB, non-root, no shell). It runs no models: it checks keys, meters, audits and routes.

OpenAI-compatible API
/openai/v1/models, /embeddings (image data URLs; text for text-embedding models), /chat/completions with streaming and the Responses API, with tool calls, run by llama.cpp on your own nodes. The Anthropic Messages API is there too, so Messages API clients can point at the hub. A cloud provider is an optional fallback the operator can configure.
Native API
POST raw image bytes to /v1/models/{id}/run and get labels or a vector back, with servedBy, via and a timing breakdown.
End-to-end sealing
A client can seal a job to the node app that runs it, so the hub, the proxy and Cloudflare relay only ciphertext. The hub still sees who, when, which model and sizes.
Virtual keys
sk-aigw-… keys per app or node: model-glob scopes, per-minute rate limits, daily quotas, instant revocation. Only HMAC-SHA256 hashes are stored.
Audit and usage
One audit entry per request (key, model, node, status, ms, bytes; never bodies or keys), usage per key per day, Prometheus metrics on a private port.
Router
Among online nodes that have the model: loaded first, then the lowest measured latency, then the fewest requests in flight. A transport failure retries once on another node.
Dashboard
Server-rendered, no JavaScript, strict CSP: nodes, models, registry, keys, usage, audit and the waitlist.
How it works: clients call the hub, the hub routes to your devicesClients send requests with an API key to the hub. The hub checks keys, quotas and the audit log and routes each job to the best node. Nodes connect outbound, so no ports are opened at home.YOUR NETWORK · DATA STAYS HEREYour appsOpenAI SDK · Android clientHome AssistantFrigate · automationsScripts & servicescurl · Python · cronHTTPS + keyHubOpenAI-compatible APIKeys · scopes · quotasAudit log · metricsHardware-aware routernodes dial out (wss)Pixel 7 ProGPU · CLIP L/14 1.45 sLiving-room TVNPU · EfficientNet 22 msLinux box · MacCPU · llama.cpp chat models

Nodes

A node is a device running the node app or kinherd-node with a node key. It dials out to the hub over one TLS WebSocket, says what it has, takes jobs and benchmarks itself. It works behind NAT and on mobile data.

DeviceRuntimeMeasured or how it installs
Android phonesLiteRT 2 on the GPU, NPU or CPU.CLIP ViT-L/14-336 at about 1.45 s per image on a Pixel 7 Pro GPU.
Android TVs and boxesLiteRT 1.4 with NNAPI on the NPU; 32-bit builds included.EfficientNet-Lite4 int8 at 22 ms on a MediaTek MDLA (TCL Google TV).
Linux (amd64, arm64)kinherd-node: TensorFlow Lite for image models, llama.cpp’s llama-server for GGUF chat models.Installed with one command, as your own user.
MacThe same kinherd-node, built from source, as a launchd agent.Chat models through llama.cpp on Apple Silicon.

No iOS node. Chat models run on Linux and Mac nodes, not on phones yet.

  • No inbound port: with "listen": false a node has no network attack surface at all.
  • Per-model queue, memory budget, lazy load and idle unload; a full queue answers 429 instead of piling up.
  • Reconnects with jittered backoff, pauses with no network and resumes when Wi-Fi or mobile data returns.
  • On the phone, same-device apps can reach the models over Binder IPC, with no network at all.

Registry

A model × device table: works, works with tweaks (and the tweak), or fails (and why), with latency. Nodes fill it in automatically, a person curates it, and the router uses it.

TCL Smart TV Pro · MediaTek MT9689 · nnapi:mtk-mdla_shim

efficientnet-lite4-int8: works with tweaks, 22 ms

Load the model from a memory-mapped buffer, not a path on shared storage: SELinux blocks the NN HAL from reading FUSE files, and NNAPI then silently falls back to its CPU reference path (~9 s per image).

The first six rows of the registry table: clip-l14-336 works on the Pixel 7 Pro GPU at 1.45 s; EfficientNet-Lite0 and Lite2 int8 work on the TCL TV's NPU at 6 and 9.4 ms; mobilenet-v2-int8 works on the Pixel CPU at 7 ms, the Pixel NNAPI at 0.78 ms and the TV NPU at 4.4 ms.
The registry table on this site’s registry page, sorted by status, first six of eleven rows (snapshot of 18 November 2026).

See the registryHow results are collected

API

Use the OpenAI SDK you already have, or the native API when you want raw bytes in and timings out.

native API
# raw image in, labels out; the hub picks the node
curl -H "Authorization: Bearer $KEY" --data-binary @photo.jpg \
  https://<your-hub>/v1/models/efficientnet-lite4-int8/run

{"model":{"id":"efficientnet-lite4-int8","version":"1.0.0"},
 "output":{"labels":[{"label":"…","score":…}, …]},
 "ms":72,"backend":"nnapi:mtk-mdla_shim","servedBy":"living-room-tv","via":"hub",
 "timings":{"decode":…,"render":…,"pack":…,"invoke":22.0,"postprocess":…}}

OpenAI-compatible APINative API reference

Marketplace (planned, not available yet)

The idea: model makers publish a specialised model (a card recogniser, a pest classifier, a receipt reader) that installs onto the buyer’s own hub, with updates handled. It is off on every hub, and paused while its privacy design is worked out.

What exists today is the package format underneath it: a manifest (inputs, outputs, accelerator preference, memory, licence, SHA-256) plus the model file, signed with Ed25519 by a publisher key that the hub and the node trust. Unknown fields and unsigned packages are refused.

Marketplace not open yet: not available in the beta

Where the hub runs

On your own machine: a server, a Pi, a NAS, a mini PC. Requests and data stay on your network. That is the only way to run it today.

There is no hosted hub and no cloud control plane. Both have been thought about (multi-tenancy, EU data residency); neither is built, and there is no date.

Run a hubFor device makersJoin the private beta