Quickstart
You run a hub (one small Docker container), give it a model, add a node (a Linux machine or an Android phone you already own), and call it with curl or an OpenAI SDK. Requests go to your hub and run on your node; nothing leaves your network.
your app / curl / OpenAI SDK ──HTTP + client key──▶ hub (Docker) ◀──outbound WebSocket── node (Linux box, phone, TV)You need Docker with Compose, curl, openssl, and the small aigw admin CLI (aigw-<os>-<arch> from the
latest release, or built with Go 1.27 or newer as below).
1. Run the hub
Section titled “1. Run the hub”git clone <repository-url> iot-ai-gateway && cd iot-ai-gateway/servergo build -o ~/.local/bin/kinherd ./cmd/kinherd # admin CLI: keys, signing, pairing codeskinherd keygen -out ~/.aigw -name publisher # your publisher key: it signs the models your nodes may runcp .env.example .envsed -i.bak -e "s/^AIGW_ADMIN_KEY=.*/AIGW_ADMIN_KEY=$(openssl rand -hex 32)/" \ -e "s/^AIGW_KEY_PEPPER=.*/AIGW_KEY_PEPPER=$(openssl rand -hex 32)/" \ -e "s|^AIGW_PUBLISHER_KEYS=.*|AIGW_PUBLISHER_KEYS=$(cat ~/.aigw/publisher.pub)|" .envdocker compose up -d --buildcurl -s localhost:8080/healthz # {"name":"home-hub","ok":true,…}export AIGW_HUB=http://localhost:8080 AIGW_ADMIN_KEY=$(grep ^AIGW_ADMIN_KEY .env | cut -d= -f2)The hub runs with host networking so it can see your LAN and announce itself to nodes (API and dashboard on
port 8080, the TLS node endpoint on 8443). On Docker Desktop (macOS, Windows) turn on host networking in its settings
(Resources → Network), or follow the comment in docker-compose.yml to publish the ports instead.
kinherd keygen prints your key’s keyId (16 hex characters). Nodes trust exactly the publisher keys you name when
you pair them. ~/.aigw/publisher.key is the secret half. The dashboard is at http://localhost:8080/admin
(sign in with the admin key).
2. Create a client key
Section titled “2. Create a client key”curl -s -H "Authorization: Bearer $AIGW_ADMIN_KEY" -d '{"name":"laptop","scopes":["*"]}' $AIGW_HUB/v1/admin/keys# → {"key":{"id":1,"name":"laptop","kind":"client","prefix":"sk-aigw-HGfF",…},"secret":"sk-aigw-…"}export AIGW_KEY=sk-aigw-… # the secret is shown onceOr on the dashboard: Keys → Create. Scopes are model globs (*, clip-*); add ratePerMinute and
dailyQuota to limit a key.
3. Give the hub a model
Section titled “3. Give the hub a model”Nodes install signed model packages from the hub, chosen by recipes (which model on which devices). Take the
EfficientNet-Lite4 image classifier from the models-v1 release, sign it and the example Linux recipe with your
key, and upload both:
REPO=$(git rev-parse --show-toplevel)mkdir -p ~/aigw-models && cd ~/aigw-modelscurl -fLO <releases-url>/download/models-v1/efficientnet-lite4-int8.tar.gzecho "3694c9763fadf8654b71988687dde95e6ff4acf5610293350d345b9271a9bf81 efficientnet-lite4-int8.tar.gz" | shasum -a 256 -ctar -xzf efficientnet-lite4-int8.tar.gzkinherd sign -key ~/.aigw/publisher.key efficientnet-lite4-int8/manifest.jsonkinherd package push efficientnet-lite4-int8kinherd sign -key ~/.aigw/publisher.key -o recipe.yaml "$REPO/docs/examples/linux-cpu-efficientnet.yaml"kinherd recipe push recipe.yamlThe model package format explains how to package your own .tflite models.
4. Add a node
Section titled “4. Add a node”One command, as your own user (never sudo):
curl -fsSL <releases-url>/latest/download/install.sh | sh -s -- --publisher <your keyId># a hub that is not on this network: … | sh -s -- --hub https://<hub-host> --publisher <your keyId>The installer detects amd64 or arm64, downloads kinherd-node and a prebuilt TensorFlow Lite runtime (glibc 2.35+:
Ubuntu 22.04+, Debian 12+, Raspberry Pi OS 12), checks both against the release’s SHA256SUMS, installs them under
~/.local, pairs the node, installs the systemd user service and prints its status. On the same network it finds
the hub by itself and asks before pairing; --fingerprint (shown on the dashboard’s Add device page) picks it
without asking.
While it pairs it prints a code such as UEFJ-3SDO and waits. On the dashboard open Add device → Approve a
device, enter the code, check the machine shown is yours, and approve; the installer carries on. (If pairing
stops with --hub https://<hub> is required, that node build predates LAN discovery: run it again with --hub.)
… | sh -s -- --uninstall removes everything again.
The node downloads the package (checking your signature), benchmarks it and starts serving. It is a polite guest:
it serves only on mains power and after the machine has been idle for 10 minutes. To try it at once, set
idleMinutes to 0 in the policy block of ~/.config/aigw-node/config.json and run
systemctl --user restart kinherd-node.
- Download the APK from the latest release (
SHA256SUMSlists its hash):app-litert2-release.apkfor phones,app-nnapi-release.apkfor Android TVs, and install it (allow installs from your browser or file manager). - On the dashboard: Add device → Create code. It shows a QR code.
- In the app: Scan QR code (or Enter a pairing code). The QR carries the hub address, the one-time code and the publisher keys to trust.
- Approve the new node on the dashboard’s Nodes page: nodes paired with a code start pending.
The app talks to the hub over HTTPS only: on the same network through the hub’s LAN endpoint (pinned through the QR code), from elsewhere through a public HTTPS address. A phone serves only while charging, with the screen off for 5 minutes, not hot and not on mobile data (Owner protections in the app’s settings).
A stub node speaks the real protocol with a fake embedding model (stub-embed, 8 dimensions):
curl -s -H "Authorization: Bearer $AIGW_ADMIN_KEY" -d '{"name":"stub","kind":"node"}' $AIGW_HUB/v1/admin/keyscd "$REPO/server" && go run ./cmd/fakenode -hub ws://localhost:8080 -token <node secret> &5. Call it
Section titled “5. Call it”curl -s -H "Authorization: Bearer $AIGW_KEY" $AIGW_HUB/v1/models # what your nodes serve right nowcurl -s -H "Authorization: Bearer $AIGW_KEY" --data-binary @photo.jpg $AIGW_HUB/v1/models/efficientnet-lite4-int8/run# → {"backend":"xnnpack-cpu","model":{"id":"efficientnet-lite4-int8",…},"output":{"labels":[…]},# "servedBy":"<node>","via":"home-hub","timings":{"decode":10.5,"invoke":29,…,"total":40.7},…}The OpenAI SDKs work against /openai/v1 for listing models and for embeddings, which need an embedding model
(CLIP on a phone, or stub-embed). A classifier answers 400 there; call it through the native API above.
# pip install openaiimport base64, osfrom openai import OpenAI
client = OpenAI(base_url="http://localhost:8080/openai/v1", api_key=os.environ["AIGW_KEY"])
for m in client.models.list(): print(m.id, m.model_extra.get("task"))
img = "data:image/jpeg;base64," + base64.b64encode(open("photo.jpg", "rb").read()).decode()emb = client.embeddings.create(model="stub-embed", input=[img])print(len(emb.data[0].embedding), "dimensions")// npm install openai, then: node quickstart.mjsimport fs from "node:fs";import OpenAI from "openai";
const client = new OpenAI({ baseURL: "http://localhost:8080/openai/v1", apiKey: process.env.AIGW_KEY });
for await (const m of client.models.list()) console.log(m.id, m.task);
const img = "data:image/jpeg;base64," + fs.readFileSync("photo.jpg").toString("base64");const emb = await client.embeddings.create({ model: "stub-embed", input: [img] });console.log(emb.data[0].embedding.length, "dimensions");curl -s -H "Authorization: Bearer $AIGW_KEY" -H 'Content-Type: application/json' \ -d "{\"model\":\"stub-embed\",\"input\":[\"data:image/jpeg;base64,$(base64 < photo.jpg | tr -d '\n')\"]}" \ $AIGW_HUB/openai/v1/embeddingsOn the home network
Section titled “On the home network”The hub and your nodes find each other on the same network without configuration: the hub announces itself on the LAN, nodes discover it when they pair, and paired nodes connect to the hub’s LAN endpoint first, falling back to the public address on their own. Inference at home keeps working when the internet is down. Devices on different subnets or VLANs: see the hub’s network settings in Run a hub.
Troubleshooting
Section titled “Troubleshooting”| Symptom | Cause and fix |
|---|---|
kinherd-node pair keeps waiting |
Approve the request: dashboard → Add device → Approve a device, enter the code the node printed. Codes expire after 15 minutes. |
Node connected, models missing from /v1/models |
Pending approval (paired with a code): dashboard → Nodes → Approve. Pending nodes are never routed. |
A model disappears; runs answer 503 unavailable |
Owner protections paused the node (phone on battery or in use, Linux machine in use or on battery). It comes back by itself; kinherd-node status or the app’s Owner protections card says why. |
404 model_not_found |
No approved, serving node has that model, or the key’s scopes do not cover it. Check GET /v1/models with the same key and kinherd-node status on the node. |
400 … not an embedding model |
/openai/v1/embeddings takes only embedding models; use POST /v1/models/{id}/run for classifiers. |
| Browser requests fail with a CORS error | /v1 and /openai/v1 send no CORS headers on purpose: call the hub from your backend and never put a client key in a web page. |
| Phone: “That hub address doesn’t look right” | The app needs an https:// hub: pair from the QR code on the same network, or give the hub a public HTTPS address. |
More in Troubleshooting.