Skip to content

Quickstart

You run a hub (one small Docker container), give it a model, add a node (a Linux machine or an Android phone you already own), and call it with curl or an OpenAI SDK. Requests go to your hub and run on your node; nothing leaves your network.

your app / curl / OpenAI SDK ──HTTP + client key──▶ hub (Docker) ◀──outbound WebSocket── node (Linux box, phone, TV)

You need Docker with Compose, curl, openssl, and the small aigw admin CLI (aigw-<os>-<arch> from the latest release, or built with Go 1.27 or newer as below).

Terminal window
git clone <repository-url> iot-ai-gateway && cd iot-ai-gateway/server
go build -o ~/.local/bin/kinherd ./cmd/kinherd # admin CLI: keys, signing, pairing codes
kinherd keygen -out ~/.aigw -name publisher # your publisher key: it signs the models your nodes may run
cp .env.example .env
sed -i.bak -e "s/^AIGW_ADMIN_KEY=.*/AIGW_ADMIN_KEY=$(openssl rand -hex 32)/" \
-e "s/^AIGW_KEY_PEPPER=.*/AIGW_KEY_PEPPER=$(openssl rand -hex 32)/" \
-e "s|^AIGW_PUBLISHER_KEYS=.*|AIGW_PUBLISHER_KEYS=$(cat ~/.aigw/publisher.pub)|" .env
docker compose up -d --build
curl -s localhost:8080/healthz # {"name":"home-hub","ok":true,…}
export AIGW_HUB=http://localhost:8080 AIGW_ADMIN_KEY=$(grep ^AIGW_ADMIN_KEY .env | cut -d= -f2)

The hub runs with host networking so it can see your LAN and announce itself to nodes (API and dashboard on port 8080, the TLS node endpoint on 8443). On Docker Desktop (macOS, Windows) turn on host networking in its settings (Resources → Network), or follow the comment in docker-compose.yml to publish the ports instead.

kinherd keygen prints your key’s keyId (16 hex characters). Nodes trust exactly the publisher keys you name when you pair them. ~/.aigw/publisher.key is the secret half. The dashboard is at http://localhost:8080/admin (sign in with the admin key).

Terminal window
curl -s -H "Authorization: Bearer $AIGW_ADMIN_KEY" -d '{"name":"laptop","scopes":["*"]}' $AIGW_HUB/v1/admin/keys
# → {"key":{"id":1,"name":"laptop","kind":"client","prefix":"sk-aigw-HGfF",…},"secret":"sk-aigw-…"}
export AIGW_KEY=sk-aigw-… # the secret is shown once

Or on the dashboard: Keys → Create. Scopes are model globs (*, clip-*); add ratePerMinute and dailyQuota to limit a key.

Nodes install signed model packages from the hub, chosen by recipes (which model on which devices). Take the EfficientNet-Lite4 image classifier from the models-v1 release, sign it and the example Linux recipe with your key, and upload both:

Terminal window
REPO=$(git rev-parse --show-toplevel)
mkdir -p ~/aigw-models && cd ~/aigw-models
curl -fLO <releases-url>/download/models-v1/efficientnet-lite4-int8.tar.gz
echo "3694c9763fadf8654b71988687dde95e6ff4acf5610293350d345b9271a9bf81 efficientnet-lite4-int8.tar.gz" | shasum -a 256 -c
tar -xzf efficientnet-lite4-int8.tar.gz
kinherd sign -key ~/.aigw/publisher.key efficientnet-lite4-int8/manifest.json
kinherd package push efficientnet-lite4-int8
kinherd sign -key ~/.aigw/publisher.key -o recipe.yaml "$REPO/docs/examples/linux-cpu-efficientnet.yaml"
kinherd recipe push recipe.yaml

The model package format explains how to package your own .tflite models.

One command, as your own user (never sudo):

Terminal window
curl -fsSL <releases-url>/latest/download/install.sh | sh -s -- --publisher <your keyId>
# a hub that is not on this network: … | sh -s -- --hub https://<hub-host> --publisher <your keyId>

The installer detects amd64 or arm64, downloads kinherd-node and a prebuilt TensorFlow Lite runtime (glibc 2.35+: Ubuntu 22.04+, Debian 12+, Raspberry Pi OS 12), checks both against the release’s SHA256SUMS, installs them under ~/.local, pairs the node, installs the systemd user service and prints its status. On the same network it finds the hub by itself and asks before pairing; --fingerprint (shown on the dashboard’s Add device page) picks it without asking.

While it pairs it prints a code such as UEFJ-3SDO and waits. On the dashboard open Add device → Approve a device, enter the code, check the machine shown is yours, and approve; the installer carries on. (If pairing stops with --hub https://<hub> is required, that node build predates LAN discovery: run it again with --hub.) … | sh -s -- --uninstall removes everything again.

The node downloads the package (checking your signature), benchmarks it and starts serving. It is a polite guest: it serves only on mains power and after the machine has been idle for 10 minutes. To try it at once, set idleMinutes to 0 in the policy block of ~/.config/aigw-node/config.json and run systemctl --user restart kinherd-node.

Terminal window
curl -s -H "Authorization: Bearer $AIGW_KEY" $AIGW_HUB/v1/models # what your nodes serve right now
curl -s -H "Authorization: Bearer $AIGW_KEY" --data-binary @photo.jpg $AIGW_HUB/v1/models/efficientnet-lite4-int8/run
# → {"backend":"xnnpack-cpu","model":{"id":"efficientnet-lite4-int8",…},"output":{"labels":[…]},
# "servedBy":"<node>","via":"home-hub","timings":{"decode":10.5,"invoke":29,…,"total":40.7},…}

The OpenAI SDKs work against /openai/v1 for listing models and for embeddings, which need an embedding model (CLIP on a phone, or stub-embed). A classifier answers 400 there; call it through the native API above.

# pip install openai
import base64, os
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8080/openai/v1", api_key=os.environ["AIGW_KEY"])
for m in client.models.list():
print(m.id, m.model_extra.get("task"))
img = "data:image/jpeg;base64," + base64.b64encode(open("photo.jpg", "rb").read()).decode()
emb = client.embeddings.create(model="stub-embed", input=[img])
print(len(emb.data[0].embedding), "dimensions")

The hub and your nodes find each other on the same network without configuration: the hub announces itself on the LAN, nodes discover it when they pair, and paired nodes connect to the hub’s LAN endpoint first, falling back to the public address on their own. Inference at home keeps working when the internet is down. Devices on different subnets or VLANs: see the hub’s network settings in Run a hub.

Symptom Cause and fix
kinherd-node pair keeps waiting Approve the request: dashboard → Add device → Approve a device, enter the code the node printed. Codes expire after 15 minutes.
Node connected, models missing from /v1/models Pending approval (paired with a code): dashboard → Nodes → Approve. Pending nodes are never routed.
A model disappears; runs answer 503 unavailable Owner protections paused the node (phone on battery or in use, Linux machine in use or on battery). It comes back by itself; kinherd-node status or the app’s Owner protections card says why.
404 model_not_found No approved, serving node has that model, or the key’s scopes do not cover it. Check GET /v1/models with the same key and kinherd-node status on the node.
400 … not an embedding model /openai/v1/embeddings takes only embedding models; use POST /v1/models/{id}/run for classifiers.
Browser requests fail with a CORS error /v1 and /openai/v1 send no CORS headers on purpose: call the hub from your backend and never put a client key in a web page.
Phone: “That hub address doesn’t look right” The app needs an https:// hub: pair from the QR code on the same network, or give the hub a public HTTPS address.

More in Troubleshooting.