Introduction
Kinherd turns devices you already own (phones, TVs, set-top boxes, Linux machines such as a Pi or a mini PC, and Macs) into one
private AI server. It was formerly called IoT AI Gateway; the old command names (such as aigw-node) keep working as aliases for
now, and a few identifiers keep the aigw prefix. It has three parts:
- The hub is the one address clients know. It exposes an OpenAI-compatible API and a native API, checks virtual keys, scopes, rate limits and daily quotas, writes an audit entry for every request, and routes each job to the best node. It runs no models itself. Today the hub is a single Go binary in Docker.
- Nodes are devices that run models. A node dials out to the hub over one WebSocket, announces its models and hardware, takes jobs, and benchmarks itself. Nodes open no inbound port, so they work behind NAT and on mobile data. Today: Android phones (LiteRT 2, GPU/NPU/CPU), Android TVs (LiteRT 1.4 + NNAPI on the NPU) and Linux machines (amd64/arm64, TensorFlow Lite + XNNPACK on the CPU).
- The compatibility registry records which model runs on which device and accelerator, how fast, and with which tweak, or why it fails. Nodes fill it automatically; the router and the public registry page use it.
clients (apps, OpenAI SDKs, Home Assistant, scripts) nodes (phone, TV, …) │ HTTPS + virtual key │ outbound WebSocket (wss://<hub-host>/v1/nodes/connect) ▼ │ hub ── keys · quotas · audit · router ◀─────────────────┘Measured so far
Section titled “Measured so far”| Model | Device | Accelerator | Result |
|---|---|---|---|
| EfficientNet-Lite4 int8 | TCL Google TV (MediaTek MT9689) | NPU (MDLA via NNAPI) | 22 ms invoke, ~72 ms per request, with one tweak |
| MobileNetV2 int8 | same TV | NPU | 4.4–5.0 ms |
| CLIP ViT-L/14-336 (fp16 weights) | Pixel 7 Pro (Tensor G2) | GPU (LiteRT) | ~1.45 s per image, 1.65 s round trip from a Mac |
Embeddings served through the hub are identical to the node’s own (cosine 1.000000); the fp16 model matches the fp32 original at cosine 1.000.
Where to start
Section titled “Where to start”QuickstartHub, model, node and first request in about 10 minutes.
Run a hubDocker Compose locally, or a Docker Swarm stack behind a CDN and a reverse proxy.
Add nodesLinux machines, Android phones and TVs: pairing, approval, build flavors.
Your first requestcurl and the OpenAI Python SDK.
ConceptsHub, nodes, routing, registry, model packages.