Skip to content

Introduction

Kinherd turns devices you already own (phones, TVs, set-top boxes, Linux machines such as a Pi or a mini PC, and Macs) into one private AI server. It was formerly called IoT AI Gateway; the old command names (such as aigw-node) keep working as aliases for now, and a few identifiers keep the aigw prefix. It has three parts:

  • The hub is the one address clients know. It exposes an OpenAI-compatible API and a native API, checks virtual keys, scopes, rate limits and daily quotas, writes an audit entry for every request, and routes each job to the best node. It runs no models itself. Today the hub is a single Go binary in Docker.
  • Nodes are devices that run models. A node dials out to the hub over one WebSocket, announces its models and hardware, takes jobs, and benchmarks itself. Nodes open no inbound port, so they work behind NAT and on mobile data. Today: Android phones (LiteRT 2, GPU/NPU/CPU), Android TVs (LiteRT 1.4 + NNAPI on the NPU) and Linux machines (amd64/arm64, TensorFlow Lite + XNNPACK on the CPU).
  • The compatibility registry records which model runs on which device and accelerator, how fast, and with which tweak, or why it fails. Nodes fill it automatically; the router and the public registry page use it.
clients (apps, OpenAI SDKs, Home Assistant, scripts) nodes (phone, TV, …)
│ HTTPS + virtual key │ outbound WebSocket (wss://<hub-host>/v1/nodes/connect)
▼ │
hub ── keys · quotas · audit · router ◀─────────────────┘
Model Device Accelerator Result
EfficientNet-Lite4 int8 TCL Google TV (MediaTek MT9689) NPU (MDLA via NNAPI) 22 ms invoke, ~72 ms per request, with one tweak
MobileNetV2 int8 same TV NPU 4.4–5.0 ms
CLIP ViT-L/14-336 (fp16 weights) Pixel 7 Pro (Tensor G2) GPU (LiteRT) ~1.45 s per image, 1.65 s round trip from a Mac

Embeddings served through the hub are identical to the node’s own (cosine 1.000000); the fp16 model matches the fp32 original at cosine 1.000.