About this project
Kinherd is a self-hosted project, built and run by one person. This page says why it exists and where it stands.
Why it exists
Homes and offices already hold capable AI hardware: phones with GPUs and TPUs, TVs with NPUs, laptops with strong GPUs. Most of it sits idle while photos, camera feeds and documents travel to someone else’s data centre. The goal is to let the devices people already own work together as one private AI server, with the keys, metering and audit log you would expect from a company setup.
The project was formerly called IoT AI Gateway; the old command names (such as aigw-node, nowkinherd-node) keep working as aliases for now. The new name says the same thing: Kinherd is a herd of the devices your household already owns, the phones, TVs and computers of the people you live with, looked after by one hub.
How it started
It started with one app and one question. A photo app ran a 582 MB CLIP model on the phone it was installed on. Could the same phone serve that model to the Mac on the desk, over the local network, with the same results? It could: about 1.5 s per photo on the Pixel’s GPU, 1.65 s round trip, and embeddings identical to the original model.
Then came the TV. Its NPU ran small int8 vision models in a few milliseconds, until a model loaded from shared storage took 9 seconds instead of 22 ms, because the vendor’s NN driver was not allowed to read the file and silently fell back to the CPU. That kind of detail is why the compatibility registry exists: someone has to write down what runs where, and with which fix.
The phone was the first hub. Today the hub is a small server in Docker, the phone, the TV and Linux and Mac computers are nodes that dial out to it, and any OpenAI-compatible client can use them.
How it is built
- Data stays where it was created unless you decide otherwise. The hub runs at your place.
- Standard interfaces. The OpenAI API for clients, a documented JSON protocol for nodes.
- Numbers are measured. Every latency on this site was measured on a real device, and the registry says which.
- Existing runtimes. LiteRT, TensorFlow Lite and llama.cpp run the models on the nodes (ONNX Runtime Web in the live demo); the project does not write its own.
Where it stands
A private beta, run by an individual, not a company. There are no investors, customers or partners to announce. What has shipped, and when, is in the changelog; to try it, join the waitlist.
Licence
The code (hub, nodes, Android app, client library and SDKs) is licensed under the Apache License 2.0, and the documentation under Creative Commons Attribution 4.0 (CC-BY-4.0). The source repository is not public yet.