Skip to content

Node protocol

A model node (the phone, the TV) connects outbound to the hub (usually a server behind a CDN and a reverse proxy) over one persistent WebSocket and receives jobs on it. Nodes may be behind NAT or on mobile data; they open no inbound port. This is the default node transport ("hub": {"transport": "websocket"}); the older HTTPS self-registration ("transport": "https", the hub calls the node back) is described in the README.

Implemented by the Android node in app/src/main/java/dev/kinherd/app/node/, the Linux node in node-linux/internal/hub (node-linux/README.md), and by the hub in server/internal/protocol and server/internal/nodes.

GET wss://<hub-host>/v1/nodes/connect
Authorization: Bearer <node token>
  • TLS: the system trust store with host-name verification (a public certificate, e.g. through Cloudflare). For a self-hosted hub with a self-signed certificate the node may pin the leaf instead ("certSha256", SHA-256 of the DER); then only that certificate is accepted and the host name is not checked.
  • Auth: the node token identifies the node; the hub names the node after it (the name in hello is a label). A refused upgrade (HTTP 401/403) makes the node retry at the slowest backoff.
  • Keep-alive: both sides send WebSocket pings (the node every 20 s), so dead connections (NAT timeouts, a lost Wi-Fi) are noticed within about 20–40 s.
  • Frames: JSON text frames, each an object with a "type". Binary frames are not used. Unknown types and unknown fields are ignored by both sides; fields may be added (optional) but never renamed or removed.
{"type": "hello", "name": "pixel-7-pro", "appVersion": "0.3-litert2",
"models": [{"id": "clip-l14-336", "version": "1", "task": "image-embedding", "backend": "litert-gpu", "state": "loaded"}],
"device": {"manufacturer": "Google", "model": "Pixel 7 Pro", "soc": "Google Tensor G2", "os": "Android 16 (API 36)",
"ramMB": 11683, "accelerators": ["gpu", "npu", "cpu"]}}
  • models[]: id, version, task, backend (null until the model has loaded), state (unloaded | loading | loaded | error). Optional extras: name, memoryMB, input, output.
  • device: soc is Build.SOC_MANUFACTURER + SOC_MODEL (API 31+), else ro.board.platform, else Build.HARDWARE. accelerators are those of the build flavor’s runtime (litert2: gpu, npu, cpu; nnapi: npu, cpu). Optional extras, used to match recipes (RECIPES.md): abi (the primary ABI, Build.SUPPORTED_ABIS[0], or the whole list as an array), osVersion (Build.VERSION.RELEASE, e.g. "14"), sdkInt (Build.VERSION.SDK_INT), cpus. The Linux node sends manufacturer/model from DMI or the device tree (Generic / Linux <arch> when there is none), soc = the device-tree SoC and/or the CPU model, os = "Linux <kernel> (<distro>)", osVersion = the kernel version, abi = ["aarch64","arm64"] or ["x86_64","amd64"], accelerators: ["cpu"] (DEVICE_PROBE.md, Hello device facts).
  • probe (optional): the node’s full device probe, DEVICE_PROBE.md (probeVersion, OS, CPU and ISA extensions, memory, GPUs/NPUs, usable compute APIs, power, thermal, micro-benchmarks). Sent by the Linux node in hello only (not in heartbeats); a few KB. It carries no host name, user, serial numbers or addresses. The hub validates and sanitises it (≤ 64 KB, probeVersion 1, unknown fields dropped), keeps the latest per node for its estimator and the anonymous registry aggregate (DEVICE_PROBE.md, Hub); a bad probe is logged and ignored, never a reason to refuse the hello. Older hubs ignore the field. device stays the matching fingerprint, the probe is extra detail.
  • direct (optional): the node’s LAN HTTPS endpoints for direct connections, {"endpoints":["https://<lan-ip>:9443"],"certSha256":"<64 hex of the DER cert>","subnets":["<lan-ip>/24"]} (DIRECT.md).
  • protections (optional): the owner protections this node enforces and whether it serves now, see Owner protections below. Older hubs ignore it; a hub with the marketplace on offers a node’s listings to other owners only when its hello carried this object.
  • features (optional): protocol extensions the node supports, e.g. ["chat-stream","cancel"] (see Chat models and streaming below). A hub uses a feature only with nodes that list it; older hubs ignore the field and older nodes simply do not send it.
  • jobKey (optional): the node’s current job key for end-to-end sealing, signed by its device key ({"v":1,"kid","pub","iat","exp","device":{"alg","pub"},"sig"}, E2E.md §2). The hub verifies it and ignores an invalid one; only nodes with a valid job key get sealed runs. Older hubs ignore it. The welcome then carries sealingNonce; the node answers {"type":"jobKeyProof","kid","sig"} (its device key’s signature over the welcome’s hub name, the nonce and the kid, E2E.md §2) and the hub replies {"type":"jobKeyStatus","kid","status":"active"|"refused","reason"}; only an active job key is routed to.
{"type": "welcome", "hub": "home-hub", "heartbeatSec": 30, "node": "phone", "maxInFlight": 4}

heartbeatSec is clamped by the node to 5–300 (default 30). Optional: node (the name the hub registered the node under, shown on the node’s status screen) and maxInFlight (the node then runs at most that many jobs at once). Optional pending: true: the node was paired but an admin has not approved it (PAIRING.md); it gets no runs and no recipes until the hub sends {"type": "approval", "status": "approved"}. Optional tickets ({"keyId","publicKey":"<base64>"}, the key that signs direct-connection tickets now) and ticketKeys (every ticket key to accept, active first); absent when direct connections are off (DIRECT.md). Optional lan ([{"url":"wss://<lan-ip>:8443/v1/nodes/connect","certSha256":"<64 hex>"}], at most 4): the hub’s LAN endpoints, see LAN-first below. Absent = the hub advertises none (a node then forgets any it saved). A node that gets no welcome within 20 s closes and reconnects. Runs that arrive before the welcome get a 503 result.

node → hub heartbeat (every heartbeatSec)

Section titled “node → hub heartbeat (every heartbeatSec)”
{"type": "heartbeat", "models": [ … as in hello … ], "queue": 0}

A node that pauses (the Linux node on battery or while its user is active) sends a heartbeat with "models": [] at once, so the hub stops routing to it without dropping the connection, and the full list again when it resumes; a run that still arrives while paused gets 503 unavailable with retryAfter.

queue: requests waiting in the node’s model queues (all callers, including same-device apps). Optional direct as in hello: missing = unchanged, {} or empty endpoints = withdrawn. Optional protections as in hello: missing = unchanged (both nodes send it in every heartbeat). Optional jobKey as in hello: missing = unchanged; a node sends it when it rotated its job key.

{"type": "run", "id": "<request id>", "model": "clip-l14-336", "contentType": "image/jpeg", "bodyB64": "<base64 image>"}

id: 1–128 characters (a number is accepted too). bodyB64: standard base64 (line breaks tolerated). Optional path (hubs with the path indicator): how the request reached this node, "hub-lan" (the client reached the hub over the LAN and this node’s link is on the LAN, e.g. through the LAN listener) or "hub-public" (either half used the public URL / Cloudflare). Purely informational, for a “Local ✓ / Via cloud” display; nodes ignore unknown values (SERVER.md). A link through the LAN listener is what makes relayed jobs eligible for local only keys. Optional stream: true: only to nodes whose hello listed chat-stream, for a chat model; the node answers with chunk frames and then the usual result (Chat models and streaming).

Sealed runs (E2E.md): contentType application/aigw-sealed means bodyB64 is a sealed request envelope addressed to the node’s job key (the hub sends these only to the node whose advertised kid it names). The node opens it with the binding (its own name, model, empty jti), checks freshness and replay, runs the inner body as usual, and answers body: {"sealed": "<base64 response envelope>"}; on an error after opening it answers the status with {"error":{"code":"<code>","message":"sealed"},"sealed":"…"} (the full error object sealed). Envelope problems before opening are plain errors: 400 sealed_invalid / sealed_stale, 409 sealed_replay / job_key_unknown. usage stays outside the ciphertext.

{"type": "result", "id": "<request id>", "status": 200, "body": { … native run JSON … }}
{"type": "result", "id": "<request id>", "status": 429, "body": {"error": {"code": "busy", "message": "…"}}, "retryAfter": 1}

body is exactly what POST /v1/models/{id}/run on the node would answer: on success {"model":{"id","version"},"output":{…},"ms","backend","servedBy","timings"}, otherwise the error object with the same status and code as the HTTP API (native API). Optional retryAfter (seconds) accompanies 429 and 503.

Optional usage reports the tokens the run consumed, for models that count them (text models; vision models simply omit it):

{"type": "result", "id": "<request id>", "status": 200, "body": { … }, "usage": {"input_tokens": 12, "output_tokens": 0}}

Both fields are non-negative integers (at most 100 000 000; other values are ignored). The hub counts them against the key’s and team’s tpmLimit and returns them to the client (usage on the run answer, usage.prompt_tokens / total_tokens on /openai/v1/embeddings, the full OpenAI usage on /openai/v1/chat/completions). Hubs that predate the field ignore it, and nodes that do not send it keep working unchanged. HTTPS-registered nodes may put the same object at the top level of their answer body.

The job goes through the node’s normal API path: model id check (404 model_not_found), body size limit (413 payload_too_large, empty body 411), image decode (415), the model’s scheduler queue (429 busy when full), load failures (503 unavailable), requestTimeoutSeconds (504), metrics and the audit log (client = the hub’s name from welcome, ip hub-link, method WS). The node runs several jobs concurrently, up to models × (queueSize + 1) (capped at 64, and at the welcome’s maxInFlight); beyond that it answers 429 busy at once. A malformed run whose id is readable gets a 400 bad_request result; without an id it is dropped. Results for a connection that has since dropped are discarded (the hub has already failed those requests).

A node advertises a chat (LLM) model with "task": "chat" in models[], like any other model:

{"id": "qwen2.5-1.5b-instruct", "version": "q4_k_m", "task": "chat", "backend": "llama.cpp-metal", "state": "loaded"}

The hub lists it on GET /openai/v1/models and routes POST /openai/v1/chat/completions for it to the node as a run with contentType: "application/json" and, as the body, the client’s OpenAI chat completion request (model, messages, temperature, tools, … unchanged) without stream and stream_options, and with max_tokens (or max_completion_tokens) capped at the hub’s openai.maxTokens (set when the client sent none). Tool calling is passed through: tools (at most 128, 256 KiB, checked by the hub), tool_choice, parallel_tool_calls, assistant tool_calls and tool messages arrive as the client sent them; the answer’s message.tool_calls / streamed delta.tool_calls (with index) and finish_reason: "tool_calls" go back unchanged (LLM.md). A node must not offer a model id that is one of the hub’s OpenAI aliases (openai.models); the hub ignores it. Images are only inline image_url data URLs (data:image/…;base64,…); the hub refuses remote URLs, so a node never has to fetch anything.

The result body is an OpenAI chat.completion object (id, created, choices[] with message and finish_reason, usage with prompt_tokens / completion_tokens); the node should also put the counts in the result’s usage (input_tokens = prompt, output_tokens = completion). The hub counts them against the key’s and team’s token limits and returns them in OpenAI’s usage. Errors are the usual error results (model_not_found, busy, unavailable, timeout, …).

Streaming (feature chat-stream). When the client asks for "stream": true and the node listed chat-stream, the run carries "stream": true and the node sends any number of

{"type": "chunk", "id": "<request id>", "data": {"id": "chatcmpl-…", "object": "chat.completion.chunk", "created": 1767225605,
"model": "qwen2.5-1.5b-instruct", "choices": [{"index": 0, "delta": {"content": "Hel"}, "finish_reason": null}]}}

in order, then the result as above (its body may be the assembled completion or just {"object":"chat.completion", "usage":{…}}; the hub uses only its status and usage). data is one OpenAI chunk object; chunks without choices (e.g. a usage-only chunk) are dropped, and the hub sends its own usage chunk when the client asked for stream_options.include_usage. A result with an error status before any chunk is a normal error answer; after chunks it ends the client’s stream with an error event. The hub buffers at most 8 MiB of chunks per run for a slow client, then cancels the run.

A node without chat-stream gets stream never set and answers in one piece; the hub turns that answer into server-sent events itself, so clients can always stream.

Cancel (feature cancel). When the client goes away (or the hub’s request timeout passes) the hub sends {"type": "cancel", "id": "<request id>"} to nodes that listed cancel; the node stops generating and should still send its result (any status, e.g. 499) with the usage spent so far: the hub waits up to 2 s for it and counts the larger of that report and its own estimate (also used when a node reports no usage at all, or a run is lost midway).

Disconnects. A link that drops before the first chunk is a transport failure like any other (the hub retries another node with the model). After a chunk was relayed the hub does not retry (the answer would start over): the client’s stream ends with an error event (peer_unavailable) and no data: [DONE].

{"type": "benchmark", "results": [
{"model": "clip-l14-336", "version": "1", "backend": "litert-gpu", "status": "works", "p50ms": 1412.3, "p95ms": 1530.8, "notes": "", "samples": 64},
{"model": "efficientnet-lite4", "version": "1", "backend": "xnnpack-cpu", "status": "tweaks", "p50ms": 95.1, "p95ms": 110.2, "notes": "fell back, skipped npu: …"},
{"model": "yolo", "version": "2", "backend": null, "status": "fails", "p50ms": null, "p95ms": null, "notes": "no accelerator could load yolo (…)"}]}
  • p50ms / p95ms: nearest-rank percentiles of the scheduler’s last 64 runner times on that backend (render, pack, invoke, postprocess: no queueing or load), 0.1 ms resolution.
  • status: works = loaded on the manifest’s first-choice accelerator; tweaks = it loaded only after falling back (notes lists the skipped accelerators and why); fails = the model failed to load (notes has the error).
  • When: after a model has 5 runs on its backend (so after its first successful loads), and for a load failure; checked after the welcome and after every heartbeat. At most every 6 h per model, except straight after a model goes from fails to working. Optional extra: samples.
  • On demand: the hub may send {"type": "benchmark"} (hub → node, optional); the node answers with every model that has at least one run or a load failure, ignoring the 6 h limit.
{"type": "recipes", "recipes": [{"schema": 1, "id": "tcl-smart-tv-pro-mt9689", "version": 1, …, "signature": {"keyId": "…", "sig": "…"}}]}

The signed recipes that match the node’s device, best match first (possibly empty), sent after the welcome and again whenever the hub’s recipes change. Each element is the recipe document exactly as signed (canonical JSON); the node verifies it with its trusted publisher keys. Format and apply steps: RECIPES.md.

{"type": "trust", "updates": [{"type": "trustUpdate", "schema": 1, "serial": 1, "prev": 0, …, "signatures": [{"keyId": "…", "sig": "…"}]}, …]}

The hub’s chain of signed publisher trust updates, oldest first, as stored (canonical JSON). The hub sends it right after the welcome, before recipes (so recipes signed by a key the chain adds verify), and again to every connected node when an admin uploads an update. A hub without updates sends no trust frame; nodes that do not know the frame ignore it. Pending nodes get it too: it carries no work, only keys.

The node applies, in serial order, each update newer than the serial it has applied, under the rules in RECIPES.md (Rotating the publisher key with a trust update): prev chained to the applied serial (any prev for a node that never applied one), inside notBefore/notAfter, signed by enough keys the node trusted before the update. It saves the new key set and serial together (atomically) before trusting them, logs every applied or refused update, and sends nothing back. Updates it has already applied are skipped silently; a frame with nothing new is a no-op. The same list is in the pairing answers (trustUpdates) and at GET /v1/trust (node key).

{"type": "recipeStatus", "recipe": "tcl-smart-tv-pro-mt9689", "version": 1, "status": "applied", "notes": "p50 invoke 31 ms"}

status: applied (installed, configured, and the verify benchmark met its expectations), failed, or skipped (the node chose not to apply it). Send the usual benchmark frame as well.

{"type": "approval", "status": "approved"}: a pending node was approved; runs and recipes follow.

{"type": "tickets", "tickets": {"keyId", "publicKey"}, "ticketKeys": [ … ]}: the ticket keys changed (an admin rotated them); replace the welcome’s keys. A node that ignores it gets the new keys on its next welcome.

{"type": "directUsage", "items": [{"jti": "…", "seq": 1, "sub": "7", "model": "clip-l14-336", "status": 200, "ms": 41, "bytes": 182344, "at": 1767225605,
"receipt": "1.4VfiqOYEaknkKBZJkN5NcglGT5Of61SVvkbdtSyGXn_zRk0_cJdtCaDrNTDGwg2tDhQdnmV1tECxMuOyBlCzAw", "bodySha256": "2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824"}]}

Requests the node served directly for hub tickets, batched (at most 512 items per frame; at in unix seconds). seq is the use number of the ticket (1 … uses; 1 for single-use tickets). receipt is the client’s X-Use-Receipt request header, copied verbatim (omit when the request had none), and bodySha256 the lower-case hex SHA-256 of the request body the node received; the node verifies the receipt against the ticket’s rpk before serving (401 invalid_receipt otherwise) and the hub charges a priced use only when it verifies. Each (jti, seq) is recorded once, so a batch may be resent after a reconnect. An item may carry the same optional usage object as a result; its tokens count against the key’s token limits. Details: DIRECT.md §5.

Code Meaning Node reaction
1001 hub shutting down reconnect with backoff
4400 first frame was not a valid hello slowest backoff (60 s)
4408 no traffic for the hub’s offline period reconnect with backoff
4409 the same node connected again; this connection is superseded slowest backoff
4429 node limit reached slowest backoff
4401 the node token was revoked (also mid-handshake) slowest backoff; a new token is needed

Exponential backoff with equal jitter: attempt n waits a uniform time between half and all of min(60 s, 1 s · 2ⁿ), never under 1 s (1 s, 1–2 s, 2–4 s, … 30–60 s), so nodes do not reconnect in lock-step after a hub restart. The counter resets on welcome. The node pauses while the device has no network (no attempts at all) and reconnects at once, with the counter reset, when the default network returns or changes (Wi-Fi ↔ mobile data); a socket on the old network would only time out. After an app restart or reboot the node reconnects when the service starts (it remembers Start/Stop).

A node pauses serving whenever its owner’s rules are not met, so lending the device never costs its owner battery, responsiveness, heat or mobile data. Paused, it stays connected, sends a heartbeat with "models": [] at once (and the full list on resume), withdraws its direct advertisement ("direct": {}), and answers any run (relayed or direct) with 503 unavailable, retryAfter 60.

"protections": {"requireCharging": true, "requireIdle": true, "idleMinutes": 5, "screenOff": true,
"maxThermal": "moderate", "allowMetered": false, "serving": false, "reason": "on battery; screen on"}
Field Android (Settings → Owner protections) Linux (config.json policy)
requireCharging only while plugged in (BatteryManager; no battery = always plugged) requireAC
requireIdle, idleMinutes only after the screen has been off this long (screenOff: true; PowerManager.isInteractive, ACTION_SCREEN_OFF) requireIdle, idleMinutes (no input, no CPU load by others)
maxThermal pauses at this PowerManager thermal status or worse (light … severe; omitted = never) —
maxTempC — maxTempC
allowMetered false: pauses while the default network is metered (mobile data, metered hotspot) always true (not detected)
serving, reason the current decision; reason only while paused same

Android defaults: phone / tablet charging only, screen off for 5 min, pause from thermal moderate, no mobile data; TV / no battery (always plugged, and serving while people watch is the point) only thermal severe and no metered data. Before the first check a node reports serving: false (“checking the device”). The dashboard shows the state on the Owner protections card; the hub shows it on Nodes and in GET /v1/nodes (protections, null when the node reports none).

So that home inference keeps working when the internet or Cloudflare is down, a node dials the hub’s LAN endpoints before the public URL it was paired with. It is zero-config: a normal install (server/docker-compose.yml, a Swarm stack, or the binary on a host) needs no address, port or firewall setting.

  • The hub’s LAN listener is on by default: a second, TLS-only listener that serves only /healthz//livez/ /readyz, the pairing endpoints and node-key paths (no dashboard, no client API). Port 8443, else the port it used last time or the next free one (8444–8453, then any), remembered in <dataDir>/lan-tls/port. Its certificate is self-signed and generated once in <dataDir>/lan-tls/ (nodes pin it, no CA needed). In the default (auto) mode it binds only the hub’s private LAN addresses, never the wildcard, so a hub that also has a public interface does not expose it; inside a bridge/overlay container it binds the container’s wildcard (the host publishes the port). Whatever it binds, it closes connections from peers that are not private (RFC 1918, ULA), loopback or link-local, and it ignores X-Forwarded-For/CF-Connecting-IP (nothing proxies it). Pairing through it has a global cap (30 code redemptions, 30 device-flow starts, 1200 polls per 10 min) on top of the per-IP limit.
  • Address detection. The hub advertises wss://<addr>:<port>/v1/nodes/connect for each private address of its interfaces: IPv4 RFC 1918 first, then IPv6 ULA (fc00::/7); loopback, link-local (169.254/16, fe80::/10: no stable or zone-free address), CGNAT/VPN (100.64/10) and public addresses are skipped, as are interfaces of container/VM/ VPN plumbing (docker*, br-*, veth*, virbr*, cni*, flannel*, cali*, vxlan*, tun*/tap*, wg*, tailscale*, zt*, …; a Proxmox vmbr0 or a NAS br0 carrying the LAN address is kept). At most 4, re-detected every minute (DHCP changes reach nodes with their next welcome and mDNS at once; new addresses get a socket).
    • Host, or a container with host networking (network_mode: host, the shipped compose file): the hub sees the host’s interfaces.
    • Container with bridge/overlay networking (/.dockerenv present and no host-only interface such as docker0 visible): its own addresses are useless to the LAN, so it detects none and relies on AIGW_HOST_ADDRS (addresses or names, optional :port when the published port differs). A Swarm stack publishes 8443 in host mode; the deployment fills AIGW_HOST_ADDRS by running the image’s /server -lan-addrs with host networking on the target host (same detection rules), and the stack also passes Swarm’s {{.Node.Hostname}} as AIGW_HOST_NAME, resolved best effort. The Docker host gateway (172.17.0.1, host.docker.internal) is no LAN address, so it is not used.
    • Overrides, all optional: AIGW_LAN_URLS replaces the detected list, AIGW_LAN_LISTEN=off disables the listener, AIGW_LAN_LISTEN=:9443 fixes the port (binds what it says, also the wildcard; no fallback).
  • mDNS / DNS-SD. The hub announces itself as _aigw._tcp (instance = hub name; SRV = LAN port, A/AAAA = the detected addresses; TXT v=1, name, fp = the LAN certificate’s SHA-256, id = its first 16 hex digits, pub = public URL, port, api/apitls = main listener, ver). No secrets. AIGW_MDNS=off disables it. It only reaches the hub’s own network segment (multicast does not cross VLANs or leave a bridge-networked container).
  • Discovery at pairing. Without a hub address, kinherd-node pair and the app’s Find the hub on this network list the announced hubs, full fingerprint first (all 64 hex digits in groups of four, the value the dashboard’s Add device page shows), then the addresses, then the name (plain text; names with control or bidi characters, or over 63 bytes, are dropped). The user compares the whole fingerprint and confirms, or passes --fingerprint <full SHA-256>; a pin already given (--cert-sha256, a payload’s hubCertSha256) must match. Pairing then goes to the hub’s LAN listener over TLS pinned to that certificate (code or device flow as usual), never to the announced pub URL or plain http: an impostor announcing the real fingerprint cannot complete the handshake. The answer, authenticated by the pin, may name the hub’s public wss:// URL as the link (then checked against the system trust store); the LAN endpoints are saved pinned. Discovery never pairs or trusts by itself: the pairing code or the admin’s approval still decides.
  • Learning the list. The hub lists its LAN endpoints in every welcome (lan). The node keeps the last list (Android: hub-lan.json in app storage, bound to the paired hub URL; Linux: lan in credentials.json; both are dropped by a new pairing), so a start with the internet down still finds the hub.
  • Order and fallback. Each round tries every LAN endpoint in order, then the public URL. A dial that fails before the welcome (refused, unreachable, TLS pin mismatch, no welcome within 5 s) moves to the next candidate at once; only when every candidate failed does the normal backoff apply, and the next round starts again with the LAN. A 401/403 or a slow close code (4400/4401/4409/4429) ends the round at once (the public URL would answer the same).
  • Stale list. When every remembered LAN endpoint failed, the node asks mDNS (2–3 s, at most once a minute; AIGW_MDNS=off on the Linux node disables it) before the public URL, and dials only answers pinned to a certificate the hub already advertised over the authenticated link (a hub that moved to another address or port). With no known pin nothing found is dialled.
  • Failback. While connected over the public URL, every 2 min (doubling after each round in which no LAN endpoint answered, up to 30 min; a new LAN list or a switch starts at 2 min again) the node checks GET https://<lan>/healthz (TLS against the pin, no token). When a LAN endpoint answers 200 and no job is running, the node closes the public connection (1000) and reconnects, LAN first. A job in flight postpones the switch to the next check.
  • Security. LAN endpoints must be wss://; ws:// entries are ignored by nodes and refused by the hub’s config, so the node token never crosses the LAN in clear. With certSha256 the node accepts exactly that leaf certificate; without it, the system trust store and host-name check apply. The pin arrives over an authenticated link (the public link, or the pinned discovery pairing), so someone on a café Wi-Fi that happens to use the same address gets a TLS handshake failure and never the token. Package/recipe downloads still use the public URL.
  • Status. Android shows “connected to home-hub (LAN)” and the endpoint in use; kinherd-node status shows (lan, wss://…) or (public, …). The app logs each NSD browse under the logcat tag aigw-mdns.

Segmented networks / VLANs (outside the normal flow)

Section titled “Segmented networks / VLANs (outside the normal flow)”

When nodes live on another VLAN or subnet than the hub (an IoT VLAN, a guest network):

  • Allow TCP (default 8443) from the node VLAN to the hub’s address on the router/firewall (for example a pass rule on the node VLAN’s interface, destination the hub, port 8443). Without it nodes simply use the public URL.
  • mDNS does not cross VLANs. Either run an mDNS reflector (the router’s Avahi package with reflection between the two interfaces, or avahi-daemon with enable-reflector=yes), or pair with the hub address / QR code instead of discovery. Already paired nodes do not need mDNS: the hub’s welcome tells them its LAN endpoints, which work across VLANs once the port is allowed.
  • The public URL keeps working as the fallback either way.

What server/ does with this protocol (deployment: Run a hub):

  • URLs: wss://hub.example.com/v1/nodes/connect from anywhere, or ws://<hub LAN address>:8080/v1/nodes/connect on the LAN. Node tokens are virtual keys with "kind": "node" (dashboard Keys, or POST /v1/admin/keys {"name":"tv","kind":"node"}); the key’s name becomes the node’s name. Before the upgrade the hub answers 401 (bad key), 403 (not a node key) or 429 in the usual error schema.
  • Limits: the hello must arrive within 15 s; frames from a node up to 16 MiB (run frames the hub sends are at most ~13.4 MiB, a 10 MB image in base64, so a node’s read limit must be at least that); no compression.
  • Liveness: the hub pings every 20 s and closes with 4408 after 90 s without frames or pongs.
  • Routing: among online nodes with the model: state loaded first, then the lowest EWMA round-trip latency for that model, then the fewest jobs in flight; at most maxInFlight (default 2) jobs per node, further requests wait up to 10 s for a slot, then get 429 busy. If the connection drops with jobs in flight they are retried on another node with the model, else the client gets 503. Only transport failures are retried; a node’s error answer or a hub timeout (requestSec, 504) is final.
  • Answers: a 200 body is passed to the client with servedBy = the node’s name and via = the hub’s name. Errors pass through unchanged except 401/403 and non-object bodies (502 peer_unavailable); model_not_found drops that model from the node until its next heartbeat. retryAfter becomes Retry-After.
  • Benchmarks go into the compatibility registry (GET /v1/registry) with the hello’s device, source: "auto" and the node’s name; a newer automatic result for the same model, version, device and backend replaces the older one. manufacturer and model in device are required for this. The estimator (GET /v1/nodes/{name}/estimate) uses a node’s own results first, then those of the same device model. The hub does not send the optional on-demand {"type":"benchmark"} request yet.
  • Owner protections: kept per node from hello and heartbeats. With the marketplace on, listings of nodes that reported none (older apps) are not offered to other owners (routing, /v1/route, /public/v1/market); their own owner keeps using them.
  • Recipes: after the welcome (and after every recipe upload or delete) the hub sends recipes; recipeStatus is recorded per recipe version and node, and counted per device on the public registry.
  • Pending nodes: keys created by pairing start pending when requireNodeApproval is on; they connect but are never routed (PAIRING.md).
  • Revocation: revoking a node key closes its sessions with 4401 at once, including one still in its handshake, and live sessions are re-checked against the key store every 60 s.
  • Direct connections: direct advertisements are kept per node (admin-only in GET /v1/nodes and the dashboard), the welcome carries the ticket keys, and directUsage items go into the audit log and usage with source: "direct" (DIRECT.md).
  • Chat: task: "chat" models are served on /openai/v1/chat/completions (nodes first, then any configured cloud provider), and, translated by the hub to the same chat run, on /openai/v1/responses and /anthropic/v1/messages (the node sees an ordinary chat request); chunks are relayed to the client as they arrive (SERVER.md).
  • Test node: server/cmd/fakenode speaks this protocol with stub models (-models stub-llm:chat adds an echoing chat model that streams); server/scripts/e2e.sh runs hub + fake node + requests end to end.