Model package format
A model package is a directory on the gateway:
/sdcard/Android/data/dev.kinherd.app/files/models/<id>/ manifest.json required, ≤ 64 KB, described below <file>.tflite the model, named by "file" labels.txt classifiers only, named by output.labels: one label per line, in output order <other files> optional, each listed in "files" with its SHA-256 .verified written by the gateway: cached SHA-256 check (do not ship it)The gateway scans models/ when the service starts. A package that fails any check is skipped, and the reason
shows in the app’s status screen, in gateway.log and in GET /v1/models (rejected). A bad package never stops the
others from loading. To install or update a package: copy the directory, then Stop and Start the gateway.
manifest.json
Section titled “manifest.json”{ "schema": 1, "id": "clip-l14-336", "version": "1", "name": "CLIP ViT-L/14-336 image encoder", "task": "image-embedding", "file": "clip_l14_336.tflite", "sha256": "<64 lowercase hex chars>", "runtime": "litert", "accelerators": ["gpu", "cpu"], "input": { "kind": "image", "size": 336, "preprocess": "clip-two-view", "layout": "NCHW", "dtype": "float32", "mean": [0.48145466, 0.4578275, 0.40821073], "std": [0.26862954, 0.26130258, 0.27577711] }, "output": { "kind": "embedding", "dim": 768, "normalize": true }, "memoryMB": 700}A classifier:
{ "schema": 1, "id": "mobilenet-v2-int8", "version": "1", "task": "image-classification", "file": "model.tflite", "sha256": "…", "runtime": "litert", "accelerators": ["npu", "cpu"], "input": { "kind": "image", "size": 224, "preprocess": "center-crop", "layout": "NHWC", "dtype": "uint8" }, "output": { "kind": "labels", "labels": "labels.txt", "topK": 5, "softmax": false }, "memoryMB": 40}Fields
Section titled “Fields”| Field | Required | Rules |
|---|---|---|
schema |
yes | Must be 1. A later, incompatible format gets a new number. |
id |
yes | [a-z0-9][a-z0-9._-]{0,63}. Must equal the directory name. Used in API paths. |
version |
yes | Non-empty string, reported in every response. |
name |
no | Display name; defaults to id. |
task |
yes | image-embedding (output must be embedding) or image-classification (output must be labels). |
file |
yes | Plain file name ending in .tflite, no paths. |
sha256 |
yes | Lowercase hex SHA-256 of file. Checked before the package is offered. |
runtime |
yes | litert (the only runtime today). |
accelerators |
yes | Ordered preference, from gpu, npu, cpu, no duplicates. The first that loads wins (see below). |
input.kind |
yes | image. |
input.size |
yes | Square side in pixels, 1–4096. |
input.preprocess |
yes | clip-two-view, center-crop or resize (see below). |
input.layout |
yes | NCHW or NHWC. |
input.dtype |
yes | float32 or uint8. |
input.mean, input.std |
float32 only | Three numbers each (R, G, B) on the 0–1 scale; std > 0. Pixel = (v/255 − mean) / std. uint8 input is raw 0–255. |
output.kind |
yes | embedding or labels. |
output.dim |
embedding | Vector length, 1–65536. |
output.normalize |
no | Embedding: L2-normalise (default true). |
output.labels |
labels | File name of the label list. |
output.labelsSha256 |
with output.labels |
Lowercase hex SHA-256 of the labels file, so the signature covers it. Required when the package is signed, unless files lists the labels file. |
output.topK |
no | Labels returned, 1–1000 (default 5). |
output.softmax |
no | Apply softmax to the (dequantised) logits (default false). |
output.quantScale, output.quantZeroPoint |
no | Dequantisation for uint8 outputs when the runtime cannot report the tensor’s own parameters (CompiledModel). The Interpreter path always uses the tensor’s parameters. |
memoryMB |
yes | Approximate resident memory when loaded, 1–64000. Used by the scheduler’s memory budget. |
nnapiAccelerator |
no | NNAPI device name used for npu in the nnapi build, e.g. "mtk-mdla_shim" ([A-Za-z0-9._-]{1,64}). When set, NNAPI’s own CPU fallback is disabled so the model really runs on that device. Ignored by the litert2 build. |
files |
no | [{"name": "<plain file name>", "sha256": "<64 lowercase hex>"}]: every file of the package other than manifest.json (the model and labels may be listed too; their hashes must then equal sha256 / output.labelsSha256). Files not covered by sha256, output.labelsSha256 or files are not part of the package. kinherd sign -hash manifest.json fills sha256, output.labelsSha256 and files from the files beside the manifest (skipping *.md notes) before signing. |
signature |
no | {"keyId": "<16 hex>", "sig": "<base64>"}, Ed25519 by a publisher key. See Signatures. |
compute |
no | Compute metadata for the hub’s device estimator (below). Covered by the signature like every field. Nodes only check that it is an object. |
license, description |
no | Free text, not interpreted. |
Unknown fields are rejected, so a typo ("accelerator") fails loudly instead of being ignored. All problems are
reported at once, e.g. input.size: must be 1..4096; accelerators[1]: duplicate gpu.
Compute metadata (compute)
Section titled “Compute metadata (compute)”Optional. It tells the hub what one request costs, so it can estimate how a device it has never run the model on would do (DEVICE_PROBE.md, Estimates). Every field is optional; absent means unknown.
"compute": { "gflopsPerInvoke": 383.1, "invokesPerRequest": 2, "weightDtype": "float16", "quant": "fp16-weights", "weightsMB": 581.8, "peakActivationMB": 45.1, "ops": ["FULLY_CONNECTED", "BATCH_MATMUL"], "delegates": {"xnnpack": true, "gpu": true}, "reference": {"device": "Google Pixel 7 Pro", "backend": "litert-gpu", "p50ms": 1450}, "source": "tools/tflite_compute.py on clip_l14_336_fp16.tflite"}| Field | Rules |
|---|---|
gflopsPerInvoke |
10⁹ operations of one interpreter invoke (a multiply-add counts 2, as in the device probe), 0–10⁶ |
invokesPerRequest |
invokes per API request, 1–64 (clip-two-view: 2); default 1 |
weightDtype |
stored weights: float32 | float16 | int8 | uint8 | int16 |
quant |
none | fp16-weights (fp32 compute; CPU backends expand the weights to fp32 at load) | dynamic-int8 (int8 weights, float activations) | int8 (full integer) |
weightsMB, peakActivationMB |
constant tensors as stored, and the peak of live intermediate tensors (MiB) |
ops |
op types carrying the operations, largest first (TFLite builtin names, ≤ 32). FULLY_CONNECTED/BATCH_MATMUL first marks a transformer: its speed follows the probe’s matmul throughput, a CNN’s the reference CNN |
delegates |
{xnnpack, gpu, nnapi, npu}: true = the whole graph is known to run on that delegate, false = known not to (that backend is then never proposed), missing = unknown |
reference |
optional measured latency per request on a known device: {device (registry device name), backend, p50ms} |
source |
how the numbers were obtained (≤ 500 characters) |
How to fill it. python3 tools/tflite_compute.py model.tflite --json [--invokes 2] (standard library only:
it parses the flatbuffer, runs nothing) prints gflopsPerInvoke (2 × MACs of CONV_2D, DEPTHWISE_CONV_2D,
TRANSPOSE_CONV, FULLY_CONNECTED and BATCH_MATMUL from the static shapes, plus one per output element of every other
op), weightDtype, quant, weightsMB, peakActivationMB (a liveness walk over the ops) and ops; copy those
keys into the manifest. It reports 0.2985 GFLOP for the probe’s reference model aigw-ref-cnn-v1, the analytic
value. delegates come from benchmarks (the registry), not from the file.
Compatibility. Older nodes reject unknown manifest fields, so a package that carries compute needs the
Android gateway and the Linux node from the release that accepts it (both check only that it is an object). The
published packages predate the field and are not re-signed for it: the hub keeps the same metadata in its
catalogue (server/seed/registry.json, models[].compute, with source saying how each number was obtained),
and a signed manifest’s compute wins over the catalogue’s when a package carries one.
Preprocessing
Section titled “Preprocessing”All modes decode the image (down-sampled so its long side stays ≤ 1024 px), then build a size × size RGB tile.
clip-two-view: two runs per image. (1) The whole photo is letterboxed on the CLIP mean colour (122,116,104); (2) the centre square crop. Both outputs are L2-normalised, summed and normalised again. This matches the Badoo Guard embedder exactly, so vectors are comparable across both.center-crop: the centre square, scaled tosize.resize: the whole image stretched tosize × size.
Accelerator selection
Section titled “Accelerator selection”The gateway walks accelerators in order and keeps the first runner that loads; GET /v1/models reports the
winner as backend. What each entry means depends on the APK’s build flavor (see Phone and TV nodes):
litert2 (phone; LiteRT 2.3.2)
| Accelerator | float32 input | uint8 input |
|---|---|---|
gpu |
CompiledModel, GPU (litert-gpu) |
skipped (CompiledModel has no uint8 tensors) |
npu |
CompiledModel, NPU (vendor dispatch library required) | skipped |
cpu |
CompiledModel, CPU (litert-cpu) |
classic Interpreter, XNNPACK (interpreter-xnnpack) |
LiteRT 2.3.2’s classic Interpreter has no delegate API (no addDelegate, no NnApiDelegate), so this build
cannot use NNAPI.
nnapi (TV; LiteRT 1.4.2, no 2.x code)
| Accelerator | float32 and uint8 input |
|---|---|
gpu |
skipped |
npu |
classic Interpreter with the NNAPI delegate: setAllowFp16(true), plus setAcceleratorName(nnapiAccelerator) and setUseNnapiCpu(false) when nnapiAccelerator is set (nnapi or nnapi:<name>). If the delegate fails to load, the same runner falls back to XNNPACK and reports interpreter-xnnpack (nnapi… failed: …). |
cpu |
classic Interpreter, XNNPACK |
A package meant for both devices can list ["gpu", "npu", "cpu"]: the phone takes gpu, the TV takes npu.
Signatures
Section titled “Signatures”PackageVerifier is pluggable. The gateway runs:
Sha256Verifier: the file must matchsha256. A successful check is cached in.verified(sha256 size mtime), so a 600 MB model is hashed once, not on every start.SignatureVerifier:"signature": {"keyId", "sig"}is a base64 Ed25519 signature over the canonical manifest (the JSON withoutsignature, RFC 8785: keys sorted, no whitespace, JavaScript number form) by a key in gateway.conftrustedPublishers. Because the manifest holdssha256, the signature also covers the model file. A package signed by a trusted key must verify; with"requireSignature": trueunsigned, legacy (publicKeyId/sig) and unknown-key packages are rejected too. Sign withkinherd sign -key <publisher>.key manifest.json(server/).
Optional output.labelsSha256 (64 hex) pins a classifier’s label list inside the signed manifest, and files
pins any other file. The hub’s package store requires them: it refuses a signed upload whose labels file has no
hash, or with a file that is unlisted, missing or has another SHA-256 (RECIPES.md, Package store).
Packages from the hub
Section titled “Packages from the hub”A node installs the packages a recipe names from GET {hub}/v1/packages/<id>/<version>/<file> (node token, ETag =
the file’s SHA-256): manifest.json first, which must be signed by a trusted publisher, then the model (signed
sha256) and the labels (output.labelsSha256, else the ETag). Files land in a hidden staging directory, are
checked as a package, and move into app-internal files/models/<id>/ with one rename (the previous version is
moved aside and deleted afterwards; an interrupted install is cleaned up on the next start). App-internal storage
is scanned before the adb-pushed external directory and wins for the same id: the NNAPI vendor driver cannot read
FUSE-backed external storage. Runners still map the model file rather than copy it.
Migration from the MVP
Section titled “Migration from the MVP”On first start the gateway moves a legacy models/clip_l14_336.tflite into models/clip-l14-336/, hashes it once and
writes the manifest above (version "1", memoryMB 700). If the migration is interrupted after the move, the next
start finishes it. An existing package is never overwritten.
Provenance: clip-l14-336 v1
Section titled “Provenance: clip-l14-336 v1”The package file clip_l14_336.tflite (610,773,616 bytes, fp16 weights, sha256
867e88f9e41e96f0d9b1d5e3a72f7d84cebe97a2a14ce1cd9e1924f5cb1e52cd) is the CLIP ViT-L/14-336 image encoder
(batch 1, input [1,3,336,336] CLIP-normalised, output the 768-d projected embedding, not L2-normalised), built
from the public openai/clip-vit-large-patch14-336 weights on Hugging Face.
- How it was made: a conversion script (not part of this repository; a venv with
torch transformers numpy litert-torch) converts the weights withlitert-torchto an fp32.tflite(~1.2 GB) and prints the cosine similarity of the rewrite and of LiteRT against the original (both ~1.0). The fp16 file was derived from that fp32 output with fp16 weight conversion; the exact command and the package versions (2026-11-10) were not recorded. To rebuild: run the script, convert the weights to fp16, check the cosine similarity against the original model on a few images (expect ≥ 0.999), then package it as above. A rebuild will not reproduce the same sha256, so publish it as a new package version.