Skip to content

Model package format

A model package is a directory on the gateway:

/sdcard/Android/data/dev.kinherd.app/files/models/<id>/
manifest.json required, ≤ 64 KB, described below
<file>.tflite the model, named by "file"
labels.txt classifiers only, named by output.labels: one label per line, in output order
<other files> optional, each listed in "files" with its SHA-256
.verified written by the gateway: cached SHA-256 check (do not ship it)

The gateway scans models/ when the service starts. A package that fails any check is skipped, and the reason shows in the app’s status screen, in gateway.log and in GET /v1/models (rejected). A bad package never stops the others from loading. To install or update a package: copy the directory, then Stop and Start the gateway.

{
"schema": 1,
"id": "clip-l14-336",
"version": "1",
"name": "CLIP ViT-L/14-336 image encoder",
"task": "image-embedding",
"file": "clip_l14_336.tflite",
"sha256": "<64 lowercase hex chars>",
"runtime": "litert",
"accelerators": ["gpu", "cpu"],
"input": {
"kind": "image", "size": 336, "preprocess": "clip-two-view",
"layout": "NCHW", "dtype": "float32",
"mean": [0.48145466, 0.4578275, 0.40821073],
"std": [0.26862954, 0.26130258, 0.27577711]
},
"output": { "kind": "embedding", "dim": 768, "normalize": true },
"memoryMB": 700
}

A classifier:

{
"schema": 1, "id": "mobilenet-v2-int8", "version": "1", "task": "image-classification",
"file": "model.tflite", "sha256": "…", "runtime": "litert", "accelerators": ["npu", "cpu"],
"input": { "kind": "image", "size": 224, "preprocess": "center-crop", "layout": "NHWC", "dtype": "uint8" },
"output": { "kind": "labels", "labels": "labels.txt", "topK": 5, "softmax": false },
"memoryMB": 40
}
Field Required Rules
schema yes Must be 1. A later, incompatible format gets a new number.
id yes [a-z0-9][a-z0-9._-]{0,63}. Must equal the directory name. Used in API paths.
version yes Non-empty string, reported in every response.
name no Display name; defaults to id.
task yes image-embedding (output must be embedding) or image-classification (output must be labels).
file yes Plain file name ending in .tflite, no paths.
sha256 yes Lowercase hex SHA-256 of file. Checked before the package is offered.
runtime yes litert (the only runtime today).
accelerators yes Ordered preference, from gpu, npu, cpu, no duplicates. The first that loads wins (see below).
input.kind yes image.
input.size yes Square side in pixels, 1–4096.
input.preprocess yes clip-two-view, center-crop or resize (see below).
input.layout yes NCHW or NHWC.
input.dtype yes float32 or uint8.
input.mean, input.std float32 only Three numbers each (R, G, B) on the 0–1 scale; std > 0. Pixel = (v/255 − mean) / std. uint8 input is raw 0–255.
output.kind yes embedding or labels.
output.dim embedding Vector length, 1–65536.
output.normalize no Embedding: L2-normalise (default true).
output.labels labels File name of the label list.
output.labelsSha256 with output.labels Lowercase hex SHA-256 of the labels file, so the signature covers it. Required when the package is signed, unless files lists the labels file.
output.topK no Labels returned, 1–1000 (default 5).
output.softmax no Apply softmax to the (dequantised) logits (default false).
output.quantScale, output.quantZeroPoint no Dequantisation for uint8 outputs when the runtime cannot report the tensor’s own parameters (CompiledModel). The Interpreter path always uses the tensor’s parameters.
memoryMB yes Approximate resident memory when loaded, 1–64000. Used by the scheduler’s memory budget.
nnapiAccelerator no NNAPI device name used for npu in the nnapi build, e.g. "mtk-mdla_shim" ([A-Za-z0-9._-]{1,64}). When set, NNAPI’s own CPU fallback is disabled so the model really runs on that device. Ignored by the litert2 build.
files no [{"name": "<plain file name>", "sha256": "<64 lowercase hex>"}]: every file of the package other than manifest.json (the model and labels may be listed too; their hashes must then equal sha256 / output.labelsSha256). Files not covered by sha256, output.labelsSha256 or files are not part of the package. kinherd sign -hash manifest.json fills sha256, output.labelsSha256 and files from the files beside the manifest (skipping *.md notes) before signing.
signature no {"keyId": "<16 hex>", "sig": "<base64>"}, Ed25519 by a publisher key. See Signatures.
compute no Compute metadata for the hub’s device estimator (below). Covered by the signature like every field. Nodes only check that it is an object.
license, description no Free text, not interpreted.

Unknown fields are rejected, so a typo ("accelerator") fails loudly instead of being ignored. All problems are reported at once, e.g. input.size: must be 1..4096; accelerators[1]: duplicate gpu.

Optional. It tells the hub what one request costs, so it can estimate how a device it has never run the model on would do (DEVICE_PROBE.md, Estimates). Every field is optional; absent means unknown.

"compute": {
"gflopsPerInvoke": 383.1, "invokesPerRequest": 2,
"weightDtype": "float16", "quant": "fp16-weights", "weightsMB": 581.8, "peakActivationMB": 45.1,
"ops": ["FULLY_CONNECTED", "BATCH_MATMUL"],
"delegates": {"xnnpack": true, "gpu": true},
"reference": {"device": "Google Pixel 7 Pro", "backend": "litert-gpu", "p50ms": 1450},
"source": "tools/tflite_compute.py on clip_l14_336_fp16.tflite"
}
Field Rules
gflopsPerInvoke 10⁹ operations of one interpreter invoke (a multiply-add counts 2, as in the device probe), 0–10⁶
invokesPerRequest invokes per API request, 1–64 (clip-two-view: 2); default 1
weightDtype stored weights: float32 | float16 | int8 | uint8 | int16
quant none | fp16-weights (fp32 compute; CPU backends expand the weights to fp32 at load) | dynamic-int8 (int8 weights, float activations) | int8 (full integer)
weightsMB, peakActivationMB constant tensors as stored, and the peak of live intermediate tensors (MiB)
ops op types carrying the operations, largest first (TFLite builtin names, ≤ 32). FULLY_CONNECTED/BATCH_MATMUL first marks a transformer: its speed follows the probe’s matmul throughput, a CNN’s the reference CNN
delegates {xnnpack, gpu, nnapi, npu}: true = the whole graph is known to run on that delegate, false = known not to (that backend is then never proposed), missing = unknown
reference optional measured latency per request on a known device: {device (registry device name), backend, p50ms}
source how the numbers were obtained (≤ 500 characters)

How to fill it. python3 tools/tflite_compute.py model.tflite --json [--invokes 2] (standard library only: it parses the flatbuffer, runs nothing) prints gflopsPerInvoke (2 × MACs of CONV_2D, DEPTHWISE_CONV_2D, TRANSPOSE_CONV, FULLY_CONNECTED and BATCH_MATMUL from the static shapes, plus one per output element of every other op), weightDtype, quant, weightsMB, peakActivationMB (a liveness walk over the ops) and ops; copy those keys into the manifest. It reports 0.2985 GFLOP for the probe’s reference model aigw-ref-cnn-v1, the analytic value. delegates come from benchmarks (the registry), not from the file.

Compatibility. Older nodes reject unknown manifest fields, so a package that carries compute needs the Android gateway and the Linux node from the release that accepts it (both check only that it is an object). The published packages predate the field and are not re-signed for it: the hub keeps the same metadata in its catalogue (server/seed/registry.json, models[].compute, with source saying how each number was obtained), and a signed manifest’s compute wins over the catalogue’s when a package carries one.

All modes decode the image (down-sampled so its long side stays ≤ 1024 px), then build a size × size RGB tile.

  • clip-two-view: two runs per image. (1) The whole photo is letterboxed on the CLIP mean colour (122,116,104); (2) the centre square crop. Both outputs are L2-normalised, summed and normalised again. This matches the Badoo Guard embedder exactly, so vectors are comparable across both.
  • center-crop: the centre square, scaled to size.
  • resize: the whole image stretched to size × size.

The gateway walks accelerators in order and keeps the first runner that loads; GET /v1/models reports the winner as backend. What each entry means depends on the APK’s build flavor (see Phone and TV nodes):

litert2 (phone; LiteRT 2.3.2)

Accelerator float32 input uint8 input
gpu CompiledModel, GPU (litert-gpu) skipped (CompiledModel has no uint8 tensors)
npu CompiledModel, NPU (vendor dispatch library required) skipped
cpu CompiledModel, CPU (litert-cpu) classic Interpreter, XNNPACK (interpreter-xnnpack)

LiteRT 2.3.2’s classic Interpreter has no delegate API (no addDelegate, no NnApiDelegate), so this build cannot use NNAPI.

nnapi (TV; LiteRT 1.4.2, no 2.x code)

Accelerator float32 and uint8 input
gpu skipped
npu classic Interpreter with the NNAPI delegate: setAllowFp16(true), plus setAcceleratorName(nnapiAccelerator) and setUseNnapiCpu(false) when nnapiAccelerator is set (nnapi or nnapi:<name>). If the delegate fails to load, the same runner falls back to XNNPACK and reports interpreter-xnnpack (nnapi… failed: …).
cpu classic Interpreter, XNNPACK

A package meant for both devices can list ["gpu", "npu", "cpu"]: the phone takes gpu, the TV takes npu.

PackageVerifier is pluggable. The gateway runs:

  1. Sha256Verifier: the file must match sha256. A successful check is cached in .verified (sha256 size mtime), so a 600 MB model is hashed once, not on every start.
  2. SignatureVerifier: "signature": {"keyId", "sig"} is a base64 Ed25519 signature over the canonical manifest (the JSON without signature, RFC 8785: keys sorted, no whitespace, JavaScript number form) by a key in gateway.conf trustedPublishers. Because the manifest holds sha256, the signature also covers the model file. A package signed by a trusted key must verify; with "requireSignature": true unsigned, legacy (publicKeyId/sig) and unknown-key packages are rejected too. Sign with kinherd sign -key <publisher>.key manifest.json (server/).

Optional output.labelsSha256 (64 hex) pins a classifier’s label list inside the signed manifest, and files pins any other file. The hub’s package store requires them: it refuses a signed upload whose labels file has no hash, or with a file that is unlisted, missing or has another SHA-256 (RECIPES.md, Package store).

A node installs the packages a recipe names from GET {hub}/v1/packages/<id>/<version>/<file> (node token, ETag = the file’s SHA-256): manifest.json first, which must be signed by a trusted publisher, then the model (signed sha256) and the labels (output.labelsSha256, else the ETag). Files land in a hidden staging directory, are checked as a package, and move into app-internal files/models/<id>/ with one rename (the previous version is moved aside and deleted afterwards; an interrupted install is cleaned up on the next start). App-internal storage is scanned before the adb-pushed external directory and wins for the same id: the NNAPI vendor driver cannot read FUSE-backed external storage. Runners still map the model file rather than copy it.

On first start the gateway moves a legacy models/clip_l14_336.tflite into models/clip-l14-336/, hashes it once and writes the manifest above (version "1", memoryMB 700). If the migration is interrupted after the move, the next start finishes it. An existing package is never overwritten.

The package file clip_l14_336.tflite (610,773,616 bytes, fp16 weights, sha256 867e88f9e41e96f0d9b1d5e3a72f7d84cebe97a2a14ce1cd9e1924f5cb1e52cd) is the CLIP ViT-L/14-336 image encoder (batch 1, input [1,3,336,336] CLIP-normalised, output the 768-d projected embedding, not L2-normalised), built from the public openai/clip-vit-large-patch14-336 weights on Hugging Face.

  • How it was made: a conversion script (not part of this repository; a venv with torch transformers numpy litert-torch) converts the weights with litert-torch to an fp32 .tflite (~1.2 GB) and prints the cosine similarity of the rewrite and of LiteRT against the original (both ~1.0). The fp16 file was derived from that fp32 output with fp16 weight conversion; the exact command and the package versions (2026-11-10) were not recorded. To rebuild: run the script, convert the weights to fp16, check the cosine similarity against the original model on a few images (expect ≥ 0.999), then package it as above. A rebuild will not reproduce the same sha256, so publish it as a new package version.