Skip to content

Run a hub

The hub is one static Go binary (pure-Go SQLite, no CGO) on a distroless, non-root image of about 17 MB. It keeps its state (keys, audit, usage, registry, waitlist) in one SQLite file under /data.

The Quickstart walks through this with a model and a node. In short:

Terminal window
cd server
cp .env.example .env # AIGW_ADMIN_KEY=$(openssl rand -hex 32), AIGW_KEY_PEPPER=$(openssl rand -hex 32),
# AIGW_PUBLISHER_KEYS=<your publisher public key> (kinherd keygen)
docker compose up -d --build
curl -s localhost:8080/healthz # {"ok":true,"name":…,"version":…,"commit":…,"uptimeSec":…}

Create a node key and a client key with the admin key (each secret is shown once):

Terminal window
ADMIN=$(grep ^AIGW_ADMIN_KEY .env | cut -d= -f2)
curl -s -H "Authorization: Bearer $ADMIN" -d '{"name":"tv","kind":"node"}' localhost:8080/v1/admin/keys
curl -s -H "Authorization: Bearer $ADMIN" -d '{"name":"mac","scopes":["*"]}' localhost:8080/v1/admin/keys

No phone at hand? Start the stub node, which speaks the real protocol with fake models:

Terminal window
go run ./cmd/fakenode -hub ws://localhost:8080 -token <node secret> &
curl -s -H "Authorization: Bearer <client secret>" localhost:8080/v1/models

The dashboard is at http://localhost:8080/admin (sign in with the admin key). Without Docker: go run ./cmd/server.

In production: Docker Swarm behind a CDN and a reverse proxy

Section titled “In production: Docker Swarm behind a CDN and a reverse proxy”

One way to run it: a single-node Docker Swarm on a Linux server, published through a CDN such as Cloudflare (TLS, WAF, rate limits) and a reverse proxy.

clients ──https──▶ CDN (TLS, WAF, rate limits, access control for /admin)
│
▼
reverse proxy ── X-Forwarded-For ──▶ container :8080 (one swarm service)
nodes ──wss://<hub-host>/v1/nodes/connect──┘ ├─ SQLite on a volume (/data)
└─ :9102 /metrics, internal network only
  1. Secrets (once, on the swarm manager):

    Terminal window
    openssl rand -hex 32 | docker secret create ai_gateway_admin_key -
    openssl rand -hex 32 | docker secret create ai_gateway_key_pepper -

    Keep a copy of the admin key in a password manager. Never rotate the pepper casually: every key hash depends on it.

  2. Deploy a stack with one replica and stop-first updates (SQLite has one writer), the container port 8080 published in host mode so the hub sees real client addresses, the secrets mounted under /run/secrets/, and a volume on /data (docs/SERVER.md, Deploy with Docker). After an update, wait until /healthz reports the new commit.

  3. Cloudflare: a proxied DNS record for the hub host, WebSockets on (for /v1/nodes/connect), and rate-limiting rules for /public/v1/waitlist, the dashboard login and the API. Put Cloudflare Access in front of /admin only, never in front of /v1, /openai, /public or /v1/nodes/connect (those use bearer keys).

  4. Reverse proxy: a backend pointing at the hub with option httpchk GET /livez, timeout server 100s and timeout tunnel 1h for node WebSockets. Deny bodies above 10 MB at the edge.

  5. Trust the proxy, and only the proxy: set AIGW_TRUSTED_PROXIES to the address the proxy connects from, and AIGW_TRUST_CF_CONNECTING_IP=true if Cloudflare is in front. The hub believes CF-Connecting-IP only when the hop before the proxy is a Cloudflare address.

  6. Allow the website: add the website’s origin to AIGW_PUBLIC_ORIGINS so its registry page and waitlist form can call /public/v1/* (CORS is an exact-origin allow-list).

Env Default Meaning
AIGW_NAME aigateway-hub hub name, returned as via
AIGW_ADMIN_KEY / _FILE none enables /v1/admin and the dashboard (≥ 32 chars)
AIGW_KEY_PEPPER / _FILE generated HMAC pepper for key hashes
AIGW_TRUSTED_PROXIES none addresses whose X-Forwarded-* are believed
AIGW_PUBLIC_ORIGINS none CORS allow-list for /public/v1
AIGW_PUBLISHER_KEYS none publisher public keys whose signed packages and recipes the hub accepts
AIGW_TRUST_DEV_KEY true also trust the repository’s DEV publisher key; false on a hub you rely on
AIGW_REQUIRE_NODE_APPROVAL true nodes paired with a code wait for approval on Nodes
AIGW_MARKETPLACE off the credits marketplace (frozen)
METRICS_ADDR none separate, never-published /metrics listener
AIGW_MAX_BODY_MB 10 request body limit

Every option is also available in a YAML config file (-config); unknown keys are rejected and all problems are reported at once.