OpenAI-compatible API
Base URL: https://<hub-host>/openai/v1, API key: a virtual key (sk-aigw-…). Existing OpenAI clients work by
changing base_url.
| Method | Path | Notes |
|---|---|---|
| GET | /models, /models/{id} |
{"object":"list","data":[{"id","object":"model","created","owned_by","task"}]}: node models plus configured chat aliases |
| POST | /embeddings |
see below |
| POST | /chat/completions |
node chat models (llama.cpp on kinherd-node) first, streaming and tool calls included; otherwise 404 unless openai.providers is configured, then passed through to the provider behind the alias |
| POST | /responses |
the Responses API on node chat models, stateless (see below) |
Embeddings
Section titled “Embeddings”input is a string or an array of up to 64 items. Each item is a data:image/…;base64,… string,
{"type":"image_url","image_url":{"url":"data:…"}} (or "image_url":"data:…"), or {"type":"text","text":…}.
encoding_format:floatorbase64(little-endian float32, the SDKs’ default).- Remote
http(s)image URLs are refused: the hub never fetches on your behalf. - Text goes only to
text-embeddingmodels; text sent to an image model is a 400 that says so. dimensions, if given, must equal the model’s size.usageis 0.
from openai import OpenAIimport base64
hub = OpenAI(base_url="https://<hub-host>/openai/v1", api_key="sk-aigw-…")
def data_url(path: str) -> str: return "data:image/jpeg;base64," + base64.b64encode(open(path, "rb").read()).decode()
res = hub.embeddings.create(model="clip-l14-336", input=[data_url("a.jpg"), data_url("b.jpg")])a, b = (d.embedding for d in res.data)print(sum(x * y for x, y in zip(a, b))) # cosine similarity (vectors are L2-normalised)curl https://<hub-host>/openai/v1/embeddings \ -H "Authorization: Bearer $KEY" -H 'Content-Type: application/json' \ -d "{\"model\":\"clip-l14-336\",\"input\":[{\"type\":\"image_url\",\"image_url\":{\"url\":\"data:image/jpeg;base64,$(base64 < a.jpg | tr -d '\n')\"}}]}"import OpenAI from 'openai';import { readFileSync } from 'node:fs';
const hub = new OpenAI({ baseURL: 'https://<hub-host>/openai/v1', apiKey: process.env.AIGW_KEY });const img = 'data:image/jpeg;base64,' + readFileSync('photo.jpg').toString('base64');const { data } = await hub.embeddings.create({ model: 'clip-l14-336', input: [img] });console.log(data[0].embedding.length);Chat completions (cloud fallback)
Section titled “Chat completions (cloud fallback)”The hub is not an LLM runtime today. With openai.providers and openai.models aliases configured, it passes
/chat/completions through to an upstream OpenAI-compatible API, with the same keys, scopes (on the alias name),
rate limits, quotas and audit as local models:
openai: providers: - {name: openai, baseURL: https://api.openai.com/v1, apiKeyEnv: OPENAI_API_KEY} models: gpt-small: {provider: openai, model: gpt-4o-mini}Errors on /openai/v1 use the same codes as the native API, plus OpenAI’s "type" field.
Responses API
Section titled “Responses API”POST /openai/v1/responses runs on the same node chat models as chat completions, with the same keys, scopes,
local-only rule, quotas and audit. It is stateless: nothing is stored, so send the whole conversation in
input each time (previous_response_id, conversation and background are refused with 400; store is
always answered as false).
input: a string, ormessage,function_callandfunction_call_outputitems;instructionsbecomes the system message.toolsof typefunctiononly; built-in tools (web search, file search, computer use) and file inputs are refused.max_output_tokensis capped by the hub’sopenai.maxTokens.stream: truesends the usual events (response.created,response.output_text.delta,response.function_call_arguments.delta, …,response.completed).- Cloud provider aliases are not served here; use
/chat/completionsfor them.
from openai import OpenAIhub = OpenAI(base_url="https://<hub-host>/openai/v1", api_key="sk-aigw-…")r = hub.responses.create(model="qwen2.5-1.5b-instruct", input="Name three uses of a spare phone.")print(r.output_text)Tool calls
Section titled “Tool calls”tools, tool_choice and parallel_tool_calls pass through to llama.cpp (at most 128 tools, 256 KiB), and answers
keep tool_calls with finish_reason: "tool_calls". The model must be set up for tools on its node
("tools": true, which starts llama-server with --jinja). Your client runs the tools; the hub and the node
never execute anything.
Sealed (end-to-end encrypted) bodies are accepted only on the native run API, not on /openai/v1: the hub would
have to read a request to translate it.