NAME
Langertha::Skeid - Dynamic routing control-plane for multi-node LLM serving with normalized metrics and cost accounting
VERSION
version 0.004
SYNOPSIS
use Langertha::Skeid;
my $skeid = Langertha::Skeid->new(
config_file => '/etc/skeid/config.yaml',
);
my $cost = $skeid->call_function('metrics.estimate_cost', {
model => 'gpt-4o-mini',
usage => { prompt_tokens => 1000, completion_tokens => 200 },
});DESCRIPTION
Langertha::Skeid is a routing control-plane for provider-style LLM operations. It keeps a live node table, routes by model/health/capacity, and records normalized token/cost usage.
Skeid is commonly used as one API edge in front of many upstream APIs (cloud + local). With pricing and usage.record/report, you can build tenant billing from one consistent ledger.
Multi-API Billing Flow
1. Define multiple nodes in config (for example OpenAI-compatible cloud APIs and local vLLM/SGLang). 2. Set model pricing via pricing or pricing.set. 3. Let tenant identity follow from the API key the caller presents. skeid keyid prints the id a given key resolves to. A deployment that authenticates callers in front of Skeid can instead pass x-skeid-key-id and set routing.trust_key_id_header. 4. Read totals by key/model/time with usage.report.
Engine IDs
nodes[].engine uses lowercased engine class names from Langertha. Examples: OpenAI => openai, OpenAIBase => openaibase, vLLM => vllm. Legacy aliases like openai-compatible are intentionally rejected. A node without engine is openaibase. See "normalize_engine_id".
Pluggable Usage Storage
The usage storage layer is pluggable. Built-in backends are jsonlog (recommended, no DBI required), sqlite, and postgresql. You can also replace the storage layer entirely via constructor callbacks or subclass override.
jsonlog backend (recommended — no DBI dependency):
# Directory mode: one JSON file per event (no collision risk)
my $skeid = Langertha::Skeid->new(
usage_store => { backend => 'jsonlog', path => '/var/log/skeid/events/' },
);
# File mode: JSON-lines appended to a single file
my $skeid = Langertha::Skeid->new(
usage_store => { backend => 'jsonlog', path => '/var/log/skeid/usage.jsonl', mode => 'file' },
);Directory mode is auto-detected when the path is an existing directory or ends with /. It writes one .json file per event, which avoids file-level locking and concurrent-write collisions entirely.
Constructor callbacks (custom backend, no subclassing):
my $skeid = Langertha::Skeid->new(
store_usage_event => sub {
my ($self, $event) = @_;
# $event is a hashref with all normalized usage columns
publish_to_nats($event);
return { ok => 1 };
},
query_usage_report => sub {
my ($self, $filters) = @_;
# $filters has: since, api_key_id, model, limit
return { ok => 1, enabled => 1, totals => { ... } };
},
);Subclass override:
package MyApp::Skeid;
use Moo;
extends 'Langertha::Skeid';
sub _store_usage_event {
my ($self, $event) = @_;
...
return { ok => 1 };
}
sub _query_usage_report {
my ($self, $filters) = @_;
...
}Optional fields. A streamed request's event also carries content_bytes: the UTF-8 byte count of the content Skeid relayed (OpenAI face) or translated (Anthropic and Ollama faces). It is set on every stream, including one whose upstream reported token counts, and is absent from non-streamed events -- a sink must not assume it is there. It is an observation, not a billing quantity: Skeid never derives token counts or cost from it. The DBI stores keep it in a nullable content_bytes column (NULL for a non-streamed or pre-existing row); jsonlog writes it as part of the event.
An audio route's event (/v1/audio/transcriptions, /v1/audio/translations) carries audio_seconds when the node said how much audio it worked on: usage.seconds of an answer whose usage.type is duration, else the duration of a verbose_json answer. A node that reports neither -- a plain json, text, srt or vtt answer, most streams -- leaves the key out: not measured, never a zero. It is recorded as the node reported it and is not priced (ADR 0021); the DBI stores keep it in a nullable audio_seconds column, jsonlog writes it as part of the event, and a report sums it in totals, by_key and by_model.
A rerank request's event (/v1/rerank, also for the /rerank spelling) carries documents: how many documents the request held. Skeid counts them off the request, so the number does not depend on what the node reports; it is on the event of a request the node answered and absent from a failed one. The node's tokens are recorded as input tokens -- a reranker generates nothing -- so the model's input_per_million prices them, wherever the node reported them: usage.prompt_tokens or usage.total_tokens, Cohere's meta.tokens.input_tokens, a TEI node's x-compute-tokens header ("usage" in Langertha::Skeid::Protocol::Rerank). documents itself is recorded and not priced, and is stored like audio_seconds: a nullable documents column, a key in jsonlog, a sum in the reports.
A documents request's event (/v1/convert/file, /v1/convert/source; api_format docling, model documents) carries documents too -- the files or sources the request held, on the event of a request the node answered -- and pages: the page count of the converted document, read off document.json_content.pages of a JSON answer of up to 8 MiB. A zip, a larger answer, an answer without json_content and a document format without pages leave pages out: not measured, never a zero. It is the unit to bill a conversion by, recorded and not priced, stored like documents ("pages" in Langertha::Skeid::Protocol::Documents).
When a callback or override is provided, the configured store is bypassed entirely and no database connection is created. DBI, DBD::SQLite and DBD::Pg are recommends dependencies — they are not required for jsonlog or when usage is handled externally. With no sink at all, "record_usage" builds no event and answers { ok => 0, enabled => 0, error => 'usage_store not configured' }. A sink that fails answers { ok => 0, error => ... } (or dies); the proxy logs that at error level as a lost usage event, naming the request id and the store backend.
Per-Key Routing Policy
Which nodes a customer key may be served from, and which models it may ask for:
policies:
standard: { deny_tags: [cloud] } # our own hardware only
burstable: {} # cloud is fine when local is full
default_policy: standard
names:
alice: k_5f0e1a2b3c4de5f60718293a4b5c6d7e8f901a2b # id from `skeid keyid <key>` -- never the key
bigcorp: k_9c8b7a6f5e4de5f60718293a4b5c6d7e8f901a2b
keys:
alice: burstable # a keys: entry may be written by name ...
bigcorp:
policy: standard
models: [house-model] # sparse override of one field
k_1122334455aae5f60718293a4b5c6d7e8f901a2b: burstable # ... or still by the raw key idResolved once at config load: a request costs one hash lookup, keys on the same profile share one policy object, and a key that takes the default is not listed at all. deny_tags filters node selection, not just the plan, so a denied node cannot be reached by asking for its own model name instead of an alias. A refusal is 403, never a capacity error.
The optional names: section maps a readable name to a customer key id (karr #17). It makes keys: entries legible and confines key rotation to a single line -- change the id a name points at, and every policy line that named the customer follows. Its values are ids from skeid keyid, never customer keys, so the config still holds no secret; it is read only at config load and never on the request path.
Identity comes from the API key the caller presented — see "key_id_for_key". See also ADR 0008 in the distribution repository.
Admin API Key
admin.api_key (or admin_api_key) controls access to proxy admin routes; admin.api_key_env (or admin_api_key_env) names an environment variable to read it from instead, so the key need not be written into a mounted file. If empty, admin routes are effectively disabled by returning 404. If set, the proxy expects Authorization: Bearer .... An explicit key (skeid serve --admin-api-key, "set_admin_api_key") outranks the config, and SKEID_ADMIN_API_KEY stands in when the config names none; a reload re-applies that order. See "admin".
Provider Manifest
GET /.well-known/langertha.json serves a provider manifest (core's Langertha::Manifest, schema v1) to a customer key -- listing only the models that key's own keys: entry names. Nothing is published automatically: the route answers 404 until the config turns it on, and a key without a manifest: entry gets 403, never the catalog.
manifest:
enabled: true
public_url: https://llm.example.com # where clients reach Skeid -- never a node URL
provider_id: example-llm # default: skeid
faces: [openai, anthropic, ollama] # default: all three
capabilities: # optional claims per model; default chat + streaming
house-model: { tools_native: true, tool_choice_auto: true }
names: # ids from `skeid keyid <key>` -- never the key
alice: k_5f0e1a2b3c4de5f60718293a4b5c6d7e8f901a2b
bob: k_9c8b7a6f5e4de5f60718293a4b5c6d7e8f901a2b
keys:
alice:
policy: burstable
manifest: { models: [house-model, qwen3-32b] }
bob:
manifest: { models: [house-model] }One endpoint per face, all under public_url and all with one api_key auth entry (the Authorization: Bearer / x-api-key scheme every face already takes): openai (openai-chat, public_url/v1), anthropic (anthropic-compat at public_url -- Skeid translates /v1/messages to the OpenAI upstream call and does not carry output_config.format, so structured output takes the synthetic-tool path) and ollama (public_url). Every listed model appears on every published face. Its capabilities default to chat and streaming, take the claims declared for it (only flags from "model_capabilities" in Langertha::Manifest::Builder), and are then cut per face to what that face's translator carries upstream -- a claim holds at that endpoint or is not made there. The lists are "openai_manifest_endpoint" in Langertha::Skeid::Protocol, "manifest_endpoint" in Langertha::Skeid::Protocol::Anthropic and "manifest_endpoint" in Langertha::Skeid::Protocol::Ollama.
Resolved at config load, like the routing policy: each key's manifest is built and validated once and stored under the key id, so a request costs one hash lookup and one key's manifest cannot be served for another. The load croaks on a model the key's routing policy does not let it reach (not granted, or served only by nodes it is denied), on an unknown capability, one no face carries, an unknown face, and an enabled manifest without public_url. A config that fails to load keeps the previous one in force, manifests included. Like every client route, it runs "maybe_reload_config" before it answers, so a changed config is in force for it too. Without a key the route answers 401. Needs a Langertha with Langertha::Manifest; on an older one the route answers 404. See ADR 0015 in the distribution repository.
CONFIGURATION
The config is a YAML file (config_file) or the hashref a config_loader returns; both go through the same loader. The top-level keys below are every one Skeid reads -- anything else in the file is ignored. A reload is all or nothing: a config that fails anywhere leaves the previous one in force, and is reported by "reload_status".
A reload makes the running config equal to the file (skeid k65), as a restart with that file would:
nodes,pricing,aliases, the policy keys (policies,default_policy,names,keys) andusage_store: present, the section replaces what is loaded --pricingtoo, so a model removed from it loses its price. Removed from the file, the section goes back to empty with the next reload: no nodes, no prices, no aliases, no policies.routing: each key present sets its value; a key removed from the file goes back to the value it had before any config was applied -- passed tonew, else from "ENVIRONMENT", else built in.The admin key: absent falls back to
SKEID_ADMIN_API_KEY, else off -- see "admin".registryandmanifest: absent means off.client_auth: removed from the file, client authentication is off again.uploads: removed from the file, the upload limits are the built-in defaults again.documents: removed from the file, remote sources are refused again.realtime: removed from the file, the realtime limits are the built-in defaults again.mcp: removed from the file, the MCP gateway is off again and its routes are unknown paths.openapi: removed from the file, the Swagger UI is off again and/docsis an unknown path;/openapi.jsonis served either way.customer_source: removed from the file, the fetcher stops and the source's rows are dropped at that reload; changed, the fetcher restarts and the rows in force stay until the new source answered. See "customer_source".
Two exceptions. A section no applied config has declared belongs to whoever set it: nodes added through the admin API or "add_node", prices from "set_model_pricing", values passed to new -- a config without that section leaves them alone. And the usage store is not removed by a reload: a usage_store taken out of the file keeps the running store in force until restart, with one warning (usage_store was removed from the config ...), because a reload that stopped recording usage events would lose billing data silently (ADR 0004). A changed usage_store is swapped at once, the new store prepared before the old one is let go.
Three settings look like config but are not read from it: "capacity_max_age_ms" and "config_reload_interval" (constructor or environment only) and "worker_count" (constructor, set by skeid serve --workers).
nodes
nodes:
- id: gpu-1 # required, unique
url: http://gpu-1:8000/v1 # required; /v1 is added when the URL does not end in it
model: qwen3-32b # default '': matches any requested model
engine: vllm # default openaibase; see "Engine IDs"
weight: 3 # default 1; round-robin weight, integer, below 1 counts as 1
max_conns: 8 # default 0 = unlimited; admission limit, see worker_max_conns
healthy: true # default true; operator state, never derived from errors
tags: [local, gb10] # default none; "local, gb10" works too, see normalize_tags
metadata: { rack: b2 } # default {}; kept on the node, never read by routing
api_key_ref: secret/skeid/remote/groq # upstream key, resolved through the key broker
api_key_env: GROQ_API_KEY # upstream key from this variable when the ref resolves none
capacity: { probe: prometheus } # default none: inflight admission, see below
rerank_format: tei # default none; a reranker that speaks TEI's dialect, see below
kind: documents # default chat; a docling-serve node, see below
api_key_header: X-Api-Key # default none: the key goes in Authorization: BearerAn entry without id or url is skipped; one with an unknown engine, rerank_format or kind, or an api_key_header that is no header name, fails the load. When a node has a key of its own it replaces the client's Authorization upstream (see Langertha::Skeid::Proxy); otherwise the client's headers go through. Key references and variable names are the only key material a config holds (ADR 0003).
The section replaces the whole inventory when it changed -- nodes added through the admin API are lost then -- and keeps the node list as it is when it did not: its inventory generation, its running probes and any health set through the admin API. Removing the section empties the inventory.
capacity selects how admission learns a node's real occupancy (ADR 0009). probe (or type) is inflight (the default, also none), ratelimit, prometheus, registry or custom; interval_ms (default 2000) sets the poll rate. Rate-limit headers are read off every upstream response whatever the block says, so ratelimit starts nothing. The keys per probe are documented in "for_node" in Langertha::Skeid::CapacityProbe, Langertha::Skeid::CapacityProbe::Prometheus and Langertha::Skeid::CapacityProbe::Registry.
rerank_format names the dialect a reranker node speaks on its rerank route when it is not the one clients speak. Without it -- vLLM, infinity started with --url-prefix /v1, Jina, Cohere -- a /v1/rerank request is relayed to {url}/rerank as it came. tei is Hugging Face text-embeddings-inference: the request is translated and sent to /rerank at the server root (the node's url with or without a trailing /v1), the answer translated back. It is the only value; see "Rerank route" in Langertha::Skeid::Proxy.
kind says what a node serves. The default, chat, is every node there was: chat completions and embeddings, matched by model. documents is a docling-serve node: it is reached by /v1/convert/file and /v1/convert/source and by nothing else, and the chat routes, the model lists and the manifest never see it (ADR 0028). Its model is not read. The documents route is planned under the routing key documents, which a policy's models grants and an alias of that name tiers (by tags); a key whose models do not name it is answered 403.
api_key_header names the header a node's key is sent in, raw, instead of Authorization: Bearer: X-Api-Key for docling-serve, which takes no Bearer token. The key source (api_key_ref, api_key_env) is unchanged and the client's own credentials are dropped in every spelling, as for any node with a key. A name that is no RFC 7230 token fails the load; Authorization is the default spelled out.
pricing
pricing:
gpt-4o-mini:
input_per_million: 0.15 # default 0
output_per_million: 0.60 # default 0
cached_input_per_million: 0.075 # optional; prompt-cache reads
cache_write_per_million: 0.1875 # optional; prompt-cache writes
'*': # fallback for every model not listed
input_per_million: 0
output_per_million: 0Keyed by the served model -- the name the node is asked for, not an alias. The section is the whole price list: a reload replaces it, it is not merged. A cache rate must be a number >= 0, or the load fails; on a Langertha that cannot price cache tokens the cache rates are dropped with one warning and cached tokens bill at input_per_million. See "set_model_pricing".
aliases
aliases:
house-model:
tiers:
- { tags: [local], model: qwen3-32b, wait_ms: 200 }
- { tags: [cloud], model: llama-3.3-70b-versatile }A client-facing model name as an ordered plan of tiers (ADR 0008). Per tier: tags, model (default: the alias name), engine and wait_ms (default 0). tiers may be given as a bare list. See "set_model_alias".
policies, default_policy, names, keys
Who may reach what; see "Per-Key Routing Policy". A policy is models (or aliases: the requested names the key may use; absent or '*' means all) and deny_tags (nodes the key must never reach). default_policy names the policy of every unlisted key; unset, an unlisted key is unrestricted. names maps a readable name to a key id from skeid keyid.
A keys entry is keyed by a key id or a names name and is either a policy name or a hash:
keys:
alice: burstable
bigcorp:
policy: standard # default: default_policy
models: [house-model] # sparse overrides: an absent field keeps the policy's value
deny_tags: [cloud]
manifest: { models: [house-model] } # see "Provider Manifest"The override fields may instead sit under an overrides hash. The load fails on a default_policy or a keys entry naming an undefined policy, on a names value that is not a non-empty string, and on two entries that resolve to one key id -- including a short (pre-ADR 0016) id beside the full id it is the prefix of. Short ids still match, with a one-time warning.
client_auth
names:
shared: k_5f0e1a2b3c4de5f60718293a4b5c6d7e8f901a2b # `skeid keyid <key>` -- never the key
client_auth:
keys: [shared, k_9c8b7a6f5e4de5f60718293a4b5c6d7e8f901a2b]Client authentication (skeid k90), for a Skeid that is itself the public entry rather than sitting behind a gateway that authenticates. With the section, the client routes -- /v1/*, /api/* and the provider manifest -- answer 401 to a request without a key or with a key whose id is not on the list, before anything is routed, forwarded or metered; /health and the /skeid/* routes keep their own rules. Without it nothing is checked, as before. See "Customer identity" in Langertha::Skeid::Proxy.
keys lists customer key ids from skeid keyid, or names: entries that map to one; a single id may stand without the list. A short (pre-ADR 0016) id matches the full id it is the prefix of, with the same deprecation warning as in keys:. The load fails on a section without keys, an unknown key in the section, and an entry that is neither a name nor a key id -- anonymous included; that message names the entry by its position, never by its value, since a value that is neither is most likely a key pasted where its id belongs (ADR 0003). An empty list loads, lets no key in, and says so once. The list is independent of keys: and default_policy: being let in does not widen what a key's policy routes it to.
The check takes the same identity routing does: with routing.trust_key_id_header the header's key id is what has to be on the list.
customer_source
customer_source:
backend: postgresql # the only backend; the default
dsn: dbi:Pg:dbname=console;host=db # or host / port / dbname, as in usage_store
user: skeid_reader
password_env: SKEID_CUSTOMER_DB_PASSWORD # never a password in the file
# from_usage_store: true # instead: the usage store's connection (postgresql);
# # user / password_env here override its own
relation: skeid.customer_keys # table or view, schema-qualified allowed
refresh_interval_s: 30 # default 30; the poll that always runs
listen: skeid_customer_keys # optional NOTIFY channel: a notification fetches sooner
max_stale_s: 0 # default 0: keep the last good rows during an outage
connect_timeout_s: 5 # default 5; also bounds each queryCustomer keys from a database relation (ADR 0026), for when another application -- a self-service console -- issues and revokes them. Skeid runs one statement, SELECT key_id, name, policy, models FROM "skeid"."customer_keys", and nothing else; the owner provides the relation, typically as a view over its own tables, and grants SELECT on it. The relation is quoted as written, so its case counts. One row per customer key id:
key_id(text, not null):k_+ 40 lowercase hex, asskeid keyidprints it.name(text): a label for warnings and/skeid/config. Never an identity, and not anames:entry: the file cannot refer to it.policy(text): a profile of the file'spolicies;NULLtakesdefault_policy.models(text[]): replaces the profile's model list, as akeys:override does;NULLkeeps it,'{}'is no models.
A row present means the key is active: the owner revokes a key by making its row disappear. The relation decides only which ids exist and which existing profile each takes -- profiles, deny_tags, aliases, nodes and the manifest stay in the file.
The rows of the last successful fetch -- the source snapshot -- are merged into the file's config at every reload, before any section is read:
Rows are checked first. A row is rejected, and grants nothing, when its
key_idis not a full key id, when another row has the samekey_id(both are rejected), whenpolicynames no profile, or whenmodelsis not a list of names. A rejected row is reported by itsname, else by its position, never by its values. No row can fail a reload.The file wins per key id. A row whose id the file's
keys:names -- directly, throughnames:, or by a short id that is its prefix -- is shadowed: itspolicyandmodelsare ignored.Every accepted row's id joins
client_auth, beside the file's own ids: the database cannot revoke an id the file lists. It does so only when the file declaresclient_auth--client_auth: { keys: [] }is enough, and its empty list is then not warned about. Withoutclient_auththe load warns once: nothing is gated, and a revoked key routes underdefault_policy.
The rows are fetched on the event loop, never on the request path (ADR 0005): every refresh_interval_s, and sooner after a NOTIFY on listen (one fetch per 0.25 s, however many arrive), each worker for itself. A snapshot whose digest changed triggers a reload from the fetch's callback; an unchanged one does nothing. The config's fingerprint is the file's plus the snapshot's digest, so a reload a request triggers still only reads the file.
When a fetch fails, the last good snapshot stays in force, the fetch is retried with the reload back-off (refresh_interval_s doubling per failure, at most 60 s), and the reason is logged when it is new. max_stale_s above 0 drops the source's rows once the last good snapshot is that old, until a fetch succeeds. A fetch that has not answered within connect_timeout_s counts as failed. skeid serve runs one fetch before it listens; when it fails, Skeid starts degraded and fail-closed -- the file's config in force, no source rows. GET /skeid/config shows the source's state ("customer_source_status"), /health only ok, failed_at and failures.
The section is checked at load: an unknown key, a password (only password_env), a backend other than PostgreSQL, a relation that is not a plain or schema-qualified identifier, a listen that is not an identifier, and from_usage_store without a PostgreSQL usage store fail it, as does a missing DBD::Pg -- a recommends, loaded only for a configured source. The connect and the query run in a child process, so a slow database never blocks the event loop. See Langertha::Skeid::CustomerSource and Langertha::Skeid::CustomerSource::DBI.
uploads
uploads:
max_bytes: 26214400 # default 26214400 (25 MiB); an integer of at least 1
documents_max_bytes: 104857600 # default 104857600 (100 MiB); an integer of at least 1The largest request body the upload routes take -- /v1/audio/transcriptions and /v1/audio/translations, the routes with a multipart/form-data body: the file plus the form around it. A larger one is answered 413 before any node is asked and without a usage event; see "Uploads" in Langertha::Skeid::Proxy. Every other route keeps the server's own request limit (Mojolicious' MOJO_MAX_MESSAGE_SIZE, 16 MiB), whatever this says. The load fails on a section that is not a hash, an unknown key in it, and a max_bytes that is not an integer of at least 1 -- there is no "unlimited". A changed value holds from the next request.
documents_max_bytes is the limit of the documents routes, /v1/convert/file and /v1/convert/source, which have their own because documents are bigger than audio. It bounds the request body: a source request carries its files as base64 in JSON, a third more than the file, and is decoded in memory -- send large files to /v1/convert/file, which is spooled to disk. The node's own time to convert is the upstream timeout, unchanged.
documents
documents:
allow_remote_sources: false # default falseSettings of the documents routes, /v1/convert/file and /v1/convert/source, which relay to nodes of kind: documents. A request that makes the node fetch from, or write to, an address the client names -- docling's http_sources, callbacks, a target that is not inbody or zip, a sources entry that is not a file -- is a server-side request from inside the cluster network, and is answered 400 before a node is chosen unless allow_remote_sources is true. There is no model name here: the nodes are chosen by their kind, and the route is authorised under the routing key documents (a policy's models). The load fails on a section that is not a hash, an unknown key, and an allow_remote_sources that is no boolean. Removed from the file, remote sources are refused again.
realtime
realtime:
idle_timeout: 300 # seconds a session may be silent, default 300
max_session_seconds: 3600 # longest a session may last, default 3600
max_message_bytes: 1048576 # largest single WebSocket message, default 1 MiB
max_queued_bytes: 4194304 # most bytes held for a slow peer, per direction, default 4 MiBThe limits of the realtime route, GET /v1/realtime (a WebSocket relayed to a node's realtime endpoint, ADR 0029; see "Realtime route" in Langertha::Skeid::Proxy). Every value is an integer of at least 1 and every key is optional. A session ends 1000 at max_session_seconds, 1009 on a message above max_message_bytes, 1013 when a peer that does not read lets more than max_queued_bytes pile up, and a socket silent for idle_timeout seconds is closed. The values are read when a session opens, so a reload holds for the next session and leaves the open ones alone. The load fails on a section that is not a hash, an unknown key and a value that is no integer of at least 1.
A realtime node holds its slot for the whole session, so max_conns of such a node means concurrent sessions; give it a node of its own and see "Realtime route" in Langertha::Skeid::Proxy for what that means under prefork workers (ADR 0010).
openapi
openapi:
swagger_ui: false # default false; true serves the Swagger UI under /docs
cache_dir: /var/cache/skeid # optional: where the UI assets are kept once fetchedGET /openapi.json is always served: an OpenAPI 3.1 description of the client routes this configuration enables (the manifest and the MCP routes appear only when their section is on), behind client_auth like every client route. The admin routes are a separate document, GET /skeid/openapi.json, behind the admin key. swagger_ui adds the UI at /docs. Skeid ships no UI assets: the first request for the UI fetches the pinned swagger-ui-dist release (5.17.14, from cdn.jsdelivr.net, checked against SHA-256 digests compiled into Skeid) and caches it in cache_dir -- default $XDG_CACHE_HOME/skeid/swagger-ui-5.17.14, else ~/.cache/skeid/..., else a per-uid directory under the temp directory -- and serves it from there from then on. With swagger_ui off Skeid makes no network request for it and /docs is an unknown path. See Langertha::Skeid::SwaggerUI and ADR 0025. The load fails on an unknown key, a swagger_ui that is not a boolean and a cache_dir that is not an absolute path.
mcp
names:
alice: k_5f0e1a2b3c4de5f60718293a4b5c6d7e8f901a2b # `skeid keyid <key>` -- never the key
mcp:
public_url: https://llm.example.com # optional: where clients reach Skeid
max_request_bytes: 10485760 # optional: request body limit of /mcp/*
servers:
bookrag: # the name in /mcp/bookrag
url: http://bookrag-mcp.internal:8080/mcp
description: Book search over the house library
api_key_ref: secret/skeid/mcp/bookrag # the server's key, through the key broker
api_key_env: BOOKRAG_MCP_KEY # ... or from this variable
headers: { X-Tenant: example } # static headers that are no secret
keys: [alice, k_9c8b7a6f5e4de5f60718293a4b5c6d7e8f901a2b]
files:
transport: stdio # a hint: runs on the customer's own machine
command: mcp-files
args: ["--root", "/srv/data"]
env: {}
description: Files on your own machine
keys: [alice]The MCP gateway (skeid k103, ADR 0022): MCP servers assigned to customer key ids. A key reaches a server assigned to it under /mcp/<name> with its own Skeid key, and /v1/mcp/servers lists what it was assigned; see "MCP routes" in Langertha::Skeid::Proxy. Without the section both routes are unknown paths, as before.
public_url is where clients reach this Skeid -- the base of the URLs in the listing. Without it the listing uses the scheme and host of the request, which behind a TLS proxy is http:// unless the server runs in reverse-proxy mode; an edge deployment sets it.
A server is transport: streamable-http (the default) or stdio:
streamable-http is relayed.
urlis where Skeid reaches it. Its credential is named like a node's:api_key_refthrough the key broker, elseapi_key_env, sent upstream asAuthorization: Bearer; a server that names one and gets no key from it is not called. A server that names neither is called without a credential -- never with the client's.headersare sent with every request and are for what is no secret:Authorization,Proxy-Authorization,X-Api-KeyandCookieare refused there, as are the headers the relay itself passes on or that belong to the connection.stdio is a hint. Such a server runs on the customer's machine; Skeid neither starts nor reaches it, and it has no
/mcp/<name>route.command,args(a list of strings) andenv(a map of strings) are shown to the assigned key as written, so they are no place for a secret.
keys is the assignment: customer key ids from skeid keyid or names: entries, a single one without the list, resolved at config load as in "client_auth" -- a short (pre-ADR 0016) id matches the full id it is the prefix of. A server without keys is assigned to nobody and says so once. description is free text for the listing.
max_request_bytes (default 10485760, 10 MiB; a positive integer) is the largest request body the MCP routes take, instead of the server's own limit: a larger one is 413, and is counted as rejected (ADR 0024). It bounds the request only; an event stream coming back has no size limit, it ends by the upstream, the client or the time limit.
Two things the gateway does not do, by design (ADR 0022 update): the client's query string goes upstream as it came, and OPTIONS, HEAD and CORS are not answered -- an MCP client is not a browser page, and a browser-based one belongs behind its own edge.
The load fails on an unknown key in the section or in a server, a key of the other transport, a server name that is not letters, digits, ., _ and - (starting with a letter or digit, at most 64), a url that is not absolute http(s) or carries user:password@, a stdio server without command, and a keys entry that is neither a name nor a key id. No message repeats a URL, a header value or a keys entry.
routing
routing:
wait_timeout_ms: 2000 # default 2000: the wait of a model that has no alias
wait_poll_ms: 25 # default 25, at least 1: how often a waiting request retries
trust_key_id_header: false # default false; see key_id_for_key
frontend_count: 1 # default 1, at least 1; see frontend_countThe defaults come from "ENVIRONMENT" when set there.
admin
admin:
api_key_env: SKEID_ADMIN_API_KEY # or api_key: "..."The key for the /skeid/* admin API. It comes from the first of these that has one (skeid k64):
- 1. An explicit key:
skeid serve --admin-api-key,build_app(admin_api_key => ...), anadmin_api_keypassed tonew, or "set_admin_api_key". - 2. The config: the first of
admin_api_key,admin_api_key_env,admin.api_keyandadmin.api_key_envthat is present decides -- set empty, or naming an unset variable, it turns the admin API off. - 3.
SKEID_ADMIN_API_KEY, when the config names none of the four.
With none of them the admin API is off (404). Every applied config re-resolves the key in this order, so a reload never replaces an explicit key, and removing the key from the config falls back to the variable. A variable is read when the config is applied, so a changed value takes effect with the next changed config.
registry
Publishes this Skeid's capacity to a fronting Skeid: enabled, secret_env, read_key_env, ttl_s (default 10), instance_id (default the hostname) and error_window_s (default 60). Documented under "registry_enabled".
manifest
The provider manifest: enabled, public_url, provider_id (default skeid), faces (default all three) and capabilities. Documented under "Provider Manifest".
usage_store
usage_store:
backend: jsonlog # jsonlog | sqlite | postgresql; inferred when absent
path: /var/log/skeid/events/ # jsonlog: log_path or path; mode dir|file (inferred), fsync
# sqlite: sqlite_path (or path, db_path), schema_file, auto_migrate (default on)
# postgresql: dsn, or host (127.0.0.1) / port (5432) / dbname or database (skeid);
# user, password or password_env, schema_file, auto_migrate (default on)
# sqlite and postgresql: flush_interval_ms (default 0) -- above 0, queue events and
# write them in one transaction per interval (write-behind);
# spool_dir (default off) -- a local directory, one per instance, that keeps
# the events the database cannot take while it is down and replays themWhere usage events go; see "normalize_config" in Langertha::Skeid::UsageStore for the inference and the defaults. usage_db_path, a top-level key, is the older spelling of a sqlite store and is read only when usage_store is absent. A changed store is swapped on reload; a removed one stays in force until restart, with a warning -- see "CONFIGURATION".
flush_interval_ms moves a database write off the request path: the request is answered at once and the queued events are written together once per interval. Queued events are held in memory until then; a process that is killed rather than stopped loses them. See "flush_interval_ms" in Langertha::Skeid::UsageStore::DBI and ADR 0005.
A failed connect to the usage database is not repeated per event: the next attempt waits for a backoff deadline (1 second, doubling up to 60). spool_dir keeps the events of an outage: they are appended to a JSON Lines file in that directory -- key ids, never a key or the database password -- instead of being reported lost, and written once the database is back, without a second row for an event already in the table. With it, Skeid also starts while the database is down. See "spool_dir" in Langertha::Skeid::UsageStore::DBI and ADR 0027.
ENVIRONMENT
Defaults for the attributes of the same meaning; a value in the config wins. Besides these, the config names variables of its own -- api_key_env on a node or an MCP server, admin.api_key_env, a usage store's and a customer source's password_env, the registry's secret_env and read_key_env -- so that no secret has to be written into it. "build_app" in Langertha::Skeid::Proxy reads the OPENBAO_* variables, SKEID_UPSTREAM_POOL and SKEID_UPSTREAM_TIMEOUT.
SKEID_ROUTE_WAIT_TIMEOUT_MS
Default of "route_wait_timeout_ms" (2000).
SKEID_ROUTE_WAIT_POLL_MS
Default of "route_wait_poll_ms" (25).
SKEID_TRUST_KEY_ID_HEADER
Default of "trust_key_id_header": 1, true, yes or on turn it on.
SKEID_FRONTEND_COUNT
Default of "frontend_count" (1).
SKEID_ADMIN_API_KEY
The admin API key when neither an explicit key nor the config names one; see "admin".
SKEID_USAGE_DB
Default of "usage_db_path": a SQLite usage store at this path when no usage_store is given, and the path of a sqlite store that names none.
SKEID_CAPACITY_MAX_AGE_MS
Default of "capacity_max_age_ms" (5000).
SKEID_CONFIG_RELOAD_INTERVAL
Default of "config_reload_interval" (1 second).
ATTRIBUTES
Every attribute can be passed to new. Those a config sets are overwritten by the next changed config that sets them (see "CONFIGURATION") -- except "admin_api_key", which new takes as an explicit key that outranks the config.
nodes
The node inventory: an arrayref of node hashrefs, in the shape of the config's nodes entries after "add_node" normalized them (default empty). Change it through "add_node", "remove_node" and "set_node_health", or assign a whole new list: routing caches what it derives from the inventory, and only those paths invalidate the cache. Editing a node hash in place does not.
model_pricing
Served model name to pricing rule, '*' as the fallback (default empty). Written by "set_model_pricing"; read through "pricing_for_model".
model_aliases
Alias name to { tiers => [...] }, normalized (default empty). Written by "set_model_alias".
policies
Policy name to resolved policy (see "resolve_policy"), from the config's policies or "set_policy" (default empty).
default_policy
The resolved policy an unlisted key routes under -- the policy object, not its name -- or undef, which leaves unlisted keys unrestricted (default undef).
key_policies
Customer key id to resolved policy, built at config load (default empty). Keys that take the default policy are not in it; see "policy_for_key".
key_names
Readable name to customer key id, from the config's names section (default empty). See "key_id_for_name".
manifest_enabled
Whether the config enables the provider manifest (default 0).
manifest_available
Whether the installed Langertha has Langertha::Manifest; with it missing an enabled manifest answers 404 instead of failing the config (default 0).
key_manifests
Customer key id to the canonical JSON of that key's manifest, built at config load (default empty). Read through "manifest_for_key".
client_auth_keys
The customer key ids allowed onto the client routes, as a hashref of id to 1, built from the config's client_auth section (default undef: client authentication is off). An empty hash lets no key in. Read through "client_auth_enabled" and "client_key_allowed".
mcp_servers
The MCP servers of the config's mcp section, as a hashref of server name to resolved entry, built at config load (default undef: the MCP gateway is off). An entry holds name, transport, description and keys -- the customer key ids it is assigned to, id to 1 -- and for a relayed server url, api_key_ref, api_key_env and headers, for a stdio one command, args and env. Read through "mcp_enabled", "mcp_server_for_key" and "mcp_servers_for_key".
mcp_public_url
Where clients reach this Skeid, for the URLs of the MCP listing (config mcp.public_url, default undef: the listing takes the URL of the request).
mcp_max_request_bytes
The largest request body, in bytes, the MCP routes accept (default 10485760, 10 MiB; config mcp.max_request_bytes). See "mcp".
openapi_swagger_ui
Whether the config switches on the Swagger UI under /docs (config openapi.swagger_ui, default 0). Off, only /openapi.json is served and the UI paths are unknown. See "openapi".
openapi_cache_dir
Where the Swagger UI assets are cached once fetched (config openapi.cache_dir, default undef: the proxy picks a per-user cache directory).
customer_source_fetcher
A code ref that builds the fetcher of a customer_source section: called with the Skeid and the normalized section, it returns an object with fetch($cb) and optionally on_notify($cb) and release (see Langertha::Skeid::CustomerSource). Without it the fetcher is a Langertha::Skeid::CustomerSource::DBI, and a config with the section needs DBD::Pg. Tests give a fake here; see "customer_source".
upload_max_bytes
The largest request body, in bytes, the upload routes accept (default 26214400, 25 MiB; config uploads.max_bytes). See "uploads".
documents_max_bytes
The largest request body, in bytes, the documents routes accept (default 104857600, 100 MiB; config uploads.documents_max_bytes). It bounds the body: for /v1/convert/source, whose files travel as base64 in JSON, that is a third more than the file. See "uploads".
documents_allow_remote_sources
Whether a documents request may name an address for the node to fetch from or call (docling's http_sources, callbacks, a target to put to). Default false; config documents.allow_remote_sources. See "documents".
realtime_idle_timeout
Seconds a realtime session may be silent before it is closed (default 300; config realtime.idle_timeout). See "realtime".
realtime_max_session_seconds
The longest a realtime session may last, in seconds (default 3600; config realtime.max_session_seconds). See "realtime".
realtime_max_message_bytes
The largest single WebSocket message a realtime session takes, in either direction (default 1048576; config realtime.max_message_bytes). See "realtime".
realtime_max_queued_bytes
The most bytes of a realtime session that may wait for a peer that is not reading, per direction (default 4194304; config realtime.max_queued_bytes). See "realtime".
trust_key_id_header
Whether a client's x-skeid-key-id (or x-api-key-id) header names the customer key id instead of the key it presented (default off, or SKEID_TRUST_KEY_ID_HEADER; config routing.trust_key_id_header). Turn it on only behind a gateway that authenticates the caller and sets that header itself: the key id selects both the routing policy and the bill.
route_wait_timeout_ms
How long, in milliseconds, a request for a model without an alias waits for a free node before it is answered 429 (default 2000, or SKEID_ROUTE_WAIT_TIMEOUT_MS; config routing.wait_timeout_ms). An alias tier waits its own wait_ms instead.
route_wait_poll_ms
How often, in milliseconds, a waiting request tries again (default 25, or SKEID_ROUTE_WAIT_POLL_MS; config routing.wait_poll_ms, at least 1). The wait is a Mojo::IOLoop timer, never a sleep.
usage_db_path
A SQLite usage database path (default SKEID_USAGE_DB, else unset). At construction it configures a SQLite store when usage_store is not given; it is also the path of a sqlite store config that names none. Kept in step with the store: set to a SQLite store's path, cleared for PostgreSQL. has_usage_db_path and clear_usage_db_path are its predicate and clearer.
usage_store
The usage store config (default empty: no store). Passed to new it is a raw usage_store config; afterwards it holds the normalized form (see "normalize_config" in Langertha::Skeid::UsageStore). Change it through "configure_usage_store".
store_usage_event
store_usage_event => sub { my ($skeid, $event) = @_; ...; return { ok => 1 } },Optional code ref that receives every usage event instead of the configured store (has_store_usage_event is its predicate). See "Pluggable Usage Storage".
query_usage_report
query_usage_report => sub { my ($skeid, $filters) = @_; ...; return { ok => 1, ... } },Optional code ref that answers "usage_report" instead of the configured store (has_query_usage_report is its predicate). It gets since, api_key_id, model and limit.
on_usage_lost
$skeid->on_usage_lost(sub { my ($skeid, $event, $error) = @_; ... });Optional code ref (has_on_usage_lost is its predicate), called for every usage event a write-behind store (usage_store.flush_interval_ms) queued and then could not write, or a store's spool (usage_store.spool_dir) could neither keep nor replay. The request was answered before the write was tried, so this is the only place that failure can be said. "build_app" in Langertha::Skeid::Proxy sets it to the proxy's usage event lost log line; without it Skeid warns. A failure of a synchronous write is the caller's to report, from the answer of "record_usage", and never comes here.
admin_api_key
The bearer token of the /skeid/* admin API in force; empty disables it, which the proxy answers with 404. Passed to new it is an explicit key (see "set_admin_api_key") that wins over the config; otherwise it is resolved from the config and SKEID_ADMIN_API_KEY whenever a config is applied -- see "admin" for the order. Read it here; set it through "set_admin_api_key", since a value written through this accessor lasts only until the next changed config.
key_broker
Optional Langertha::Skeid::KeyBroker that resolves a node's api_key_ref per request (has_key_broker is its predicate). "build_app" in Langertha::Skeid::Proxy passes a Langertha::Skeid::KeyBroker::OpenBao when the OPENBAO_* variables are set.
config_file
Path of a YAML config (see "CONFIGURATION"; has_config_file is its predicate). Read at construction and again whenever its mtime moves. Give either this or "config_loader", not both. A path that is not an existing file dies at construction (config file not found: ...); a file that disappears later keeps the config in force (see "maybe_reload_config").
config_loader
A code ref that returns the config as a hashref, instead of a config_file. It is called with the Skeid object, at construction and then from call_function at most once per "config_reload_interval".
It may return ($config, $version). A defined version is the change detector: the same version as last applied means nothing changed and the config is not applied again. Without a version Skeid digests the returned structure (hash keys sorted) instead. Either way an unchanged config is a no-op, and a changed one whose nodes section is unchanged keeps the node list -- its inventory generation, its running capacity probes and any health set through the admin API. A code ref inside the config (a custom probe's code) digests by identity, so a loader that builds a fresh closure every call should return a version.
A config file follows the same rules, minus the throttle for a new file version: an mtime change whose content is unchanged applies nothing. A version that fails to parse is retried with the same bounded back-off as a failing loader, while a different mtime is read immediately. An explicit config.reload does too, so a value taken from the environment (admin.api_key_env, a usage store's password_env) is read again only when the config itself changes.
config_reload_interval
The least time, in seconds, between two runs of a config_loader (default 1; fractions allowed; 0 runs it on every dispatch, as before skeid #38).
A loader-based config is re-read from call_function, which every request passes through several times. Without a throttle every request would rerun the loader. A config file normally pays only one stat per dispatch; only retries of one failed file version are throttled.
last_reload_error
Why the last config reload failed, or undef when the last one succeeded. A failed reload keeps the previous config in force; see "maybe_reload_config". last_reload_error_at is when (epoch seconds), reload_failures how many reloads in a row have failed.
capacity_max_age_ms
How long a capacity reading is trusted, in milliseconds (default 5000, 0 disables expiry).
A stale reading is worse than none: it describes a node as it was, and admission acts on it as if it were now. Past this age a reading is ignored and inflight decides again, which is the same behaviour as having configured no probe at all (ADR 0009).
worker_count
How many worker processes share this configuration (default 1).
inflight and max_conns are per-process, so N workers would each admit up to max_conns to a node that can only serve one number — max_conns: 8 across 4 workers would permit 32, silently. Setting this makes each worker take its share instead (ADR 0010). Anything on a timer is spread the same way, so the process group's aggregate poll rate stays what was configured.
frontend_count
How many separate Skeid frontends stand in front of one node (default 1).
worker_count partitions max_conns across the prefork workers of one process; this partitions it across the distinct Skeid hosts sharing a node — a number only the operator knows, because two frontends are two processes on two machines with nothing between them to count each other's inflight (there is no probe and no shared state; that absence is the whole reason this has to be declared rather than detected). With F frontends each admits its share max_conns/F, so the group as a whole never admits more than the node was configured for (ADR 0009, ADR 0012).
It composes with worker_count: the two divisors multiply, so one worker's share is max_conns/(F*N) — partitioned across frontends first, then across workers, which integer division makes the same thing either way. Unlike the worker divisor it does not scale anything on a timer: separate frontends each run their own probes and hold their own vault token, the same reason vault renewal is not scaled per worker (ADR 0010).
Explicit and opt-in: the default 1 is today's behaviour, unchanged. An operator who runs two frontends and forgets to say so over-admits the node by a factor of two, silently — which is exactly the failure this field exists to prevent.
registry_enabled
Whether this Skeid publishes its registry snapshot on GET /skeid/registry/snapshot (default off; ADR 0017). Set by the config's registry block:
registry:
enabled: true
secret_env: SKEID_REGISTRY_SECRET # required when enabled; the HMAC key
ttl_s: 10 # how long a snapshot may be believed
instance_id: skeid-b # default: the hostname
error_window_s: 60 # how far back errors_in_window counts
read_key_env: SKEID_REGISTRY_READ_KEY # optional; a bearer for the snapshot route onlyThe fronting tier reads the snapshot with Langertha::Skeid::CapacityProbe::Registry. The secret is taken from the environment only and kept in memory (ADR 0003); an enabled registry whose variable is empty or shorter than 32 bytes does not load, because a snapshot is never published unsigned or weakly signed. Neither does one without a credential to read it: an admin API key or a registry read key.
read_key_env names the variable holding the registry read key (skeid #49). The snapshot route accepts it as a bearer token besides the admin API key; it opens no other route, so a fronting tier holding it cannot add nodes or flip health. A named variable that is empty does not load. registry_secret, registry_read_key, registry_ttl_s, registry_instance_id and registry_error_window_s hold the other fields.
METHODS
add_node
$skeid->add_node(id => 'gpu-1', url => 'http://gpu-1:8000/v1', model => 'qwen3-32b',
tags => ['local'], max_conns => 8);Adds a node, replacing any node with the same id. Takes the fields of a config nodes entry and fills in their defaults (see "nodes"). Returns 1. Croaks without id or url, on an unknown engine, rerank_format or kind, on an api_key_header that is no header name, and on a registry capacity block that "validate_config" in Langertha::Skeid::CapacityProbe::Registry rejects. The nodes.add function and POST /skeid/nodes call it.
remove_node
my $removed = $skeid->remove_node('gpu-1');Removes a node, and with it its capacity reading and failure history, so a node later added under the same id starts clean. Returns 1 when a node was removed, else 0.
normalize_tags
my $tags = Langertha::Skeid->normalize_tags(['Local', 'gb10']);
my $tags = Langertha::Skeid->normalize_tags('local, gb10');Tags are lowercased, trimmed, de-duplicated and kept in the order first seen. A plain string is accepted and split on commas or whitespace, because a hand-written config says tags: local, gb10 at least as often as it says a YAML list.
list_nodes
my $nodes = $skeid->list_nodes;The inventory as an arrayref of shallow copies of the node hashes.
list_models
my $models = $skeid->list_models(api_key_id => $key_id);
# [ { model => 'house-model', engine => '' }, { model => 'qwen3-32b', engine => 'vllm' } ]The names a client can put in model, sorted by name, each once: the distinct model of the nodes plus the alias names. A node without a model lists nothing -- it matches any name, so none reaches it in particular. A name that is both a node model and an alias is one entry. engine is the first node's engine for a node model, empty for an alias.
The list is narrowed by the policy of api_key_id (none means the default policy), so a key is never shown a name "route_plan" would refuse it: a name its models do not grant is left out, an alias with every tier denied is left out, and a node model that only nodes carrying a denied tag serve is left out. This backs /v1/models and /api/tags.
set_node_health
$skeid->set_node_health('gpu-1', 0); # take it out of rotationSets the operator health flag. An unhealthy node is not eligible for routing. Health is operator state: no error, timeout or 429 ever sets it. Returns 1 when the node exists, else 0. Setting the value it already has changes nothing, round-robin position included.
set_model_pricing
$skeid->set_model_pricing('gpt-4o-mini', {
input_per_million => 0.15,
output_per_million => 0.60,
cached_input_per_million => 0.075, # optional
cache_write_per_million => 0.1875, # optional
});Sets the pricing rule for one served model ('*' is the fallback) and returns it as stored. Absent input and output rates are 0. Croaks without a model or a pricing hash, and on a cache rate that is not a number >= 0. On a Langertha that cannot price prompt-cache tokens the cache rates are dropped, with one warning per process.
pricing_for_model
my $rule = $skeid->pricing_for_model('gpt-4o-mini');The rule for a model, else the '*' rule, else a rule that prices everything at 0.
reload_config
my $config = $skeid->reload_config;Reads the config source and applies it, returning the config hashref. A config whose fingerprint matches the one last applied changes nothing. Dies when the source cannot be read or the config does not apply; the previous config then stays in force and "reload_status" reports the failure. The config.reload function calls it; the request path uses "maybe_reload_config", which never dies.
reload_status
my $status = $skeid->reload_status;
# { ok => 0, error => '...', failed_at => '2026-09-25T16:08:30Z', failures => 3 }Whether the last config reload succeeded, and if not, why, when, and how many times in a row. Served by the admin route GET /skeid/config; the public /health carries only ok, failed_at and failures, never the message, which can name customers and models.
set_admin_api_key
$skeid->set_admin_api_key($key); # explicit: wins over the config, on every reload
$skeid->set_admin_api_key(''); # back to the config's key, or SKEID_ADMIN_API_KEYSets the explicit admin API key -- what skeid serve --admin-api-key and build_app(admin_api_key => ...) set, and what an admin_api_key passed to new is. It outranks the config: a reload re-applies the order of "admin" under it instead of replacing it. An empty or undefined key clears it, and the key the config resolved to takes over at once. Returns the admin API key now in force.
maybe_reload_config
my $applied = $skeid->maybe_reload_config;Reloads the config if its source may have changed: the file's mtime moved, or a config_loader is due ("config_reload_interval"). Returns true when a changed config was applied. call_function runs it on every dispatch, so it never dies: a reload that fails is logged, recorded in "reload_status", and the request goes on under the config that was in force before (the reload is all or nothing). A failing source is then retried with a back-off -- the interval doubling per failure, from at least a second up to a minute. A loader that keeps returning the same broken config is not applied again; a changed config-file mtime bypasses the failed version's retry window. A config file that has disappeared is not a reload at all: the config in force stays, and the loss is warned once until the file is back. Only construction and an explicit config.reload still die on a bad or missing config.
start_customer_source
$skeid->start_customer_source;Starts fetching the config's customer_source on Mojo::IOLoop: the first fetch on the next loop turn, then one per refresh_interval_s. A source added, changed or removed by a later reload is started, restarted or stopped with it. "build_app" in Langertha::Skeid::Proxy calls this; a Skeid that is not serving (skeid usage) never fetches. Returns true when a source is configured.
stop_customer_source
Stops fetching. The snapshot in force stays.
customer_source_startup_fetch
my $ok = $skeid->customer_source_startup_fetch;Runs one fetch now and waits for it, at most connect_timeout_s, by running the event loop -- so only before the loop is started: skeid serve calls it before it listens. Returns true when the fetch succeeded and its rows are in force. When it fails, Skeid starts degraded and fail-closed: the file's config is in force and no source rows are, and the fetch is retried with the back-off. Returns false without a started source.
customer_source_release
Lets the fetcher close its database connections; the next fetch opens new ones. skeid serve calls it before the prefork manager forks, so each worker opens its own. It also queues a fetch for the next turn of the loop: the manager never runs the loop, each worker inherits the queue and fetches as it starts -- which opens its connections, the listen one included, at once, not one refresh_interval_s after the start.
customer_source_status
my $status = $skeid->customer_source_status; # undef without a customer_source
# { ok => 1, relation => 'skeid.customer_keys', last_success_at => '...', failed_at => undef,
# failures => 0, error => undef, stale => 0,
# rows => { accepted => 120, rejected => 1, shadowed => 2 },
# rejected => [ { row => "'acme'", reason => '...' } ], shadowed => [ "'bigcorp'" ] }The state of the customer source, for GET /skeid/config. ok is true once a fetch succeeded and the last one did; error says why not. stale is true while max_stale_s has dropped the rows. rows counts what the last merge accepted (shadowed ones included), rejected and shadowed; rejected and shadowed list the first 50 by label -- the row's name, else its position -- never by value. /health carries only ok, failed_at and failures.
customer_names
my $names = $skeid->customer_names; # { 'k_5f0e...' => 'acme', ... }The display names of the customer source's accepted rows, by customer key id: whom a key id belongs to, for GET /skeid/config. Rows without a name are absent. Key ids and the labels the owning application chose, never a key. Empty without a customer source.
customer_report
my $report = $skeid->customer_report; # undef without a customer_source
# { accepted => 120, rejected => [ { row => "'acme'", reason => '...' } ], shadowed => [ "'bigcorp'" ] }What the last merge did with the source's rows: how many it accepted, and which it rejected or shadowed by label (first 50 of each) with the reason -- never a value. GET /skeid/config carries it as customer_report.
manifest_for_key
my $json = $skeid->manifest_for_key($api_key_id); # UTF-8 JSON bytes, or undefThe provider manifest published to a customer key id, as canonical JSON, or undef when the key has no manifest: grant, the manifest is disabled, or this Langertha has no Langertha::Manifest. Built at config load; see "Provider Manifest".
configure_usage_store
$skeid->configure_usage_store({ backend => 'jsonlog', path => '/var/log/skeid/events/' });Normalizes a usage_store config and, when it differs from the current one, prepares the new store before disconnecting the old. Returns the normalized config. Croaks on a config "normalize_config" in Langertha::Skeid::UsageStore rejects and on a store that fails to prepare, leaving the old store in place. The usage.configure function calls it.
supported_engine_ids
my $ids = $skeid->supported_engine_ids; # [ 'aki', 'akiopenai', 'anthropic', ... ]The engine ids a node may name, sorted: a built-in list, plus whatever Langertha->available_engine_ids reports when the installed Langertha has it.
normalize_engine_id
my $id = $skeid->normalize_engine_id('Langertha::Engine::vLLM'); # 'vllm'Lowercases an engine name and strips a Langertha::Engine:: or LangerthaX::Engine:: prefix. Returns the empty string for an empty value and croaks on an id that is not in "supported_engine_ids".
record_usage
my $res = $skeid->record_usage(
api_key_id => $key_id, model => 'qwen3-32b', requested_model => 'house-model',
node_id => 'gpu-1', status_code => 200, ok => 1, duration_ms => 840,
metrics => $normalized, # from normalize_metrics
);Builds one usage event and hands it to the sink: the "store_usage_event" callback, a subclass's _store_usage_event, or the configured store. Returns the sink's answer, or { ok => 0, enabled => 0, error => 'usage_store not configured' } without building an event when there is no sink.
The event carries created_at, request_id, api_format, endpoint, api_key_id, provider, engine, model (served), requested_model (default model), node_id, route_url, status_code, ok, duration_ms, error_type, error_message, and from metrics the token counts (input_tokens, output_tokens, total_tokens, cached_tokens, cache_write_tokens), tool_calls and the costs (cost_input_usd, cost_output_usd, cost_total_usd, cost_cache_read_usd, cost_cache_write_usd). Costs arrive already priced; nothing is priced here. content_bytes, audio_seconds, documents and pages are set only when given. The usage.record function calls it.
flush_usage
my $res = $skeid->flush_usage;Writes the usage events a write-behind store still holds (see "flush" in Langertha::Skeid::UsageStore::DBI) and returns its answer, or { ok => 1, written => 0 } when the store holds nothing back. bin/skeid calls it when the server stops, and an application embedding the proxy should call it before it exits: queued events live in memory only. Replacing the store on a reload and destroying this object flush as well.
usage_report
my $report = $skeid->usage_report(since => '2026-09-01T00:00:00Z', api_key_id => $id,
model => 'qwen3-32b', limit => 50);Asks the "query_usage_report" callback, a subclass's _query_usage_report or the configured store for a report (its shape: Langertha::Skeid::UsageStore). Empty filters are dropped; limit defaults to 20 and is capped at 500. Without a store it returns { ok => 0, enabled => 0, error => 'usage_store not configured' }. The usage.report function, GET /skeid/usage and skeid usage call it.
estimate_cost
my $cost = $skeid->estimate_cost(model => 'gpt-4o-mini',
usage => { prompt_tokens => 1000, completion_tokens => 200 });Prices a usage with the model's rule ("pricing_for_model") or an explicit pricing rule, and returns the hash form of the Langertha::Cost. The usage is a hash, a Langertha::Usage, or read from a response. The metrics.estimate_cost function calls it.
normalize_metrics
my $metrics = $skeid->normalize_metrics(model => 'gpt-4o-mini', response => $upstream_body,
tool_calls => \@calls, duration_ms => 840, engine => 'openaibase', route => '/v1/chat/completions');What a usage event is built from: the usage (as for "estimate_cost") priced into a Langertha::UsageRecord, returned as its hash, with tool_calls counted by name. Adds cached_tokens and cache_write_tokens when the installed Langertha::Usage reads them. Optional provider, started_at, finished_at and pricing_version are carried into the record. The metrics.normalize function calls it; the proxy runs it on every upstream answer, streamed or not.
worker_max_conns
my $share = $skeid->worker_max_conns($node);This process's share of a node's max_conns (ADR 0010, ADR 0012). With one worker and one frontend that is the configured value; otherwise it is the configured value divided by the number of processes sharing the node — frontend_count separate Skeid hosts times the worker_count prefork workers of this one — so the group as a whole never admits more than was asked for.
Never less than 1 when a limit is set: a process that may admit nothing is a process that does nothing. That means a max_conns below that combined process count cannot be honoured, and "worker_share_warnings" is what says so out loud.
worker_share_warnings
warn $_ for @{ $skeid->worker_share_warnings };The nodes whose max_conns cannot be divided among the processes sharing them without exceeding it — frontend_count frontends times worker_count workers. Returned rather than warned so the caller decides where they go; bin/skeid prints them at startup.
Silence here would be the bad kind: the operator wrote a number, and the process group is about to ignore it.
set_model_alias
$skeid->set_model_alias('our-fast-model', {
tiers => [
{ tags => ['local'], model => 'qwen3-32b', wait_ms => 200 },
{ tags => ['cloud'], model => 'llama-3.3-70b-versatile' },
],
});Defines a client-facing model name as an ordered list of tiers. A bare arrayref of tiers is accepted as shorthand for { tiers => [...] }.
Per tier: tags selects nodes, model is the model actually asked of them (defaulting to the alias name itself, for the case where nodes carry that name), engine optionally constrains the engine, and wait_ms is how long to wait for capacity in this tier before falling through to the next.
wait_ms defaults to 0. Writing tiers means "try here, then there"; waiting is the exception you opt into, and a tier that waits by default would send traffic to a paid cloud only after a delay nobody asked for -- or, worse, make a cheap tier look slow.
set_policy
$skeid->set_policy('standard-local-only', { deny_tags => ['cloud'] });Defines a named policy profile. Profiles are the "standard setups" most customers take unchanged; a customer needing something precise gets the profile plus overrides, or a profile of their own.
resolve_policy
my $policy = $skeid->resolve_policy({ models => ['house-model'], deny_tags => ['cloud'] });Turns a policy spec into the immutable form routing uses: models (or aliases) becomes a lookup hash of the requested model names the key may ask for, absent meaning all of them, and deny_tags becomes a normalized tag list.
key_id_for_key
my $id = Langertha::Skeid->key_id_for_key('sk-alice-secret'); # k_5f0e...The customer key id derived from the presented API key. This is the name a keys: entry has to use, and skeid keyid prints it, because the config must be able to name a customer without holding that customer's key.
It is the full SHA-1 hex digest of the key (160 bits), not a secret: it identifies, it does not authenticate. What authenticates is that the caller presented the key it was derived from.
Before ADR 0016 the id was the first 12 hex digits of the same digest (k_5f0e1a2b3c4d), so an old id is the prefix of the new one. A config may still name a customer by its short id: it matches the key whose full id starts with it, with a one-time deprecation warning at config load, and a config that lists both a short id and a full id it is the prefix of fails to load. Usage events keep the id they were recorded under -- events from before the change carry the short id, later ones the full id; nothing is migrated.
policy_for_key
my $policy = $skeid->policy_for_key($key_id); # k_5f0e...The policy a customer key routes under. Unlisted keys take the default policy, which is what makes a deployment with ten thousand identically-configured customers a config with zero key entries. Returns undef when no policies are configured at all.
key_id_for_name
my $id = $skeid->key_id_for_name('alice'); # k_5f0e1a2b3c4de5f60718293a4b5c6d7e8f901a2b, or undefThe customer key id a readable name maps to under the config names: section, or undef when the name is not registered. The registry is a config-authoring convenience -- it lets a keys: entry be written by name and confines key rotation to one line -- and is resolved to ids at config load, so it never appears on the request path.
client_auth_enabled
if ($skeid->client_auth_enabled) { ... }Whether the config has a client_auth section, so that the client routes answer 401 to a key not on its list. See "client_auth".
client_key_allowed
my $ok = $skeid->client_key_allowed($key_id); # k_5f0e...Whether a customer key id may use the client routes: always true while client authentication is off, otherwise true only for an id on the client_auth list -- one hash lookup. A short (pre-ADR 0016) id on the list matches the full id it is the prefix of, as in keys:. anonymous -- no key presented -- is never on the list.
mcp_enabled
if ($skeid->mcp_enabled) { ... }Whether the config has an mcp section, so that the MCP routes exist. See "mcp".
mcp_server_for_key
my $server = $skeid->mcp_server_for_key('bookrag', $key_id); # the resolved entry, or undefThe MCP server of that name, if it is assigned to this customer key id -- two hash lookups. Undef for an unknown name and for a server assigned to other keys alike: the caller must not be able to tell the two apart. A short (pre-ADR 0016) id in a server's keys matches the full id it is the prefix of, as in client_auth; anonymous is assigned nothing.
mcp_servers_for_key
my $servers = $skeid->mcp_servers_for_key($key_id); # [ $server, ... ], sorted by nameEvery MCP server assigned to this customer key id, relayed and stdio ones alike. Empty for a key with no assignment and while the gateway is off.
route_plan
my $plan = $skeid->route_plan(model => 'our-fast-model', api_key_id => $key_id);
# { tiers => [...], permitted => 1, reason => '' }The ordered tiers to try for a requested model, under the policy of the key that asked. A model with no alias yields a single implicit tier that selects on the name itself and inherits the global route_wait_timeout_ms, which is what makes an aliasless config behave exactly as it did before aliases existed.
kind (default chat, else documents) is the kind of node the route is over: it is put on every tier, so the selection and its round-robin cursor are over nodes of that kind only. The documents route plans under the model documents (ADR 0028).
permitted is false when the policy does not grant this model, or when every tier of it was denied. Both mean the same thing to a caller — this key may not reach this model — and neither is a capacity problem, so they must not be reported as one.
Tiers carry the policy's deny_tags down into node selection. Dropping a denied tier is only the reporting half; without the node-level filter, a key denied cloud could still reach a cloud node by asking for its raw model name instead of the alias.
select_nodes
my $local = $skeid->select_nodes(tags => ['local']);Nodes carrying every listed tag, as copies. No tags selects everything. Selection is by tag, never by node id, so a config can talk about local or cloud without naming machines. deny_tags drops every node carrying any of those tags. kind ('' for chat, or documents), when given, keeps the nodes of that kind only; without it every kind is selected. Health is not considered. The nodes.select function calls it.
pick_node
my $node = $skeid->pick_node(model => 'qwen3-32b', tags => ['local'], deny_tags => ['cloud']);The next node for a selection by weighted round-robin (over the eligible nodes, ordered by id), skipping every node admission would refuse right now. Returns a copy of the node with inflight and route_key added, or nothing. Picking does not admit: the caller still has to "start_request", which can refuse when another request took the slot in between. The route.next function calls it.
route_state
my $state = $skeid->route_state(model => 'qwen3-32b', tags => ['local']);
# { model, engine, tags, deny_tags, eligible_count, available_count,
# has_eligible, has_available }How many nodes a selection matches (eligible) and how many of those admission would take now (available). No eligible node is a 503; eligible but none available is worth waiting for. The route.state function calls it.
set_capacity_reading
$skeid->set_capacity_reading('gpu-1', used => 6, limit => 8, source => 'prometheus');
$skeid->set_capacity_reading('groq-1', retry_after_ms => 2000, source => 'ratelimit');Records what a probe found. One shape for every probe, so admission never learns where a number came from (ADR 0009):
used/limit— occupancy.limit0 or absent means the probe measured something it cannot turn into a ceiling, so it does not constrain admission.retry_after_ms— do not send anything here until it elapses. What a429means.source— which probe, for reports. Never consulted by admission.quota-- which of a provider's quotas (requests,tokens) the reading is about -- likewise.at— when the reading was taken (default now), andexpires_at— an absolute moment after which it is dropped even inside "capacity_max_age_ms". The registry probe uses both: a snapshot is as old as itsgenerated_at, and never outlives its ownttl.interval_ms— how often this source reports, for a probe on a timer. Stored with the reading; it sets how long a tighter reading can hold a looser one from another source off (below). A passive observation (a response's rate-limit headers) passes none.
A reading only ever narrows what max_conns already allows, and expires after "capacity_max_age_ms". Probing is a background activity: calling this from a request handler is a bug unless the reading was a by-product of a response already in hand.
The tighter reading wins across sources, while it is current (ADR 0017). A source always replaces its own last reading. A reading from a different source is held off by the current one only when the current one is tighter (a pending backoff is tightest, then used/limit, then a reading without a limit) and either
it carries a pending backoff, or
it is younger than the longer of the two sources'
interval_ms-- neither has had a full poll since the tighter reading was taken.
Otherwise the incoming reading replaces it. So a registry snapshot saying "empty" cannot lift a 429 backoff a response just recorded; a fresh, tight probe reading is not lifted by a roomy response or by a faster, looser probe before its own next poll; and a passive reading that is never refreshed (remaining: 0 with no reset, from the last response before traffic stopped) cannot keep a fresh probe out for longer than one of that probe's polls. Returns the reading in force afterwards.
capacity_reading
my $reading = $skeid->capacity_reading('gpu-1'); # or nothingThe node's current capacity reading, or nothing when no probe has reported or the last report has aged out. A backoff outlives the age limit: a provider that said "not for another 30 seconds" told us something that is still true.
forget_capacity
$skeid->forget_capacity('gpu-1'); # or all of them with no argument
$skeid->forget_capacity('gpu-1', source => 'registry'); # only if that source holds itDrops probe readings, so inflight decides again. What a node removal calls, and what a probe calls when it can no longer reach its source — reporting nothing beats reporting last hour.
With source, the reading is dropped only when that source wrote it. A probe forgetting what it said must not wipe a backoff another source recorded (ADR 0017).
capacity_header_names
The response headers worth looking at, so a caller on the request path can pull those few by name instead of walking every header of every response.
observe_response_headers
$skeid->observe_response_headers('groq-1', \%headers, status => 429);The zero-cost probe: commercial providers do not publish queue depth, but they do put their rate-limit state on every response Skeid already receives. Reading it costs no extra request (ADR 0009).
Providers meter requests and tokens separately, and for an LLM API the token budget is usually what runs out first. Both are read, and the one closest to exhausted decides — compared as a fraction, since the two are not the same unit.
Retry-After on a 429 becomes a backoff. It deliberately does not touch healthy: rate-limited is busy, not broken, and nothing would ever flip that back.
Returns the reading it recorded, or nothing when the response said nothing useful.
start_request
$skeid->start_request('gpu-1') or ...; # refusedAdmits one request to a node and counts it in inflight. Returns 1 when admitted, 0 when the node is unknown, unhealthy or full. Every 1 must be paired with a "finish_request" on every path -- errors, timeouts, client aborts -- or the node loses that slot until restart. The request.start function calls it.
finish_request
$skeid->finish_request('gpu-1', ok => 1, duration_ms => 840);
$skeid->finish_request('gpu-1', aborted => 1); # the client hung up
$skeid->finish_request('gpu-1', rejected => 1); # the request was the client's to fixReleases the slot "start_request" took and counts the outcome for "node_metrics". There are four outcomes: ok, failed (none of the others), aborted -- the client went away -- and rejected -- the client sent something no node would take (400, 404, 413, 422; not 401, 403 or 429, which speak of the node's key or capacity). Both say nothing about the node. A failed request counts toward the registry snapshot's errors_in_window and last_failure_at; an aborted one is counted apart and toward neither; a rejected one is counted apart and shows as rejected_in_window. aborted wins over rejected, which wins over ok. Never touches healthy. Returns 1. The request.finish function calls it.
registry_snapshot
my $snapshot = $skeid->registry_snapshot;What this Skeid publishes to a fronting tier (ADR 0017): schema version 1, instance, generated_at, ttl, workers, and per node id, tags, healthy, inflight, max_conns (this process's share), errors_in_window, last_failure_at, rejected_in_window (requests the client's own fault ended, counted apart; ADR 0024) and, when a current reading exists, capacity (used, limit, source, retry_after).
Built from a whitelist, so it cannot carry what it must not: no node URL, no key reference, no metadata, no customer key id, no policy, no usage. Langertha::Skeid::Registry signs it.
node_metrics
my $one = $skeid->node_metrics('gpu-1');
my $all = $skeid->node_metrics;Volatile per-process counters for operations, never billed: node_id, inflight, started, ok, error, aborted, rejected, duration_ms_total, and capacity while a current reading exists. With no id, an arrayref with one entry per node. Served by GET /skeid/metrics/nodes.
call_function
my $result = $skeid->call_function('route.plan', { model => 'house-model', api_key_id => $id });The command surface the proxy drives Skeid through. Every call first runs "maybe_reload_config". Croaks on an unknown name, on args that are not a hashref, and where a function names a required argument.
function args returns
metrics.estimate_cost as estimate_cost estimate_cost
metrics.normalize as normalize_metrics normalize_metrics
pricing.set model, pricing the stored rule
nodes.add a node { ok }
nodes.remove id { ok }
nodes.list { nodes }
nodes.select tags, deny_tags { nodes } (select_nodes)
nodes.set_health id, healthy { ok }
nodes.metrics id (optional) { metrics }
alias.set name, alias (or the tiers inline) { ok }
policy.set name, policy (or the spec inline) { ok }
policy.for_key api_key_id { policy }
route.plan model, engine, api_key_id, kind route_plan
route.next model, engine, tags, deny_tags, kind { node } (pick_node)
route.state model, engine, tags, deny_tags, kind route_state
capacity.set id, and the reading { capacity }
capacity.get id { capacity }
capacity.observe id, headers, status { capacity }
capacity.forget id (optional) { ok }
request.start id { ok }
request.finish id, ok|aborted|rejected, duration_ms { ok }
config.reload { config } (dies on failure)
config.status reload_status
usage.record as record_usage record_usage
usage.report since, api_key_id, model, limit usage_report
usage.configure usage_store (or the config inline) { usage_store }
engines.list { engines }Add a function here rather than reaching into the object from the app.
SEE ALSO
Langertha::Skeid::Proxy -- the HTTP application on top of this control plane, and its routes
skeid --
serve,usageandkeyidLangertha::Skeid::CapacityProbe -- node capacity probes
Langertha::Skeid::UsageStore -- where usage events go
Langertha::Skeid::KeyBroker -- resolving a node's
api_key_refLangertha::Skeid::Registry -- the signed Skeid-to-Skeid snapshot
Langertha -- engines, usage, pricing
SUPPORT
Issues
Please report bugs and feature requests on GitHub at https://github.com/Getty/langertha-skeid/issues.
IRC
Join #langertha on irc.perl.org or message Getty directly.
CONTRIBUTING
Contributions are welcome! Please fork the repository and submit a pull request.
AUTHOR
Torsten Raudssus <torsten@raudssus.de> https://raudssus.de/
COPYRIGHT AND LICENSE
This software is copyright (c) 2026 by Torsten Raudssus.
This is free software; you can redistribute it and/or modify it under the same terms as the Perl 5 programming language system itself.