NAME

Langertha::Skeid::Proxy - Multi-format LLM proxy (OpenAI, Anthropic, Ollama) powered by Langertha::Skeid routing

VERSION

version 0.004

SYNOPSIS

use Langertha::Skeid::Proxy;
use Mojo::Server::Daemon;

my $app = Langertha::Skeid::Proxy->build_app(config_file => '/etc/skeid/skeid.yaml');
Mojo::Server::Daemon->new(app => $app, listen => ['http://127.0.0.1:8090'])->run;

# or simply
skeid serve --config /etc/skeid/skeid.yaml --listen 127.0.0.1:8090

DESCRIPTION

The Mojolicious application in front of Langertha::Skeid. It speaks three client formats -- OpenAI, Anthropic and Ollama -- and makes one kind of upstream call, an OpenAI-shaped POST to the node routing picked (ADR 0001); the translation lives in Langertha::Skeid::Protocol and its per-format modules. Everything else -- nodes, routing, admission, pricing, usage -- is the control plane's, driven through "call_function" in Langertha::Skeid, and is configured there (see "CONFIGURATION" in Langertha::Skeid). skeid serve runs it.

The request path is asynchronous throughout (ADR 0005): waiting for capacity is a timer, key resolution goes through "key_async" in Langertha::Skeid::KeyBroker, and the upstream call never blocks the loop.

Client routes

No Skeid credential is needed on these unless the config has a client_auth section; then every one but /health answers 401 to a key not on its list. See "Customer identity". The MCP routes exist only with an mcp section and always need a key; see "MCP routes".

GET  /health                     {status: ok, proxy: skeid, config_reload: {...},
                                  customer_source: {...} with a customer_source}
GET  /.well-known/langertha.json provider manifest for the presented key
GET  /v1/models                  OpenAI: node models and alias names, once each
POST /v1/chat/completions        OpenAI chat; a stream is relayed byte for byte
POST /v1/embeddings              OpenAI embeddings
POST /v1/audio/transcriptions    OpenAI audio: multipart upload, relayed as the node answers
POST /v1/audio/translations      the same, to the node's translations endpoint
POST /v1/rerank                  rerank: relayed as the node answers, translated for a TEI node
GET  /v1/realtime                realtime transcription: a WebSocket relayed as a session
POST /rerank                     the same route under its other name
POST /v1/messages                Anthropic Messages, streamed or not
POST /api/chat                   Ollama chat; streams unless "stream": false
POST /api/generate               Ollama generate; streams unless "stream": false
GET  /api/tags                   Ollama: the same names as /v1/models
GET  /api/ps                     Ollama: always an empty list
GET  /openapi.json               OpenAPI 3.1 description of the routes this config enables
GET  /docs                       Swagger UI; only with openapi.swagger_ui, else 404
GET  /v1/mcp/servers             the MCP servers assigned to the presented key
ANY  /mcp/:name                  POST, GET, DELETE: relayed to that MCP server
ANY  /mcp/:name/*path            the same, to a sub-path of it

/health stays ok while a config reload is failing -- the proxy serves under the config it kept -- and shows the reload state without its message. Likewise a failing customer source ("customer_source" in Langertha::Skeid): customer_source carries ok, and failed_at and failures while it fails, never the message. /v1/models and /api/tags list the node models and the alias names, each once, so a client can discover what it may put in model. The list follows the policy of the key presented (no key: the default policy): a name the key's models do not grant, an alias with every tier denied and a model only denied nodes serve are left out. Unhealthy nodes are included. A node without a model is not listed -- it matches any requested name, so no name reaches it in particular. The manifest route answers 404 unless the config enables it, 401 without a key and 403 for a key without a grant; see "Provider Manifest" in Langertha::Skeid.

Audio routes

/v1/audio/transcriptions and /v1/audio/translations take OpenAI's multipart/form-data upload -- file, model and whatever else the client sends -- and relay it to {node url}/audio/transcriptions or /audio/translations, the endpoint vLLM (Whisper) and speaches both serve. Nothing is translated in either direction (ADR 0021): there is one dialect for the request, and the answer is the node's own. What Skeid knows of that wire is in Langertha::Skeid::Protocol::Audio.

The request is routed like a chat request -- by the form's model field, through aliases, tiers, the key's policy, eligibility and admission -- and the form goes upstream part for part: each part with its own headers and content, in the client's order, under the client's boundary. Fields Skeid does not know and fields that repeat (timestamp_granularities[]) pass through. One part is rewritten: model, when an alias tier serves another model than the one asked for. The upload itself is never copied: the server spools a part above 256 KiB to a temporary file, and the upstream request is sent from that file. The client's Expect header is not passed on.

Skeid reads two fields. model has to be there exactly once, as a non-empty value of at most 1024 bytes; anything else is a 400 before a node is touched -- with two, Skeid would route by one while the node may serve the other. stream picks the relay: the last one in the form decides, and it is on for true, 1, on, yes, t and y in any case, which is what the nodes read as true. A part's name is taken from its Content-Disposition however it is written (quoted or bare, any case); a name in the extended name* notation is refused.

The answer comes back as the node gave it -- its status, its headers but for the framing ones, its body untouched: JSON, verbose_json, plain text, srt, vtt. A stream is relayed byte for byte like a chat stream, in whichever event dialect the node speaks -- OpenAI's and speaches' transcript.text.delta / transcript.text.done, or vLLM's transcription.chunk frames ending in [DONE] -- and nothing is added to the form to make the node report usage: a vLLM node sends its usage frame when the client's form carries stream_include_usage=true.

The usage event has api_format openai and the route as its endpoint. Tokens are recorded when the node reports them (OpenAI's usage.type: tokens, vLLM's stream usage frame); audio_seconds when it reports a duration -- usage.seconds of a usage.type: duration block, else the top-level duration of a verbose_json answer; a streamed request also has content_bytes, the UTF-8 bytes of the text relayed. A node that reports neither count leaves an event with a status and a duration and no audio_seconds key: not measured, which is not zero. See "Pluggable Usage Storage" in Langertha::Skeid.

Realtime route

GET /v1/realtime?model=<name> upgraded to a WebSocket is live transcription over OpenAI's realtime protocol: the client sends input_audio_buffer.append frames and gets conversation.item.input_audio_transcription.delta and .completed events. Skeid relays the frames to {node url}/realtime and back in the node's own events (ADR 0029, ADR 0021): nothing is translated, so the beta transcription_session.update and the GA session.update both go through as the client speaks them. What Skeid knows of that wire is in Langertha::Skeid::Protocol::Realtime; the session itself is Langertha::Skeid::Proxy::Realtime.

The handshake is a request. Everything that can refuse a session answers as plain HTTP before a socket exists: 401 from client_auth, 400 without exactly one model in the query (intent=transcription alone names none), 403 from the key's policy, 503 for a model no node serves, 429 after the route wait. Then Skeid resolves the node's key and opens the node's WebSocket first; the client is upgraded only when the node accepted, so a node that refuses is a 502 (and its slot is given back), never a socket that was accepted and closed. A client that leaves meanwhile gives its slot back as well. The node is called with its own key and with none of the client's Sec-WebSocket-* headers, Origin or cookies.

A key from a browser. A page cannot set Authorization on a WebSocket. On this route, and only when neither Authorization nor x-api-key is present, the key is read from the Sec-WebSocket-Protocol entry openai-insecure-api-key.<key> and goes through the same client gate, so a key that is not allowed is a 401 before the upgrade. It is never answered: the subprotocol Skeid selects is realtime (or none), and it is neither logged nor forwarded.

The model of a session is the handshake's. Every client text frame is JSON-decoded -- never tested by substring, since the node decodes it again and JSON spells one text many ways. Each string model in it is compared with the session's: the same one is rewritten to the served model (an alias tier), any other is not forwarded -- the client gets an error event and the session goes on, on the model it started with. What goes to the node is the decoded document, encoded again. A frame that is not a JSON object, and any binary frame, closes the session 1003.

A session is one slot and one usage event. The slot is taken at the handshake and given back exactly once when the session ends -- from either side, from a limit or from a graceful stop -- so max_conns of a realtime node counts concurrent sessions; give it a node of its own, with max_conns at least the worker count, since every prefork worker admits its share and keeps the sessions it accepted (ADR 0010). One usage event is written when the session closes: endpoint /v1/realtime, duration_ms the session's length, the usage blocks of its completed transcriptions summed and priced, audio_seconds where a block is a duration, content_bytes the text relayed, and a status: 200 for a close from either side (any code), 499 for a client that vanished without a close frame, 502 for a node that closed with an error or died, 413 for a message above realtime.max_message_bytes, 400 for a frame a session cannot carry. A count the node never reported is absent, not zero.

Bounds. The realtime section of the config ("realtime" in Langertha::Skeid): a message above max_message_bytes closes 1009, bytes a peer does not read past max_queued_bytes close 1013, silence for idle_timeout seconds and max_session_seconds of age close 1000 with a reason. On a graceful stop (SIGQUIT, the event loop's finish) every open session is closed 1001 and its event written before the process exits.

Rerank route

/v1/rerank and /rerank are one route. It takes the request shape Cohere set and vLLM, Jina and infinity took over -- model, query, documents, optionally top_n, return_documents and whatever else the node understands -- and answers with the node's results. It is not a streamed route.

Four things are checked before a node is picked, each a 400 that costs no slot and no usage event: the body is a JSON object, model is a non-empty string, query is a string, documents is a non-empty array. What a document may be is left to the node. The request is then routed like a chat request -- by model, through aliases, tiers, the key's policy, eligibility and admission.

By default the request is relayed (ADR 0021): it goes to {node url}/rerank with every field as the client sent it and model replaced by the served model, and the node's answer comes back as it is -- status, headers but for the framing ones, body byte for byte. That covers vLLM, infinity started with --url-prefix /v1, Jina and Cohere, and Skeid does not even out what differs between them: document is { "text": ... } on vLLM and Cohere and a plain string on infinity, and vLLM returns the documents whether return_documents asked for them or not. The body is decoded and written again on the way, so the node sees exactly one model -- the one Skeid routed by.

A node with rerank_format: tei ("nodes" in Langertha::Skeid) is Hugging Face text-embeddings-inference, which speaks another dialect, and is translated in both directions. The request goes to /rerank at the server root -- the node's URL with or without a trailing /v1 -- as query, texts (each document a string, or an object's string text), return_text for return_documents and truncate when the client sent them, and nothing else. TEI's bare array becomes { model, results, usage }: model the served model, results sorted by score with relevance_score and, when return_documents asked for it, document => { text }, cut to top_n; usage.total_tokens from TEI's x-compute-tokens header when it sent one. A request such a node cannot take -- a document that is not text, a top_n that is not a whole number of at least zero -- is a 400 as well, but one known only after the node was admitted: the slot is given back, and one failed usage event is written. An answer that is not TEI's array is a 500 api_error, as for any translation that fails. The wire of both dialects is Langertha::Skeid::Protocol::Rerank's.

The usage event has api_format openai and endpoint /v1/rerank for both spellings. documents is the number of documents the request carried, on the event of a request the node answered and on no other. The node's tokens are recorded as input tokens and as the total, so the model's input_per_million prices them, from wherever that node reports them: usage.prompt_tokens, else usage.total_tokens, else Cohere's meta.tokens.input_tokens, or a TEI node's x-compute-tokens. A node that reports none leaves an event without tokens. infinity counts characters unless it runs with lengths_via_tokenize; Skeid records what the node says. See "Pluggable Usage Storage" in Langertha::Skeid.

Documents routes

POST /v1/convert/file and POST /v1/convert/source are the synchronous calls of docling-serve under its own paths (skeid k94, ADR 0028), so a docling client only changes its base URL. They are relayed like the audio routes: the multipart form goes upstream part for part, a source body (JSON, files as base64) is decoded once and written again, and the node's answer -- JSON, or a zip when several documents or outputs come back -- is relayed byte for byte, whatever its status. The asynchronous calls are not offered.

The request has no model. It is planned over the nodes of kind: documents ("nodes" in Langertha::Skeid) under the routing key documents: a key whose policy lists models must list documents or is answered 403; an alias named documents tiers the nodes by tags; chat requests never reach these nodes and these never reach chat nodes, and /v1/models lists none of it. Before a node is chosen, a request that makes the node fetch from or call an address the client names -- http_sources, callbacks, a target that is not inbody or zip, a sources entry that is not a file, in any spelling, named twice or not -- is a 400 that names the field and costs no slot and no event, unless documents.allow_remote_sources.

The body limit is uploads.documents_max_bytes (default 100 MiB), applied as for audio before the body is read: 413, or 401 for a caller client_auth refuses. A form is spooled to disk and sent from there; a source body is decoded in memory, once, which is CPU on the event loop proportional to its size -- big files belong to /v1/convert/file. The node's key goes in the header its api_key_header names (X-Api-Key for docling-serve), the client's credentials are dropped. The wait for a conversion is the upstream timeout, unchanged.

The event has api_format docling, model documents, documents (files or sources the request held, on a request the node answered) and pages, read off document.json_content.pages of a JSON answer up to 8 MiB -- nothing is decoded beyond that, a zip is not decoded at all, and an answer without a page count leaves pages out. No tokens, nothing priced. See Langertha::Skeid::Protocol::Documents.

MCP routes

The MCP gateway (skeid k103, ADR 0022): a customer reaches the MCP servers the config assigned to its key ("mcp" in Langertha::Skeid) with that key, one route per server. Skeid relays MCP's Streamable HTTP transport and reads none of it -- no JSON-RPC, no tool list, no session. What it knows of the wire is in Langertha::Skeid::Protocol::MCP.

Both routes sit behind one check, in this order. Without an mcp section they are unknown paths and answer as one, 404, whatever the caller presents. Then client authentication, as on every client route ("Customer identity"). Then a key at all: no key is 401 Missing API key also while client_auth is off, because a server is assigned to a key id and no key has none. The identity is the one every route uses, routing.trust_key_id_header included.

GET /v1/mcp/servers lists the servers assigned to the presented key, sorted by name:

{ "object": "list", "data": [
  { "name": "bookrag", "transport": "streamable-http",
    "url": "https://llm.example.com/mcp/bookrag", "description": "..." },
  { "name": "files", "transport": "stdio", "command": "mcp-files",
    "args": ["--root", "/srv/data"], "env": {}, "description": "..." } ] }

url is mcp.public_url (else the scheme and host of this request) plus /mcp/<name>. A stdio entry is a hint for a server on the customer's own machine. The upstream URL, the key reference, the variable name and the static headers are never in it. A key with no assignment gets an empty list. The answer is Cache-Control: private, no-store.

POST, GET and DELETE on /mcp/<name> are relayed to the server's url; a path after the name is appended to it segment by segment, and the client's query to its query. Everything that is not a relayed server of the presented key is the same 404 not_found: an unknown name, a server assigned to another key, a stdio entry, a sub-path with a . or .. segment or a backslash. A key cannot tell which names exist.

Upstream go the method, the body as the bytes it came in, and of the client's headers Content-Type, Accept, Last-Event-ID and every Mcp-* header (Mcp-Session-Id, Mcp-Protocol-Version) -- nothing else, so never the client's Authorization, X-Api-Key or cookies. Then the server's static headers, and Authorization: Bearer with the server's own key when it names a key source. A server that names one which yields no key is not called: 503 upstream_key_unavailable, logged with the key reference.

The answer comes back as the server gave it: its status, its headers but for the hop-by-hop ones, Content-Length and Content-Encoding, and its body. The response is opened when the upstream's headers arrive, before any body byte, and every chunk is written as it is read, so an event stream (text/event-stream) is not buffered and a standing GET stream is open for the client before its first event. An answer without a body (202, 204) is relayed as one. The upstream's own 4xx and 5xx are relayed too -- a 404 for an expired session has to reach the client -- with two exceptions, both answered 502 upstream_error and logged. An upstream 401, because towards the client 401 means its Skeid key was not let in, while the upstream's is about the key Skeid presented. And a redirect (3xx): its Location is an upstream URL, and Skeid follows none -- configure the URL it leads to, a trailing slash included. A server that cannot be reached is 502 upstream_error; a request body over the server's request limit is 413.

The upstream call runs under the upstream timeout of "build_app" (SKEID_UPSTREAM_TIMEOUT), and the client's connection is given the same time: a standing stream is cut after it, and an MCP client reconnects with Last-Event-ID. A client that hangs up closes the upstream connection. There is no routing and no admission -- an MCP server is not a node: no max_conns, no health, no capacity probe.

Each relayed request writes one usage event with api_format mcp, /mcp/<name> as its endpoint (without the sub-path), the customer key id, the server's url as route_url, the status and the duration; no model, node, tokens or cost. ok is the status below 500. A refusal for want of an upstream key is a 503 event, an upstream 401 a 502 event with error_type upstream_unauthorized, a redirect one with upstream_redirect, an unreachable server a 502 event. A client that leaves before a plain answer is complete is the 499 client_abort of every route; an event stream that was opened is recorded under its status however it ends, since closing it is how such a stream ends. 401 and 404 write nothing.

Uploads

The audio routes are the upload routes: their request limit is the config's uploads.max_bytes ("uploads" in Langertha::Skeid, default 26214400), not the server's own (Mojolicious' 16 MiB, which every other route keeps). A larger body is answered 413 in the OpenAI error shape, with no node touched and no usage event.

The limit is applied when the request's head has arrived, before its body is read. A request that declares a larger Content-Length is answered at once and its connection closed; a body of undeclared length (chunked) is cut off once it is 1 MiB past the limit, and measured exactly if it ends before that. With client_auth, a caller the list does not let in is cut off the same way -- its 401 does not wait for, or store, the upload. The limit is read per request, so a changed uploads.max_bytes holds from the next one.

A client that sends Expect: 100-continue (curl does, above 1 MiB) gets no 100 Continue: the server underneath does not send one. It gets a 401 or 413 right away, and otherwise sends the body after its own timeout.

Registry route

GET  /skeid/registry/snapshot    signed capacity snapshot, for a fronting Skeid

Bearer token: the admin API key or the registry read key (registry.read_key_env), and nothing else accepts the read key. 404 when neither is configured or the registry is not enabled, 401 for a wrong token, 503 while the signing secret is missing. The body is signed in X-Skeid-Registry-Signature and sent Cache-Control: no-store. See "registry_enabled" in Langertha::Skeid and ADR 0017.

Admin routes

Bearer token: the admin API key ("admin" in Langertha::Skeid). Without one configured every /skeid/* route answers 404; a missing or wrong token answers 401.

GET  /skeid/nodes                {nodes}
POST /skeid/nodes                body: a node, as a config nodes entry -> {ok, nodes}, or 400
POST /skeid/nodes/:id/health     body: {"healthy": true|false} -> {ok}
GET  /skeid/config               {reload}: the config reload status, with its message;
                                 {customer_source}: the customer source status, with its message;
                                 {customer_names}: customer key id => name of the source's rows;
                                 {customer_report}: rows accepted, rejected, shadowed (labels, reasons)
GET  /skeid/metrics/nodes        {metrics}: per-node counters, never billed
GET  /skeid/usage                ?since=&api_key_id=&model=&limit= (default 50) -> the report
GET  /skeid/openapi.json         OpenAPI 3.1 description of the admin routes

Changes made here live in this process only: a changed nodes section in the config replaces them, and under --workers each write reaches one worker (ADR 0010).

Customer identity

Skeid does not authenticate customers unless the config has a client_auth section ("client_auth" in Langertha::Skeid). The key a client presents (Authorization: Bearer, else x-api-key) derives the customer key id ("key_id_for_key" in Langertha::Skeid; no key is anonymous), which selects the routing policy and is recorded on the usage event. With routing.trust_key_id_header a x-skeid-key-id (or x-api-key-id) header names the key id instead.

With client_auth, that key id has to be on the section's list. Every client route checks it first -- /v1/models, /v1/chat/completions, /v1/embeddings, the /v1/audio/* routes, /v1/rerank and /rerank, /v1/messages, every /api/* route, /.well-known/langertha.json and, with an mcp section, the MCP routes -- after picking up a changed config ("maybe_reload_config" in Langertha::Skeid), so a list change holds from the next request. No key is 401 Missing API key, a key whose id is not listed 401 Invalid API key, both with WWW-Authenticate: Bearer realm="skeid", and nothing else happens for that request: the body is not parsed, no node is admitted or called, no usage event is written, and Skeid logs nothing about it. The answer names neither the key nor its id. /health and the /skeid/* routes are not checked; they have their own rules. Being let in widens nothing: the key's routing policy still applies.

The upstream call

The node URL gets /v1 added unless it ends in it, then /chat/completions or /embeddings (an audio route: /audio/transcriptions or /audio/translations, see "Audio routes"; rerank: /rerank, for a TEI node without the /v1, see "Rerank route"); the body carries the served model (an alias tier's model), everything else as the client sent it or as translated. The client's headers go upstream except the hop-by-hop ones, Host, Content-Length and Accept-Encoding. When the node has a key of its own -- api_key_ref through the key broker, else api_key_env -- it replaces Authorization and the client's Authorization and x-api-key are dropped, however the client spelled them. A node that names neither forwards the client's own key. A node that names one and gets no key from it -- the broker fails or is not running, the variable is unset or empty -- is not called at all, and neither is one that left the inventory after it was selected: the request is refused with 503 (see "Errors"), so the client's key never stands in for the node's. An answer that came from a node carries x-skeid-node with the node id. Rate-limit headers and 429s on every response feed "observe_response_headers" in Langertha::Skeid.

Each admitted request gets its request.finish on every path and one usage event, failures included.

A client that hangs up

A client that closes its connection before the answer is complete ends its request at that moment. While it waits for capacity it stops waiting and takes no slot. Once a node was called, the upstream connection is closed -- which is how the node learns to stop generating -- and is not returned to the pool; the slot is given back with a request.finish marked aborted -- counted apart, not as a node error (and a request the node refused as the client's own fault, 400, 404, 413 or 422, is marked rejected and counted apart the same way, ADR 0024), so the node's error counter and the registry snapshot's errors_in_window stay untouched -- and the one usage event is written with ok = 0, status_code 499 and error_type client_abort, priced from the usage the stream had reported until then (nothing, for a request that was not streamed). A client that leaves while the node's key is still being resolved gives its slot back too, but nothing was forwarded, so no usage event is written. When the node had already finished and only the rest of the answer was still being written out, the request stays what it was: finished and metered by the node's answer.

Errors

A key the client_auth list does not let in is 401 (OpenAI: type invalid_request_error, code invalid_api_key; Anthropic: authentication_error; see "Customer identity"). An upload over uploads.max_bytes is 413 invalid_request_error; an audio request that is not a multipart form, or whose form has no usable model field, is 400 invalid_request_error, as is a rerank request without a model, a string query or documents. A request no node may serve for this key is 403 permission_error; a model no healthy node serves is 503 model_not_found; eligible nodes that stay full past the wait are 429 rate_limit_error; an upstream failure is its status (or 502) with type upstream_error; a node whose own key cannot be resolved is 503 upstream_key_unavailable, logged with the key reference and recorded as a failed usage event. The body is shaped for the face that was called: OpenAI's {error: {message, type}}, Anthropic's envelope ("error_body" in Langertha::Skeid::Protocol::Anthropic) on /v1/messages, and Ollama's {error: "..."} on /api/*. A stream that fails after it opened ends with the face's in-band error event where it has one.

The MCP routes answer their own errors in the OpenAI shape: 401 without a key, 404 not_found for whatever is not a relayed server of the presented key, 413 for a body over the request limit, 502 upstream_error for a server that is unreachable, answers 401 or redirects, 503 upstream_key_unavailable for one whose key is missing. Every other status on /mcp/<name> is the MCP server's own, body included; see "MCP routes".

METHODS

build_app

my $app = Langertha::Skeid::Proxy->build_app(%options);

Builds the Mojolicious application. Options:

• config_file -- the config to build a Langertha::Skeid from.

• skeid -- an existing Langertha::Skeid to serve instead; config_file and the OpenBao detection below are then not used.

• admin_api_key -- the explicit admin API key ("set_admin_api_key" in Langertha::Skeid): it wins over the config's, on every reload. Empty leaves the key to the config and SKEID_ADMIN_API_KEY.

• worker_count -- how many prefork workers share the nodes ("worker_count" in Langertha::Skeid), set before any admission or probe timer reads it.

• swagger_ui -- a Langertha::Skeid::SwaggerUI object to serve /docs from; a test passes one with an injected fetcher so nothing reaches the network. Default: one that fetches the pinned release.

The Skeid's "on_usage_lost" in Langertha::Skeid is set to this app's usage event lost log line, so a usage event a write-behind store could not write is logged like one whose synchronous write failed. An embedding application that wants its own hook sets it after build_app.

With both OPENBAO_ROLE_ID and OPENBAO_SECRET_ID set, a Langertha::Skeid::KeyBroker::OpenBao at OPENBAO_ADDR (default http://127.0.0.1:8200) becomes the key broker; if its login fails the proxy warns and runs without one. A broker that can renew its token starts renewing on a timer. The capacity probes of every node are started and restarted whenever the probed part of the inventory changes, and the customer source of the config, if it has one, starts fetching ("start_customer_source" in Langertha::Skeid).

Upstream connections time out after 10s to connect, and at most SKEID_UPSTREAM_POOL (default 100) are kept. An upstream request may take SKEID_UPSTREAM_TIMEOUT seconds (default 300; a positive integer, anything else counts as unset) and may be silent for all of them -- the time to the first token is silence on the wire. The client's connection is given the same time on top of the server's own inactivity timeout, on the routes that call an upstream and for that request only: the proxy never closes a request its upstream is still working on, and is still there to answer when the upstream timed out. Every other route stays under the server's timeout. The client's side is read from the user agent when a request arrives, so whoever changes $app->ua->request_timeout afterwards changes both sides, and should set $app->ua->inactivity_timeout to match.

The app has a skeid helper returning the control plane.

route_table

my @entries = Langertha::Skeid::Proxy->route_table;

Every route the app registers, as one entry each -- the single source of the router and of the OpenAPI documents (ADR 0025). An entry is a hashref:

• id, group (the bridge it sits behind), methods, paths (Mojolicious patterns), handler (a coderef taking the controller)

• document: client or admin for the OpenAPI document that lists it, absent for a route no document lists (the UI)

• requires: manifest, mcp or registry -- the config section that has to be on for the route to be listed

• websocket: the route is registered as a WebSocket route (only an upgrade request reaches it) and documented with a 101; query lists the query parameters the document names (name, required, description)

• tag, summary, description, request, request_type, response, stream, public: what the document says about it

A route added to the app any other way has no entry; the test suite fails on it.

SEE ALSO

• Langertha::Skeid -- the control plane and its configuration

• skeid -- the command that runs this app

• Langertha::Skeid::Protocol::Anthropic, Langertha::Skeid::Protocol::Ollama -- the translated faces

• Langertha::Skeid::Protocol::Audio -- the wire of the relayed audio routes

• Langertha::Skeid::Protocol::Rerank -- the wire of the rerank route, and its TEI dialect

• Langertha::Skeid::Protocol::Realtime, Langertha::Skeid::Proxy::Realtime -- the wire and the session of the realtime route

• Langertha::Skeid::Protocol::MCP -- the wire of the MCP gateway

• Langertha::Skeid::KeyBroker::OpenBao -- upstream keys from OpenBao

SUPPORT

Issues

Please report bugs and feature requests on GitHub at https://github.com/Getty/langertha-skeid/issues.

IRC

Join #langertha on irc.perl.org or message Getty directly.

CONTRIBUTING

Contributions are welcome! Please fork the repository and submit a pull request.

AUTHOR

Torsten Raudssus <torsten@raudssus.de> https://raudssus.de/

COPYRIGHT AND LICENSE

This software is copyright (c) 2026 by Torsten Raudssus.

This is free software; you can redistribute it and/or modify it under the same terms as the Perl 5 programming language system itself.

cpanm Langertha::Skeid

perl -MCPAN -e shell
install Langertha::Skeid