NAME
Langertha::Skeid::Protocol::Rerank - The rerank routes -- relayed as the node speaks, translated for a TEI node
VERSION
version 0.004
DESCRIPTION
Serves POST /v1/rerank and its alias POST /rerank. Reranking has no OpenAI definition; the shape clients speak is the one Cohere set and vLLM, Jina and infinity took over:
{ "model": "...", "query": "...", "documents": [ "...", ... ], "top_n": 3, "return_documents": true }
{ "results": [ { "index": 2, "relevance_score": 0.98, "document": { "text": "..." } }, ... ],
"usage": { "total_tokens": 41 } }It is a relayed route (ADR 0021): a node that speaks this shape gets the client's body with only model replaced, and its answer goes back untouched. What differs between those nodes is therefore left to differ -- document is { "text": ... } on vLLM and Cohere and a plain string on infinity, vLLM returns the documents whether asked or not.
One kind of node does not speak it at all: Hugging Face text-embeddings-inference (TEI), whose only rerank route is POST /rerank at the server root, taking query and texts and answering a bare array of { index, score, text } in document order, its token count in a response header. A node marked rerank_format: tei is translated here, in both directions -- a translation on the upstream side of the hub, where every other one sits on the client side. This module is the only place that knows either wire: the field names of the request, TEI's names, and the three places a reranker reports tokens.
Everything here is a class method.
routes
my @paths = Langertha::Skeid::Protocol::Rerank->routes;The client paths of the rerank route, sorted: /rerank and /v1/rerank. Both are one route.
endpoint
The endpoint a rerank request's usage event carries, whichever spelling the client used: /v1/rerank.
upstream_path
The path a rerank request is sent to at the node: /rerank, below the node's /v1 or, for a format "at_server_root" names, at the server root.
formats
my @names = Langertha::Skeid::Protocol::Rerank->formats; # ('tei')The values a node's rerank_format can take, sorted.
normalize_format
my $format = Langertha::Skeid::Protocol::Rerank->normalize_format($node{rerank_format});A node's rerank_format as it is kept: lowercased and trimmed, the empty string for none -- the node speaks the client's shape and is relayed. Croaks on a value that is not one of "formats", so a misspelled format fails the config load instead of leaving a TEI node to be called in a dialect it does not speak.
at_server_root
my $root = Langertha::Skeid::Protocol::Rerank->at_server_root($format);Whether a format's rerank route sits at the server root instead of below /v1. True for tei: a node URL written with or without a trailing /v1 is called at {root}/rerank.
request_problem
my $problem = Langertha::Skeid::Protocol::Rerank->request_problem($c->req->json);Why a decoded request body cannot be routed, as a message for the client, or undef when it can: a body that is not a JSON object, a model that is not a non-empty string, a query that is not a string, documents that is not a non-empty array. Checked before a node is picked, so none of these costs a slot or a usage event. What a document may be is not checked here -- that is the node's to say, and nodes differ.
document_count
my $count = Langertha::Skeid::Protocol::Rerank->document_count($body);How many documents a request carries: what the usage event records as documents. Counted off the request, so it is there whatever the node reports.
request_to_upstream
my $upstream = Langertha::Skeid::Protocol::Rerank->request_to_upstream($body, $served_model, $format);The body sent to the node. Without a format it is the client's own, every field as sent, with model replaced by the served model (an alias tier's, ADR 0008).
For tei it is translated: query as it is, documents as texts -- a document is a string or an object with a string text --, return_documents as return_text and truncate when the client sent them, and nothing else: TEI serves one model and takes no model, and top_n is applied to the answer. A document that is not text, and a top_n that is not a whole number of at least zero, throw a Langertha::Skeid::Protocol::Refusal with a message for the client: such a request cannot be put to this node.
response_from_upstream
my $answer = Langertha::Skeid::Protocol::Rerank->response_from_upstream(
$payload, $headers, $body, $served_model, $format);What the client gets for the answer of a node that has a format: $payload is the decoded answer, $headers the Mojo::Headers it came with, $body the client's request. For tei, the bare array becomes
{ "model": <served model>,
"results": [ { "index": 2, "relevance_score": 0.98, "document": { "text": "..." } }, ... ],
"usage": { "total_tokens": 41 } }sorted by score, highest first (equal scores in document order), cut to top_n when that is at least 1 (0, null and no top_n are all of them). document is there when the client asked with return_documents and the node returned the text; usage when the node's x-compute-tokens header carries a count. Dies on an answer that is not such an array -- the proxy answers that as a failed translation. A node without a format is not translated and this is not called for it.
usage
my $usage = Langertha::Skeid::Protocol::Rerank->usage($payload, $headers, $format);
# { prompt_tokens => 41, completion_tokens => 0, total_tokens => 41 }, or undefThe tokens a rerank answer reports, as the usage block of the one dialect Skeid meters (metrics.normalize), or undef when the node reported none. A reranker generates nothing, so what it counts is input: the block carries the count as prompt_tokens and as the total, and the model's input_per_million prices it.
Where the count is read: usage.prompt_tokens, else usage.total_tokens (vLLM, infinity, Jina; older vLLM and Jina report only the total); else meta.tokens.input_tokens (Cohere); for a tei node the x-compute-tokens response header. A value that is not a whole number of at least zero counts as not reported. Cohere's meta.billed_units.search_units is not read: it is not a token count. infinity counts characters, not tokens, unless it runs with lengths_via_tokenize -- Skeid records what the node says.
SEE ALSO
"Rerank route" in Langertha::Skeid::Proxy, Langertha::Skeid::Protocol::Audio, Langertha::Skeid::Protocol, "Pluggable Usage Storage" in Langertha::Skeid
SUPPORT
Issues
Please report bugs and feature requests on GitHub at https://github.com/Getty/langertha-skeid/issues.
IRC
Join #langertha on irc.perl.org or message Getty directly.
CONTRIBUTING
Contributions are welcome! Please fork the repository and submit a pull request.
AUTHOR
Torsten Raudssus <torsten@raudssus.de> https://raudssus.de/
COPYRIGHT AND LICENSE
This software is copyright (c) 2026 by Torsten Raudssus.
This is free software; you can redistribute it and/or modify it under the same terms as the Perl 5 programming language system itself.