/ 
Langertha-0.504
/ Langertha::Role::StreamUsage

NAME

Langertha::Role::StreamUsage - Role for an engine whose stream request asks for the token usage

VERSION

version 0.504

SYNOPSIS

package My::CompatibleEngine;
use Moose;
extends 'Langertha::Engine::OpenAIBase';
with 'Langertha::Role::StreamUsage';

# Caller side
my ( $content, $chunks ) = await $engine->chat_stream_realtime_f(
  messages => [ { role => 'user', content => 'Hello' } ],
);
my $usage = $engine->aggregate_usage($chunks);   # the prompt and completion tokens

DESCRIPTION

Composing this role gives the engine the stream_usage capability: its chat_stream_request asks the provider for the stream's token usage, and the usage arrives as the stream's last chunk. On the OpenAI chat wire that is stream_options => { include_usage => true } on the request body, set by "chat_stream_request" in Langertha::Role::OpenAICompatible; the provider then sends a frame with an empty choices list and the usage after the finish chunk, which parse_stream_chunk turns into a content-less, non-final Langertha::Stream::Chunk carrying usage. Collect it with "aggregate_usage" in Langertha::Role::Chat.

The role has no attributes or methods: it only marks that the engine's wire documents the option. Some OpenAI-compatible servers reject a body field they do not know, so the option is sent only where the provider's API reference names it (ADR 0038). Engines without the role send no stream_options; whether their stream reports usage anyway is the provider's affair (DeepSeek and OpenRouter always do, Groq also under x_groq.usage).

A stream_options the caller passes to the stream request replaces the engine's whole, so stream_options => { include_usage => JSON->false } turns the usage off for one request.

SEE ALSO

SUPPORT

Issues

Please report bugs and feature requests on GitHub at https://github.com/Getty/langertha/issues.

IRC

Join #langertha on irc.perl.org or message Getty directly.

CONTRIBUTING

Contributions are welcome! Please fork the repository and submit a pull request.

AUTHOR

Torsten Raudssus <getty@cpan.org>

COPYRIGHT AND LICENSE

This software is copyright (c) 2026 by Torsten Raudssus https://raudssus.de/.

This is free software; you can redistribute it and/or modify it under the same terms as the Perl 5 programming language system itself.