/ 
App-karr-0.600
/ App::karr::Foundation::ChainStore

NAME

App::karr::Foundation::ChainStore - karr-foundation chain and run-log storage under refs/karr-foundation/*

VERSION

version 0.600

SYNOPSIS

use App::karr::Foundation::ChainStore;

my $chain = App::karr::Foundation::ChainStore->new(
    git => App::karr::Git->new( dir => $hub_repo ) );

$chain->write_chain( [
    { id => 1, kind => 'ticket', repo => '/srv/karr',  ticket => 41 },
    { id => 2, kind => 'ticket', repo => '/srv/other', ticket => 12 },
    { id => 3, kind => 'plan',   needs => [ 1, 2 ],
      precheck => 'ticket_status == todo', on_stall => 'plan' },
], limits => { concurrent => 4 } );

my @now = $chain->ready_steps;          # steps 1 and 2, concurrently

DESCRIPTION

The shared half of karr-foundation's fleet execution: the planned chain of steps and the log of the runs that worked through it. Both live in git refs under refs/karr-foundation/, so every machine and every person sees the same picture.

refs/karr-foundation/chain/meta         the chain header (YAML)
refs/karr-foundation/chain/step/<id>    one step (YAML)
refs/karr-foundation/log/<date>-<id>    one run's log (JSON lines)

What does not live here is just as deliberate. Which agent commands exist on this machine, whether they currently work and when to try them again is execution, it is local, and it belongs to App::karr::Foundation::Agents and agents.state. That is why a step names no agent and why passing one is refused rather than ignored: a chain is shared state, an agent is a property of a machine, and a chain that named one would plan work that cannot run anywhere else.

A chain gets in through karr-foundation plan, which reads one YAML or JSON document and hands it to "parse_chain_document" and "write_chain" -- the same path a Perl caller takes, and the only way in: karr set-refs refuses this namespace outright, because a schema, a cycle check and compare-and-swap updates are not things a generic ref writer can honour.

The chain is a DAG

A step lists the step ids it needs. Steps with no edge between them may run concurrently, which is how the planner expresses parallelism: it leaves edges out rather than serialising by hand. "ready_steps" is the whole query -- the pending steps whose needs are all done -- and the runner (#186) is what turns that into processes.

Cycles are refused at "write_chain", not discovered at run time: a chain with a cycle has steps that can never become ready, and a store that accepted one would answer "nothing to do" forever while looking healthy.

A stale precheck costs time, not correctness

A step may carry a precheck -- ticket_status == todo -- which is the condition the planner assumed when it wrote the step. A step whose precheck no longer holds is not executed: it is marked stale ("mark_stale") and the planner is called again. This is what stops a chain that has gone out of date from doing damage, and it is why an unreadable precheck (an unknown fact, a missing fact) counts as not holding -- every uncertainty falls to the side that costs a planning round rather than the side that runs the wrong thing.

Run logs are segmented, like the activity log

refs/karr-foundation/log/<date>-<id> is one ref per run, and an entry is appended to the newest segment of that ref until it reaches "segment_max_bytes"; the next entry opens ...<run>+000001. A ref holds a blob and a blob is rewritten whole on every append, so an uncapped log ref makes each entry cost a copy of the entire history -- quadratic, and measured at about 4.6 GB of objects for a 1 MB log (#171, "Segments" in App::karr::ActivityLog). The +NNNNNN spelling is deliberately the same one the activity log uses; this store keeps its own copy of the mechanism rather than sharing one, because the activity log's version is tangled up with identity encoding and pre-#75 ref names that have no meaning here.

Retention is the other half of that bound: "prune_logs" drops runs older than "keep_days" and, whatever their age, everything past the newest "keep_runs". It runs by itself when a run log is opened ("auto_prune"), because a retention policy that only runs when somebody types a command bounds nothing.

SEE ALSO

App::karr::Foundation, App::karr::Foundation::Agents, App::karr::ActivityLog, App::karr::Git

git

The App::karr::Git for the hub repository that carries the fleet namespace. Required.

write_chain

my $chain_id = $store->write_chain( \@steps, %opt );

Replaces the chain with @steps and returns the new chain id. Options are limits (passed through to the header untouched -- what a limit means is the runner's business), note, planner, and force.

Every step is validated first ("validate_chain"): anything wrong with the chain raises a user error and nothing is written.

The header ref is written last and is the commit point: "ready_steps" only considers steps whose chain matches the header, so a reader that arrives half-way through a replacement sees the chain it saw before, then the new one, and never a mixture. Both meta and the step refs are separate refs, so this is not a git-level atomic switch -- the window is one where nothing is ready, not one where the wrong thing runs.

validate_chain

my $steps = $store->validate_chain( \@steps, %opt );

Everything "write_chain" checks before it writes a ref, and no ref written: every step against the step schema, ids unique, every needs entry naming a step of the same chain, the graph acyclic, and -- unless force is passed -- no step of the chain still in state running. Returns the validated steps, which are normalised copies rather than the caller's own hashes; anything else raises a user error.

Split out of "write_chain" because karr-foundation plan --dry-run has to be able to say "this chain is good" without writing it, and a dry run checking a chain from its own copy of the rules would be a second opinion rather than the same one.

parse_chain_document

my ( $steps, %header ) = $store->parse_chain_document( $document );

Takes the decoded document karr-foundation plan reads -- YAML or JSON, and JSON only because a YAML parser reads it -- and returns the two arguments "write_chain" takes: the step list, and the header options limits, note and planner.

Two spellings are accepted, because both say the same thing: a mapping with a steps: list and the header keys beside it, or a bare list, which is the step list. The second is what "write_chain"'s own first argument looks like, so a planner writing only steps has written a whole document.

No step is looked at here -- that is "validate_chain", which the write path runs whatever route the steps arrived by. What this checks is the envelope: the document is one of the two shapes, steps: is there and is a list, limits: is a mapping, note: and planner: are plain values, and no other key is present. force is deliberately not among them: replacing a chain that still has a running step is a decision the caller makes on the command line, not one the plan grants itself.

my $header = $store->header;   # { id => ..., created => ..., limits => ... }

The chain header, or {} when no chain is written. limits comes back exactly as it was handed in.

steps

my @steps = $store->steps;

Every step ref that exists, oldest chain generation included, sorted by id (numeric ids numerically, before named ones). Deliberately unfiltered so a half-written or superseded chain can still be looked at; "ready_steps" is where the header decides what may actually run.

step

my $step = $store->step($id);

One step, or undef when there is no such ref.

update_step

my $new = $store->update_step( $id, sub {
    my ($step) = @_;
    return undef unless ( $step->{state} // 'pending' ) eq 'pending';
    $step->{state} = 'running';
    return $step;
} );

Read-modify-write on one step, compare-and-swap guarded. The callback receives the step as it is on the ref and returns the step to write, or undef to decline. Returns the written step, or undef when the step does not exist or the callback declined.

The guard is what makes this usable from more than one foundation tick: two callers that both read pending do not both write running: the loser's write is refused, the callback is called again with what the winner left behind, and it declines. That is the whole exclusion mechanism for a concurrent runner, and it is the board's own ("retry_contended" in App::karr::Git, "write_ref_cas" in App::karr::Git) rather than a second one.

The step handed to the callback carries the chain it was written for, so a caller that has been away long enough for the planner to replace the chain can see that and decline.

mark_stale

$store->mark_stale( $id, 'ticket 41 is no longer todo' );

Marks a step stale: its precheck no longer holds, so it must not be executed. Records the reason and when. Returns the updated step, or undef when the step is gone or was already stale.

Calling the planner afterwards is the runner's move, not this store's -- what is stored here is the fact, so that the next tick, on another machine, sees the same one.

ready_steps

my @ready = $store->ready_steps;

The steps of the current chain that may run right now: state pending, and every step they need in state done. They may all run at once -- that is what the missing edges mean.

Only steps whose chain matches "header" are considered, so a chain with no header, or the leftovers of one being replaced, is never run.

clear_chain

my $removed = $store->clear_chain;

Removes the header and every step ref, and returns how many refs went. The header goes first, so a reader in between finds a chain that is not ready rather than one that is half there.

parse_precheck

my $p = $store->parse_precheck('ticket_status == todo');
# { fact => 'ticket_status', op => '==', value => 'todo' }

Reads a precheck expression. undef or blank yields undef -- a step without a precheck has nothing to go stale on. Anything else must be <fact> == <value> or <fact> != <value>, with the value optionally quoted; a value that cannot be read raises a user error, which is why "write_chain" parses every precheck before it writes anything.

Which facts exist is not enumerated here on purpose: this store knows the grammar, the runner (#186) knows what it can measure, and a fact this store had to be taught about would put half of one decision in two places.

precheck_holds

my $ok = $store->precheck_holds( $step, { ticket_status => 'todo' } );

True when the step's precheck still holds against the facts it is handed, and true for a step that has none. The caller supplies the facts because measuring them means reading a board, which is execution.

A fact the caller did not supply makes the precheck not hold, whichever operator it uses. There is no reading of != under which "I could not find out" should let a step run: an unanswerable precheck is exactly the case this mechanism exists for, and marking the step stale costs a planning round, while running it costs whatever the step does.

segment_max_bytes

How large a run-log segment may grow before the next entry opens the following one; 8192 by default, the same cap and the same reason as "segment_max_bytes" in App::karr::ActivityLog. Set explicitly only by tests, which have to see a rotation without writing thousands of entries.

keep_days

How many days of run logs "prune_logs" keeps; 14 by default. 0 means no age limit, the same spelling max_turns and max_runtime use for "no limit".

keep_runs

How many run logs "prune_logs" keeps regardless of age, newest first; 500 by default, 0 for no ceiling. This is the one that actually bounds the namespace: a fleet busy enough to matter fills two weeks with more refs than anyone wants to fetch.

auto_prune

Whether opening a new run log prunes the old ones first; true by default. Once per run is cheap (one ref listing) and it is the only moment at which the namespace grows, so retention that hangs off it cannot be forgotten.

new_run_id

my $run = $store->new_run_id;   # "2026-08-17-142530a3f91c"

Mints a run name: the UTC date, then the UTC time and six random hex digits. The date leads so the refs sort chronologically, which is what makes retention a matter of looking at the front of a sorted list.

log_run

$store->log_run( $run, event => 'step', step => 3, detail => 'done' );

Appends one JSON entry to a run's log, timestamping it unless ts is given. Returns 1 when the entry landed and 0 after a warning when it could not: a run log records what already happened, so failing to write it must not take the run down with it -- the same rule "log_entry" in App::karr::ActivityLog follows. A $run that is not a run name is the exception, and raises: that is a caller mistake, not a write that failed.

The append is compare-and-swap guarded against the newest segment, re-resolved on every attempt, so two writers cannot lose each other's entries and a rotation is just another lost race. Opening a run (the first entry) prunes old runs first when "auto_prune" is set.

run_ids

my @runs = $store->run_ids;

Every run that has a log, oldest first -- which is plain lexical order, because the name starts with the date. Segments are folded back into the run they belong to.

run_entries

my @entries = $store->run_entries($run);

The decoded entries of one run, oldest first, read across every segment.

prune_logs

my @gone = $store->prune_logs;                      # the configured policy
my @gone = $store->prune_logs( keep_days => 2 );    # or an explicit one

Drops the run logs the retention policy no longer keeps -- everything older than "keep_days", plus everything past the newest "keep_runs" -- and returns the run names it removed. Every segment of a removed run goes.

Deleting these refs leaves no tombstone: "delete_ref" in App::karr::Git only records those for refs/karr/*, so a pruned run is gone locally and stays on the remote until the sync of this namespace (#190) says otherwise.

SUPPORT

Issues

Please report bugs and feature requests on GitHub at https://github.com/Getty/karr/issues.

IRC

Join #langertha on irc.perl.org or message Getty directly.

CONTRIBUTING

Contributions are welcome! Please fork the repository and submit a pull request.

AUTHOR

Torsten Raudssus <getty@cpan.org>

COPYRIGHT AND LICENSE

This software is Copyright (c) 2026 by Torsten Raudssus <torsten@raudssus.de> https://raudssus.de/.

This is free software, licensed under:

The Artistic License 2.0 (GPL Compatible)