/ 
Acme-Parataxis-v0.1.1
/ Acme::Parataxis::Generator

NAME

Acme::Parataxis::Generator - Stackful, Lazily-pulled Iterator

SYNOPSIS

use Acme::Parataxis::Generator;

my $fib = Acme::Parataxis::Generator->new( sub ($y) {
    my ( $a, $b ) = ( 0, 1 );
    while ( $a < 100 ) {
        $y->($a);
        ( $a, $b ) = ( $b, $a + $b );
    }
} );

my $total = 0;
while ( defined( my $n = $fib->next ) ) { $total += $n }
say $total;                          # 232

DESCRIPTION

A generator runs its body in a private fiber and pulls it one value at a time. The body is ordinary synchronous code: it can loop, return, call other subroutines and recurse deeply, and $y->(...) works from any depth. Each $y->($value) suspends the body exactly where it stands - every lexical, every loop counter, the whole call stack is still there - and the next ->next picks it up from that same point.

Nothing in the body runs until the first next, so a generator over an expensive source costs nothing until you start pulling, and an infinite sequence is fine as long as you stop pulling.

Two things the body must not do: park (see "Parking inside a body injects a bogus value"), and pull the generator it is itself running in (see "Re-entrant next ends the generator"). Everything else is plain Perl.

You can pull from the mainline, from a run's main fiber, from a fiber you spawned, or from a nursery child, in any order. The first generator a process builds may be created wherever you like; nothing needs to be primed first.

CONSTRUCTOR

Generator->new( sub ($y) { ... } )

my $gen = Acme::Parataxis::Generator->new( sub ($y) {
    $y->($_) for 1 .. 3;
} );

is $gen->next, 1;
is $gen->next, 2;
is $gen->next, 3;
ok !defined $gen->next;

Returns a generator. $y is the yield function: call $y->($value) to produce the next value and suspend until the next pull. It takes exactly one value, so $y->() and $y->( 1, 2 ) are both croaks - reported against __ANON__, since $y is a closure and has no name of its own. Whatever the body returns is discarded. Exactly one argument is accepted by new itself - see "The constructor is strict".

METHODS

next( )

if ( defined( my $v = $gen->next ) ) { handle($v) }
else                               { last }

Resumes the body and returns the value it yielded. Once the body has finished, returns undef and keeps returning it.

A undef the body itself yielded comes back exactly the same way, so is_done is the only way to tell a real value from the end of the sequence.

If the body threw, the exception is raised on your stack at the next that provoked it, and the generator is finished afterwards. An exception object arrives as that same object, not as a string - which is not what Acme::Parataxis::Future does, since that croaks the message.

In list context this returns a one-element list.

is_done( )

while ( !$gen->is_done ) { handle( $gen->next ) }

Returns 1 once the body has finished, whether it returned or threw, and 0 before that. This is the reliable way to drain a sequence that can contain undef.

Recipes

Drain a generator without stopping at a yielded undef

sub drain ($gen) {
    my @out;
    while (1) {
        my $v = $gen->next;
        last if $gen->is_done;              # the pull that finished the body, not a value
        push @out, $v;                      # a yielded undef is a value, and is kept
    }
    return @out;
}

my $gen = Acme::Parataxis::Generator->new( sub ($y) { $y->($_) for undef, 'after' } );
my @got = drain($gen);                      # ( undef, 'after' )

The while ( defined( my $v = $gen-next ) )> form used in "SYNOPSIS" stops at the first undef, so a sequence that contains one silently truncates.

Take only the first few values

sub take ($gen, $n) {
    my @out;
    while ( @out < $n ) {
        my $v = $gen->next;
        last if $gen->is_done;
        push @out, $v;
    }
    return @out;
}

my $gen = Acme::Parataxis::Generator->new( sub ($y) { $y->($_) for 1 .. 1_000 } );
say join ' ', take( $gen, 3 );               # 1 2 3

The body is never resumed again, so the remaining 997 values are never computed and the abandoned generator is drained when $gen goes out of scope.

Build a lazy pipeline

my $squares = Acme::Parataxis::Generator->new( sub ($y) {
    my $nums = Acme::Parataxis::Generator->new( sub ($z) { $z->($_) for 1 .. 10 } );
    while ( defined( my $n = $nums->next ) ) { $y->( $n**2 ) }
} );

say $squares->next for 1 .. 3;              # 1 4 9

$nums is a lexical in the outer body's frame, so it survives across every suspension. The inner body is entered once and then suspended: after three pulls its loop has advanced as far as 3 and no further, and it is not created at all until the first pull.

Handle a failure without losing the error object

my $gen = Acme::Parataxis::Generator->new( sub ($y) {
    $y->($_) for 1 .. 3;
    die Acme::Parataxis::Error::Timeout->new( wait_reason => [ 'fetch', __FILE__, __LINE__ ] );
} );

while (1) {
    my $v = eval { $gen->next };
    if ( my $err = $@ ) {
        say 'failed: ' . $err->kind;        # timeout - the object crossed the resume intact
        last;
    }
    last if $gen->is_done;
    say "got $v";
}

# got 1
# got 2
# got 3
# failed: timeout

Behavior Notes

A yielded undef is the end of the sequence as far as next is concerned

next cannot tell a body that yielded undef from a body that has finished; both return undef. The usual while ( defined( my $v = $gen-next ) )> loop therefore stops at the first undef the body produces:

my $gen = Acme::Parataxis::Generator->new( sub ($y) { $y->($_) for undef, 'after' } );
my @got;
while ( defined( my $v = $gen->next ) ) { push @got, $v }
say scalar @got;                           # 0 - and 'after' never arrives

Test for the end with is_done instead, as "Drain a generator without stopping at a yielded undef" does.

The body's return value is discarded

A generator's last value always comes from a $y->(...) call. If the body returns without yielding again, the next next returns undef and the generator is finished, whatever the body returned:

my $gen = Acme::Parataxis::Generator->new( sub ($y) { $y->(1); return 'final' } );
say $gen->next;                            # 1
say defined( $gen->next ) ? 'a value' : 'undef';   # undef

Use a trailing $y->($value) if the final value matters.

Parking inside a body injects a bogus value

The body runs in a fiber but never enters the scheduler's run queue, so a park inside it does not behave. Every park adds the scheduler's own resume token to the sequence as if the body had yielded it:

my $gen = Acme::Parataxis::Generator->new( sub ($y) {
    $y->('before');
    await_sleep 20;
    $y->('after');
} );

say $gen->next while !$gen->is_done;        # before WAITING after

await_sleep, await_read, await_write and anything else that parks all do this. Keep the body synchronous and pull from the consumer side instead, where parking is a normal thing to do.

An abandoned generator is drained only if it is suspended at a yield

Dropping a generator whose body is parked at a $y->(...) call resumes it with a private drain marker, the yield dies on that marker, and the fiber finishes normally. Dropping one whose body is parked somewhere else - inside await_sleep, say - cannot do that, so the fiber is torn down and everything after the park is skipped:

my $reached = 'never';
{
    my $gen = Acme::Parataxis::Generator->new( sub ($y) {
        $y->('x');
        await_sleep 5_000;
        $reached = 'ran';                   # never reached: the body is torn down, not resumed
    } );
    $gen->next;
}
say $reached;                              # never

The drain marker is a private scalar ref rather than a sentinel string, so no value a body yields can be mistaken for it and drained silently.

The constructor is strict

new takes exactly one CODE ref. Anything else is a croak, and the message names the actual problem:

Acme::Parataxis::Generator->new;                 # Too few arguments ... (got 1; expected 2)
Acme::Parataxis::Generator->new('nope');         # Generator->new() requires a CODE ref
Acme::Parataxis::Generator->new( sub { }, 1 );   # Too many arguments ... (got 3; expected 2)

Note that the extra-argument case is rejected even though a plain-looking new( sub { ... }, @whatever ) would be a reasonable thing to want.

One fiber stays parked, but only for a generator built on the mainline

Resuming a die path through the very first fiber a process allocates is unreliable on Windows, where it escapes every anchored error trap, so a generator must never land on fid 0. On the mainline nothing else has allocated a fiber yet, so the first generator there parks one reserved fiber to take fid 0, and get_live_fiber_count() goes from 0 to 1 and stays there:

Acme::Parataxis::get_live_fiber_count();         # 0 in a fresh process
my $gen = Acme::Parataxis::Generator->new( sub ($y) { $y->(1) } );
Acme::Parataxis::get_live_fiber_count();         # 1

Inside a run there is nothing to reserve. The run's own main fiber already holds fid 0 before your block executes, so a generator built there lands on fid 1 or later. It adds only its own fiber while it is alive: once it is exhausted or dropped, get_live_fiber_count() is back to whatever it was, with nothing parked for the lifetime of the process.

Re-entrant next ends the generator

Pulling a generator from inside its own body re-enters a fiber that is already running. It does not raise, and it does not return the value you want - the generator simply finishes:

my $gen;
$gen = Acme::Parataxis::Generator->new( sub ($y) {
    $y->('one');
    my $v = $gen->next;                 # re-entrant: ends the generator
    $y->("re-entered with " . $v );
} );

say $gen->next;                          # one
say defined( $gen->next ) ? 'a value' : 'undef';   # undef
say $gen->is_done;                       # 1

If a body needs to consume another sequence, give the two generators separate frames and let the consumer pull from outside, as in "Build a lazy pipeline".

How It Works

  • Asymmetric pull. Each next resumes the private fiber with a fresh scheduler call (the runtime's coro_call), and each yield parks it back. Exhaustion finishes the fiber through the normal scheduler teardown.

  • Errors cross the resume. The body runs inside a fiber-local eval; a die raised on a resumed trip is caught there (the scheduler's own error trap can miss a die that fires across a resumption) and parked in a lexical that next dereferences and rethrows, so no die ever escapes across a resumption boundary. Because the error is rethrown rather than interpolated, an exception object arrives as itself.

  • Abandoned generators drain. Discarding a generator whose fiber is suspended mid-yield resumes it with a drain marker; the yield function dies on it, the fiber stops and finishes normally, and perl unwinds its own scopes. This is deliberately preferred over tearing a suspended, eval-bearing fiber down mid-shot. See "An abandoned generator is drained only if it is suspended at a yield" for the one case this does not cover.

  • Reserved first fiber. On Windows, resuming a die path through the very first fiber a process ever allocates can defeat every anchored error trap, so the module keeps one permanently parked fiber and generators always start on a later slot. A run's own main fiber already holds that slot, so the reserved fiber is only created for a generator built on the mainline.

SEE ALSO

Acme::Parataxis, Acme::Parataxis::Future, Acme::Parataxis::Channel

AUTHOR

Sanko Robinson https://github.com/sanko

LICENSE

Copyright (C) Sanko Robinson.

This library is free software; you can redistribute it and/or modify it under the terms found in the Artistic License 2.