NAME
DBIx::Loop - non-blocking DBI on your event loop
VERSION
Version 0.06
SYNOPSIS
use DBIx::Loop;
use DBIx::Loop::Loop::IOAsync;
my $adapter = DBIx::Loop::Loop::IOAsync->new;
my $db = DBIx::Loop->connect(
'dbi:SQLite:dbname=app.db', '', '',
{ RaiseError => 1 },
loop => $adapter, workers => 4,
);
# non-blocking: the query runs on a pool worker (or, for Pg, on the
# connection's own socket) while the loop keeps serving
$db->query("SELECT * FROM pets WHERE id = ?", $id)->on_ready(sub {
my $res = shift->get; # { rows => [[...],...], columns => [...] }
...
});
# transactions, pinned to one connection
$db->txn(sub {
my ($tx) = @_;
$tx->do("INSERT INTO pets (name) VALUES (?)", 'rex')
->then(sub { $tx->do("UPDATE counts SET pets = pets + 1") });
})->on_ready(sub { ... });DESCRIPTION
DBIx::Loop runs DBI queries without blocking an event loop, behind one future-returning API. It is about concurrency and latency isolation, not per-query speed. DBD::SQLite and DBD::Pg already sit at parity with their C libraries (a direct-libsqlite3 comparison showed a dead heat), but a blocking query inside an event-driven server stalls every connection on that worker. DBIx::Loop keeps the loop live.
DBIx::Loop is not an event loop and ships none - you always supply a loop adapter (IO::Async, Mojo::IOLoop, AnyEvent, Hyperman, ...). The engine - object, capability probe, both backends, transactions, and DBIx::Loop::Future - is C.
THE TWO BACKENDS
DBI has no standard async API, so DBIx::Loop carries two backends behind the one interface and picks per driver at connect time (see "capability"):
The worker pool (universal). For drivers with no async surface - SQLite is the extreme: in-process, synchronous, un-yieldable - the blocking call runs on a forked worker holding its own connection, framed over a socketpair the loop watches. Works with every DBD. Workers use
prepare_cached, crashed workers respawn (a storm cap applies), andmax_queuebounds backpressure.Native fd async (Pg; opt-in fast path). DBD::Pg exposes a non-blocking execute and the connection's socket, so queries fire with
pg_async, the loop watchespg_socket, and results collect on readiness - no workers, no serialization. One query per connection is in flight (a libpq limit); the rest queue.
Either way the result is the same shape: query resolves to { rows => [ [...], ... ], columns => [ ... ] } (arrayref rows - they benchmarked ~4x faster to build than hashrefs) and do to { rows_affected => $n, insert_id => $id } (insert_id best-effort via last_insert_id).
A note on SQLite and more than one worker: each worker is another process with its own connection, and SQLite takes one writer at a time for the whole file. Reads run concurrently; writes serialise, and a writer that keeps losing the race gets database is locked back once DBD::SQLite's busy timeout (30 seconds by default) is spent - which on a loaded machine a deep burst of concurrent writes can genuinely do. For write-heavy SQLite either use workers => 1, which costs no responsiveness (the queue still keeps the loop free) and removes the contention outright, or raise sqlite_busy_timeout. Client/server engines have no such limit.
CONSTRUCTORS
connect
my $db = DBIx::Loop->connect($dsn, $user, $pass, \%attr,
loop => $adapter, # required; or 'auto'
workers => 4, # pool backend: worker count
max_queue => 0, # pool backend: pending cap (0 = unbounded)
);Connects via DBI->connect and keeps the connect arguments so pool workers can open their own handles (a live handle cannot cross a fork). loop => 'auto' adapts an already-loaded loop (Mojo::IOLoop, then IO::Async, then AnyEvent) and croaks when none is loaded - there is no built-in loop, by design.
new
my $db = DBIx::Loop->new(dbh => $dbh, loop => $adapter);Wrap an existing handle. Without connect arguments the pool backend cannot fork workers, so queries on a bare wrapped handle run synchronously; use connect for the non-blocking pool.
METHODS
query / do
my $future = $db->query($sql, @bind); # SELECT-style: rows + columns
my $future = $db->do($sql, @bind); # writes: rows_affected, insert_idBoth return a future immediately. ->on_ready, ->then, ->else chain; ->get returns the result once ready (awaiting a pending future is the adapter's job: $adapter->await($future)). Failures carry the DBI error string.
txn
my $future = $db->txn(sub {
my ($tx) = @_;
# $tx->query / $tx->do are pinned to ONE connection
return $tx->do(...)->then(sub { $tx->do(...) });
});Acquires a pool slot (waiting when all are busy), runs BEGIN, calls the block with a DBIx::Loop::Txn handle pinned to that connection, then COMMITs - or ROLLBACKs when the block dies or its returned future fails, failing the outer future with the original error. The block may return a plain value or a future; the outer future resolves to it after commit.
Plain $db statements during a transaction run on other slots - they never join the transaction and never steal its connection. A txn inside a block is an independent transaction on another slot (beware awaiting one while its parents hold every slot). If a worker dies mid-transaction the transaction fails - it is never silently resumed on the respawned connection. Pool backend only for now; native (Pg) transactions arrive with the native connection pool.
The DBI select family
The familiar DBI conveniences exist as async counterparts - same names, same result shapes, wrapped in a future:
$db->selectall_arrayref($sql, @bind) # -> [ [...], [...] ]
$db->selectrow_arrayref($sql, @bind) # -> [...] or undef
$db->selectrow_array($sql, @bind) # -> (list of first-row values)
$db->selectrow_hashref($sql, @bind) # -> { col => val } or undef
$db->selectcol_arrayref($sql, @bind) # -> [ first column... ]
$db->selectall_hashref($sql, $key, @bind) # -> { key_val => {row} }
$db->selectall_rowhash($sql, @bind) # -> [ { col => val }, ... ]selectall_rowhash is DBI's selectall_arrayref($sql, { Slice => {} }): every row as a hashref, in the order the server returned them. selectall_hashref cannot stand in for it - keying rows by a column destroys the ordering, which is exactly what keyset pagination depends on.
Unlike DBI these take @bind directly (no \%attr slot). There is deliberately no prepare/execute/fetchrow_* statement-handle surface: query/do subsume prepare+execute+fetch in one round trip, pool workers already reuse statements via prepare_cached, and row-at-a-time fetching across an async boundary would cost a round trip per row.
capability
'native' when the driver has a usable async surface (DBD::Pg with its async constants loaded), else 'pool'. Probed per handle at construction.
loop / dbh / disconnect
Accessors, and teardown: disconnect reaps pool workers (failing any in-flight or queued futures with a clear message) and closes the handle.
LOOP ADAPTERS
An adapter is five methods - add_reader($fd,$cb), add_writer($fd,$cb), remove($fd), timer($after,$cb), new_future() - plus await and, on loops with a native future type, to_native to bridge a query future into that ecosystem (IO::Async Future, Mojo::Promise, AnyEvent condvar). One conformance suite (t/lib/AdapterConformance.pm) proves every adapter behaves identically. Adapters with a C-side loop (Hyperman) can install a C vtable so readiness dispatches with no Perl call frame.
OBSERVING STATEMENTS
DBIx::Loop->on_exec(\&start, \&done)
Watch every statement this process runs, on all three backends.
DBIx::Loop->on_exec(
sub {
my ($is_query, $sql, $nbind) = @_;
return { at => Time::HiRes::time(), sql => $sql }; # the token
},
sub {
my ($token, $res, $err) = @_;
my $ms = (Time::HiRes::time() - $token->{at}) * 1000;
warn "slow query (${ms}ms): $token->{sql}" if $ms > 100;
},
);start fires before a statement runs; whatever it returns is the token, handed back to done when that statement's future settles, so the two halves correlate without a lookup table. done fires exactly once for every start, with exactly one of $res and $err. done is optional.
This is the only hook that sees everything. The native backend speaks the database's wire protocol and never builds a DBI statement handle, so $dbh->{Callbacks}, DBI::Profile and DBI->trace observe nothing at all on the fast path. on_exec hangs off dbil_exec, the one place every backend passes through.
What it is not given
The bind values. $sql is the prepared statement, and $nbind is how many values there were - and that is deliberate: the values are the literal data, the names and tokens and card numbers, while the prepared text carries placeholders exactly where they would have been. That is the right thing to record anyway, and making it the only thing on offer means nobody has to remember the rule at three in the morning.
Three things to know
Registration is process-global and permanent. Not per handle. There is no deregistration, so register at boot - and note that the callbacks, and everything they close over, live as long as the process does.
An observer does not observe itself. The obvious thing to build here is a query log, and the obvious place to write one is a database - which would run a statement from inside the observer for a statement, for ever. Any statement issued from inside a callback is therefore skipped by the callbacks, both halves, and runs normally otherwise.
A death is warned, not propagated. An observer is a bystander, and a broken bystander must not fail the statement it was watching. If either callback dies, the death becomes a warning naming which half it came from, and the statement carries on.
Returns 1, or 0 when the observer table is full (DBIL_ABI_MAX_OBSERVERS, 8). Croaks if the callbacks are not code references, at registration, where the mistake is.
This costs one Perl call per statement, paid only by a process that asked for it: a statement with no observer registered allocates nothing. Where that is too much, the same registry takes C function pointers through the C ABI's on_exec below - the same contract with no Perl on the path.
C ABI
An XS module can run statements and consume the resulting futures without a Perl frame in between. include/dbil_abi.h declares a versioned function-pointer table:
connect-DBI->connectplus the constructor, as one call. ReturnsNULLand sets an error SV rather than croaking, so a consumer can fall back instead of dying at boot.hyperman_adapter- the Hyperman loop adapter, built on a loop the caller names. Always name the loop: an adapter left to choose for itself when no loop is running constructs one that nothing will ever run, and its futures never settle.exec-query/dowith the binds already in an AV.exec_shaped-execplus oneselect*reshape as a single call, so the intermediate future of query-then-reshape is never built.is_future/future_state/future_values/future_error- settle-path reads.is_futureanswers for any SV at all, returning false for anything that is not one rather than dereferencing whatever it was handed, so it is safe to probe with.future_valueswrites borrowed SVs into an array the caller supplies; a stack array is the expected use.future_on_ready- a C continuation: no closure is compiled and no Perl frame runs when the result lands. This is the entry that matters most.future_new/future_done1/future_fail- for bridging into another future type.reshape- theselect*transforms over a result you already hold. Returns an error SV instead of croaking, because on the chaining path it runs inside a settle and must produce a failed future, not a die.on_exec(start, done, ud)(v2) - observe statements.static void *start(pTHX_ int is_query, const char *sql, STRLEN len, int nbind, void *ud); static void done(pTHX_ void *token, SV *res, SV *err, void *ud);startfires before a statement runs and returns an opaque token;donefires exactly once when that statement's future settles, with the token back, so the two correlate without a lookup table. It covers all three backends, because it hangs offexecand off the futureexecreturns.sqlis the prepared statement andnbindthe number of bind values. The bind values are deliberately not passed: they are the literal data, and the prepared text carries placeholders exactly where they would have been.Registration is process-global and there is no deregistration; register at boot. Neither callback may croak. Returns 1, or 0 when the table is full. A statement with no observer registered allocates nothing.
"DBIx::Loop->on_exec(\&start, \&done)" is the same registry reached from Perl, for a consumer that is not an XS module: it registers a pair of shims holding coderefs. Same contract, one Perl call per statement instead of none, and the two rules C can state but Perl cannot keep - "do not croak", and "do not run a statement from in here" - become "a death is warned and the statement carries on" and "a statement issued from inside a callback is not observed".
The table is resolved at runtime through DBIx::Loop::_abi_ptr and gated on its abi_version, so there is no link-time coupling and the two distributions upgrade independently; entries are only ever appended. Reach the header with ExtUtils::Depends:
my $pkg = ExtUtils::Depends->new('My::Module', 'DBIx::Loop');Two things about the table are deliberate. Nothing in it allocates a block the caller must free - pairing a malloc in one shared object with a free in another is a good way to discover that each can carry its own heap. And there is no destructor for a connection or a future: both are Perl objects that already own their lifetime correctly, so hold a reference and let refcounting do it. What the table hands back +1, you SvREFCNT_dec.
AUTHOR
LNATION <email@lnation.org>
LICENSE AND COPYRIGHT
This software is Copyright (c) 2026 by LNATION <email@lnation.org>.
This is free software, licensed under:
The Artistic License 2.0 (GPL Compatible)