NAME

App::FileSync::Protocol - the line-delimited JSON wire protocol

SYNOPSIS

my $connection = App::FileSync::Protocol->new(in => $reader, out => $writer);
$connection->send({ cmd => 'hello', proto => 2 });
my $reply = $connection->recv();
$connection->send_data({ cmd => 'chunk', p => $path }, $bytes);

DESCRIPTION

Each control message is one JSON object on one line. A message announcing a payload carries len (the bytes on the wire) and, when the payload was deflated, z; the payload follows the line directly. The connection counts payload bytes in each direction for the transfer report.

When reading or writing fails because the pipe itself failed (the peer exited, ssh was refused), "transport_failed" becomes true, so a caller can show the transport's own diagnostics instead of a protocol error.

METHODS

new

Purpose: wrap a pair of handles as a protocol connection.

Args: in and out (handles), optional language.

Returns: the connection.

Side Effects: puts both handles in binary mode and makes out autoflush.

Usage:

my $connection = App::FileSync::Protocol->new(in => \*STDIN, out => \*STDOUT);

EXAMPLE

my $connection = App::FileSync::Protocol->new(in => $transport->reader, out => $transport->writer);

API SPECIFICATION

INPUT

{
	in       => { type => 'handle' },
	out      => { type => 'handle' },
	language => { type => 'string', optional => 1 },
}

OUTPUT

{ type => 'object', isa => 'App::FileSync::Protocol' }

MESSAGES

Key            Meaning                       Resolution
args.invalid   a handle is missing           pass in and out

FORMAL SPECIFICATION

new(in, out) = ⟨in, out, buffer ↦ ⟨⟩, closed ↦ false, sent ↦ 0, received ↦ 0⟩

send

Purpose: send one control message.

Args: \%message.

Returns: 1.

Side Effects: writes one line.

Usage:

$connection->send({ cmd => 'plan' });

EXAMPLE

$connection->send({ ok => 1, off => $offset });

API SPECIFICATION

INPUT

[
	{ name => 'self',    type => 'object' },
	{ name => 'message', type => 'hashref' },
]

OUTPUT

{ type => 'integer', min => 1, max => 1 }

MESSAGES

Key                       Meaning                          Resolution
protocol.closed           the connection was closed        a caller bug
protocol.write_failed     the pipe would not take it       see the transport's stderr

FORMAL SPECIFICATION

pre ¬closed
out' = out ⁀ json(message) ⁀ "\n"

recv

Purpose: read one control message.

Args: none.

Returns: the decoded message hashref.

Side Effects: reads from in; blocks until a whole line arrives.

Usage:

my $reply = $connection->recv();

EXAMPLE

my $hello = $connection->recv();
die unless $hello->{ok};

API SPECIFICATION

INPUT

[ { name => 'self', type => 'object' } ]

OUTPUT

{ type => 'hashref' }

MESSAGES

Key                   Meaning                        Resolution
protocol.malformed    the line is not JSON           version mismatch or corrupt stream
protocol.not_object   the JSON is not an object      version mismatch
protocol.eof          the peer closed the connection see the peer's log
protocol.read_failed  the pipe failed                see the transport's stderr

FORMAL SPECIFICATION

in = line ⁀ "\n" ⁀ rest ⇒ recv = json⁻¹(line) ∧ in' = rest
pre json⁻¹(line) ∈ OBJECT

send_bytes

Purpose: write raw bytes (the payload after a header).

Args: $bytes; empty or undef writes nothing.

Returns: 1.

Side Effects: writes to out. Not counted in "bytes_sent"; use "send_data" for counted payloads.

Usage:

$connection->send_bytes($payload);

EXAMPLE

$connection->send({ len => length $bytes });
$connection->send_bytes($bytes);

API SPECIFICATION

INPUT

[
	{ name => 'self',  type => 'object' },
	{ name => 'bytes', type => 'string', optional => 1 },
]

OUTPUT

{ type => 'integer', min => 1, max => 1 }

MESSAGES

Key                       Meaning                          Resolution
protocol.closed           the connection was closed        a caller bug
protocol.write_failed     the pipe would not take it       see the transport's stderr

FORMAL SPECIFICATION

out' = out ⁀ bytes

send_data

Purpose: send a header and its payload, compressing when worthwhile.

Args: \%header, $bytes.

Returns: 1.

Side Effects: writes the header (with len and, if compressed, z) and the payload; adds the wire length to "bytes_sent".

Usage:

$connection->send_data({ cmd => 'chunk', p => $path }, $bytes);

EXAMPLE

$connection->send_data({ ok => 1, eof => 1 }, $last_chunk);

API SPECIFICATION

INPUT

[
	{ name => 'self',    type => 'object' },
	{ name => 'message', type => 'hashref', optional => 1 },
	{ name => 'bytes',   type => 'string', optional => 1 },
]

OUTPUT

{ type => 'integer', min => 1, max => 1 }

MESSAGES

As for "send".

FORMAL SPECIFICATION

let (w, z) = deflate(bytes)
out' = out ⁀ json(header ⊕ {len ↦ #w} ⊕ (z ⇒ {z ↦ 1})) ⁀ "\n" ⁀ w
sent' = sent + #w

read_data

Purpose: read the payload a header announced.

Args: \%header as returned by "recv".

Returns: the payload, decompressed if it was sent compressed.

Side Effects: reads len bytes; adds them to "bytes_received".

Usage:

my $bytes = $connection->read_data($header);

EXAMPLE

my $reply = $connection->recv();
my $bytes = $connection->read_data($reply);

API SPECIFICATION

INPUT

[
	{ name => 'self',    type => 'object' },
	{ name => 'message', type => 'hashref' },
]

OUTPUT

{ type => 'string' }

MESSAGES

Key                      Meaning                                 Resolution
protocol.bad_length      len is not a number or exceeds 64 MiB   a peer bug
protocol.inflate         the payload cannot be decompressed      a corrupt stream
protocol.corrupt         the compressed payload is corrupt       a corrupt stream
protocol.too_large       it would inflate beyond 64 MiB          a hostile or buggy peer

FORMAL SPECIFICATION

pre header.len ∈ 0 .. MAX_WIRE
read_data(h) = inflate(take(h.len, in))   if h.z
             = take(h.len, in)            otherwise
received' = received + h.len

read_line

Documented with "read_bytes" below.

read_bytes

Purpose: read up to the next newline, or exactly $length bytes.

Args: read_bytes($length).

Returns: the line (without its newline) or the bytes.

Side Effects: consumes them from the buffer, reading from in as needed.

Usage:

my $line = $connection->read_line();
my $bytes = $connection->read_bytes(1024);

EXAMPLE

my $payload = $connection->read_bytes($header->{len});

API SPECIFICATION

INPUT

[
	{ name => 'self',   type => 'object' },
	{ name => 'length', type => 'integer', min => 0 },
]

OUTPUT

{ type => 'string' }

MESSAGES

Key                       Meaning                          Resolution
protocol.negative         a negative length was asked for  a caller bug
protocol.eof              the peer closed the connection   see the peer's log
protocol.read_failed      the pipe failed                  see the transport's stderr

FORMAL SPECIFICATION

read_line: in = l ⁀ "\n" ⁀ r ∧ "\n" ¬in; ran l ⇒ result = l ∧ in' = r
read_bytes(n): in = b ⁀ r ∧ #b = n ⇒ result = b ∧ in' = r

bytes_sent

Documented with "bytes_received" below.

bytes_received

Purpose: payload bytes that crossed the wire in each direction.

Args: none.

Returns: a byte count (headers and control messages excluded).

Side Effects: none.

Usage:

printf "sent %d bytes\n", $connection->bytes_sent;

EXAMPLE

my %summary = (bytes_sent => $connection->bytes_sent, bytes_received => $connection->bytes_received);

API SPECIFICATION

INPUT

[ { name => 'self', type => 'object' } ]

OUTPUT

{ type => 'integer', min => 0 }

MESSAGES

None.

FORMAL SPECIFICATION

bytes_sent = sent ; bytes_received = received

transport_failed

Purpose: whether the last failure was the pipe rather than the protocol.

Args: none.

Returns: 1 after a read or write on the underlying handles failed, or the peer closed the connection; otherwise 0.

Side Effects: none.

Usage:

my $detail = $connection->transport_failed ? $transport->error_text : undef;

EXAMPLE

eval { $connection->recv() } or print "pipe failed\n" if $connection->transport_failed;

API SPECIFICATION

INPUT

[ { name => 'self', type => 'object' } ]

OUTPUT

{ type => 'integer', min => 0, max => 1 }

MESSAGES

None.

FORMAL SPECIFICATION

transport_failed ⇔ ∃ an I/O failure or end of file on in or out

close

Purpose: close both handles.

Args: none.

Returns: 1, or nothing if already closed.

Side Effects: closes in and out; later sends raise protocol.closed.

Usage:

$connection->close;

EXAMPLE

$connection->send({ cmd => 'done' });
$connection->close;

API SPECIFICATION

INPUT

[ { name => 'self', type => 'object' } ]

OUTPUT

{ type => 'integer', optional => 1 }

MESSAGES

None; close errors are ignored, as the session is over.

FORMAL SPECIFICATION

closed' = true

LIMITATIONS

There is no cap on the length of a control line, because the plan for a large tree is one line; see "LIMITATIONS" in App::FileSync.
The size check on a compressed payload happens after inflating it, so a hostile peer can make the receiver allocate a large buffer once. The peer is authenticated by ssh before it can send anything.
The method names send, recv and close shadow Perl builtins; call them as methods only.

AUTHOR

Nigel Horne, <njh at nigelhorne.com>

LICENCE AND COPYRIGHT

Usage is subject to the GPL2 licence terms. If you use it, please let me know.

To install App::FileSync, copy and paste the appropriate command in to your terminal.

cpanm

cpanm App::FileSync

CPAN shell

perl -MCPAN -e shell
install App::FileSync

For more information on module installation, please visit the detailed CPAN module installation guide.

	Global
`s`	Focus search bar
`?`	Bring up this help dialog

	GitHub
`g` `p`	Go to pull requests
`g` `i`	Go to GitHub issues (only if GitHub is preferred repository)

	POD
`g` `a`	Go to author
`g` `c`	Go to changes
`g` `i`	Go to issues
`g` `d`	Go to dist
`g` `r`	Go to repository/SCM
`g` `s`	Go to source
`g` `b`	Go to file browse

Search terms
module: (e.g. module:Plugin)
distribution: (e.g. distribution:Dancer auth)
author: (e.g. author:SONGMU Redis)
version: (e.g. version:1.00)

NAME

SYNOPSIS

DESCRIPTION

METHODS

new

EXAMPLE

API SPECIFICATION

INPUT

OUTPUT

MESSAGES

FORMAL SPECIFICATION

send

EXAMPLE

API SPECIFICATION

INPUT

OUTPUT

MESSAGES

FORMAL SPECIFICATION

recv

EXAMPLE

API SPECIFICATION

INPUT

OUTPUT

MESSAGES

FORMAL SPECIFICATION

send_bytes

EXAMPLE

API SPECIFICATION

INPUT

OUTPUT

MESSAGES

FORMAL SPECIFICATION

send_data

EXAMPLE

API SPECIFICATION

INPUT

OUTPUT

MESSAGES

FORMAL SPECIFICATION

read_data

EXAMPLE

API SPECIFICATION

INPUT

OUTPUT

MESSAGES

FORMAL SPECIFICATION

read_line

read_bytes

EXAMPLE

API SPECIFICATION

INPUT

OUTPUT

MESSAGES

FORMAL SPECIFICATION

bytes_sent

bytes_received

EXAMPLE

API SPECIFICATION

INPUT

OUTPUT

MESSAGES

FORMAL SPECIFICATION

transport_failed

EXAMPLE

API SPECIFICATION

INPUT

OUTPUT

MESSAGES

FORMAL SPECIFICATION

close

EXAMPLE

API SPECIFICATION

INPUT

OUTPUT

MESSAGES

FORMAL SPECIFICATION

LIMITATIONS

AUTHOR

LICENCE AND COPYRIGHT

Module Install Instructions