NAME

App::FileSync::Store - the server-side files of one repo

SYNOPSIS

my $store = App::FileSync::Store->new(root => '/srv/filesync', repo => 'docs');
my $lock  = $store->acquire_lock() or die 'busy';
my $index = $store->load_index();
my $scan  = $store->walk_master($index);
$store->save_index($scan->{manifest});

METHODS

new

Purpose: open a repo on the server.

Args: root, repo, optional config (the repo config, read from disk when omitted) and language.

Returns: the store.

Side Effects: reads the repo config.

Usage:

my $store = App::FileSync::Store->new(root => $root, repo => $repo);

EXAMPLE

my $store = App::FileSync::Store->new(root => '/srv/filesync', repo => 'docs');
print $store->master_dir, "\n";

API SPECIFICATION

INPUT

{
	root     => { type => 'string', min => 1 },
	repo     => { type => 'string', min => 1 },
	config   => { type => 'hashref', optional => 1 },
	language => { type => 'string', optional => 1 },
}

OUTPUT

{ type => 'object', isa => 'App::FileSync::Store' }

MESSAGES

Key              Meaning                       Resolution
args.invalid     root or repo missing          pass both
store.bad_repo   the repo name is not an id    use letters, digits, '.', '_', '-'
config.no_repo   the repo does not exist       filesync-admin repo create

FORMAL SPECIFICATION

new(r, n, c) = ⟨root ↦ r, repo ↦ n, config ↦ c or load_repo(r, n), ignore ↦ Ignore(config.ignore)⟩

root

Documented with "journal_path" below.

repo

Documented with "journal_path" below.

config

Documented with "journal_path" below.

master_dir

Documented with "journal_path" below.

meta_dir

Documented with "journal_path" below.

index_path

Documented with "journal_path" below.

trash_dir

Documented with "journal_path" below.

lock_path

Documented with "journal_path" below.

journal_path

Purpose: where the repo's pieces live.

Args: none.

Returns: the server root, the repo name, the config hashref, and the paths of the master tree (ROOT/master/REPO), the metadata directory, the index (master.json), the trash, the lock file and this month's journal (journal/YYYY-MM.jsonl, UTC).

Side Effects: none.

Usage:

my $master = $store->master_dir;

EXAMPLE

print join("\n", $store->master_dir, $store->index_path, $store->journal_path), "\n";

API SPECIFICATION

INPUT

[ { name => 'self', type => 'object' } ]

OUTPUT

{ type => 'string' }

MESSAGES

None.

FORMAL SPECIFICATION

master_dir = root / "master" / repo
meta_dir = root / ".filesync" / "repos" / repo
index_path = meta_dir / "master.json"
journal_path = meta_dir / "journal" / yyyy-mm(now) ⁀ ".jsonl"

base_path

Purpose: where a client's base (its state at its last sync) is stored.

Args: $client, a client id.

Returns: META/clients/CLIENT.json.

Side Effects: none.

Usage:

my $path = $store->base_path('laptop-1');

EXAMPLE

unlink $store->base_path($client) if $forget;

API SPECIFICATION

INPUT

[
	{ name => 'self',   type => 'object' },
	{ name => 'client', type => 'string', min => 1 },
]

OUTPUT

{ type => 'string', min => 1 }

MESSAGES

Key              Meaning                              Resolution
store.bad_client the id is not a valid client id      use letters, digits, . _ -

FORMAL SPECIFICATION

pre client ∈ Id
base_path(client) = meta_dir / "clients" / client ⁀ ".json"

acquire_lock

Purpose: take the repo's session lock without waiting.

Args: none.

Returns: the lock handle, or undef when another session holds it.

Side Effects: creates the lock file.

Usage:

my $lock = $store->acquire_lock() or return refuse('locked');

EXAMPLE

if (my $lock = $store->acquire_lock) { ... }

API SPECIFICATION

INPUT

[ { name => 'self', type => 'object' } ]

OUTPUT

{ type => 'handle', optional => 1 }

MESSAGES

Key         Meaning                         Resolution
io.append   the lock file cannot be made    check permissions

FORMAL SPECIFICATION

acquire_lock = Lock::acquire(lock_path, nonblock)

abs_path

Purpose: the absolute path in the master for a relative path.

Args: $rel.

Returns: the path under "master_dir".

Side Effects: none. Raises if $rel is unsafe or if a directory on the way is a symlink, so nothing can be written outside the master through one.

Usage:

my $abs = $store->abs_path($path);

EXAMPLE

my $abs = $store->abs_path('notes/plan.txt');

API SPECIFICATION

INPUT

[
	{ name => 'self', type => 'object' },
	{ name => 'rel',  type => 'string' },
]

OUTPUT

{ type => 'string', min => 1 }

MESSAGES

Key              Meaning                                  Resolution
path.unsafe      the path fails validate_relpath          rename the file
store.symlink    a parent directory is a symlink          remove the symlink

FORMAL SPECIFICATION

pre rel ∈ SafePath ∧ ∀ a : ancestors(master / rel) ∩ under(master) • kind a ≠ symlink
abs_path(rel) = master / rel

load_index

Documented with "save_base" below.

save_index

Documented with "save_base" below.

load_base

Documented with "save_base" below.

save_base

Purpose: read and write the master's index and a client's base.

Args: save_index(\%entries); load_base($client); save_base($client, \%entries).

Returns: the entries hashref (empty when the file does not exist); 1 for the save methods.

Side Effects: the save methods replace the file atomically.

Usage:

my $index = $store->load_index();
$store->save_base($client, $base_after);

EXAMPLE

my $base = $store->load_base('laptop-1');
print scalar(keys %{$base}), " paths known to laptop-1\n";

API SPECIFICATION

INPUT

[
	{ name => 'self',    type => 'object' },
	{ name => 'client',  type => 'string', optional => 1 },
	{ name => 'entries', type => 'hashref', optional => 1 },
]

OUTPUT

{ type => 'hashref' }

MESSAGES

Key            Meaning                       Resolution
config.parse   the file is not valid JSON    run filesync-admin reindex
io.write       the file cannot be written    check space

FORMAL SPECIFICATION

load(p) = json(FS(p)).entries if p ∈ dom FS ∧ is a hash, else ∅
save(p, e): FS'(p) = json(⟨version ↦ 1, entries ↦ e⟩) ⁀ "\n"

walk_master

Purpose: scan the master, reusing indexed hashes where size and mtime match.

Args: optional \%cache (usually the current index; {} forces a rehash).

Returns: the "scan" in App::FileSync::Scan result: { manifest, stats, skipped }.

Side Effects: reads the master tree.

Usage:

my $scan = $store->walk_master($store->load_index);

EXAMPLE

my $fresh = $store->walk_master({});
printf "%d files\n", $fresh->{stats}{files};

API SPECIFICATION

INPUT

[
	{ name => 'self',  type => 'object' },
	{ name => 'cache', type => 'hashref', optional => 1 },
]

OUTPUT

{ type => 'hashref' }

MESSAGES

Key               Meaning                     Resolution
store.no_master   the master directory is gone   restore it or recreate the repo

FORMAL SPECIFICATION

pre isdir(master_dir)
walk_master(c) = scan(master_dir, ignore, c)

ensure_parent

Purpose: create the directory a master path will be written into.

Args: $rel.

Returns: the parent directory.

Side Effects: creates missing directories.

Usage:

$store->ensure_parent($path);

EXAMPLE

$store->ensure_parent('new/deep/dir/file.txt');

API SPECIFICATION

INPUT

[
	{ name => 'self', type => 'object' },
	{ name => 'rel',  type => 'string', min => 1 },
]

OUTPUT

{ type => 'string', min => 1 }

MESSAGES

As for "abs_path", plus io.mkdir.

FORMAL SPECIFICATION

isdir'(dirname(abs_path(rel)))

temp_file

Purpose: an exclusive temporary file beside a master path.

Args: $rel, the path the temporary file will become.

Returns: [ $filehandle, $path ].

Side Effects: creates the file (and its directory).

Usage:

my ($fh, $temp) = @{ $store->temp_file($path) };

EXAMPLE

my ($fh, $temp) = @{ $store->temp_file('notes/plan.txt') };
print {$fh} $content;
close $fh;
$store->commit_temp('notes/plan.txt', $temp, time, oct(644));

API SPECIFICATION

INPUT

[
	{ name => 'self', type => 'object' },
	{ name => 'rel',  type => 'string', min => 1 },
]

OUTPUT

{ type => 'arrayref', min => 2, max => 2 }

MESSAGES

As for "abs_path", plus io.temp.

FORMAL SPECIFICATION

temp_file(rel) = App::FileSync::IO::temp_file(dirname(abs_path(rel)))

commit_temp

Purpose: move a verified temporary file into the master.

Args: $rel, $temp, optional $mtime and $mode.

Returns: the absolute path written.

Side Effects: see "install_file" in App::FileSync::IO.

Usage:

$store->commit_temp($path, $temp, $mtime, $mode);

EXAMPLE

$store->commit_temp('a.txt', $temp, 1_700_000_000, oct(600));

API SPECIFICATION

INPUT

[
	{ name => 'self',  type => 'object' },
	{ name => 'rel',   type => 'string', min => 1 },
	{ name => 'temp',  type => 'string', min => 1 },
	{ name => 'mtime', type => 'number', optional => 1 },
	{ name => 'mode',  type => 'integer', optional => 1 },
]

OUTPUT

{ type => 'string', min => 1 }

MESSAGES

As for "install_file" in App::FileSync::IO and "abs_path".

FORMAL SPECIFICATION

commit_temp(rel, t, m, mode) = install_file(t, abs_path(rel), m, mode)

move_to_trash

Purpose: move a master path into the repo's trash.

Args: $rel, optional $epoch.

Returns: where it went, or undef if it was not there.

Side Effects: see "move_to_trash" in App::FileSync::Trash.

Usage:

my $trashed = $store->move_to_trash($path, time);

EXAMPLE

my $where = $store->move_to_trash('old/report.txt');

API SPECIFICATION

INPUT

[
	{ name => 'self',  type => 'object' },
	{ name => 'rel',   type => 'string', min => 1 },
	{ name => 'epoch', type => 'number', optional => 1 },
]

OUTPUT

{ type => 'string', optional => 1 }

MESSAGES

As for "move_to_trash" in App::FileSync::Trash and "abs_path".

FORMAL SPECIFICATION

abs_path(rel) ¬in; dom FS ⇒ result = ⊥
abs_path(rel) ∈ dom FS ⇒ result = Trash::move_to_trash(abs_path(rel), rel, trash_dir, epoch)

clean_stale_temps

Documented with "prune_trash" below.

prune_trash

Purpose: housekeeping at the start of a session.

Args: clean_stale_temps(max_age => SECONDS); prune_trash(days => N, dry_run => BOOL). Defaults: one hour, and the repo's trash_days (30).

Returns: the number of temporaries removed; the arrayref of trash buckets removed.

Side Effects: deletes stale temporaries from the master and expired trash.

Usage:

$store->clean_stale_temps();
$store->prune_trash();

EXAMPLE

my $pruned = $store->prune_trash(dry_run => 1);

API SPECIFICATION

INPUT

{
	max_age => { type => 'integer', min => 0, optional => 1 },
	days    => { type => 'number', optional => 1 },
	dry_run => { type => 'boolean', optional => 1 },
}

OUTPUT

{ type => 'scalar' }

MESSAGES

None.

FORMAL SPECIFICATION

clean_stale_temps = Scan::clean_stale_temps(master_dir, age)
prune_trash = Trash::prune(trash_dir, days)

journal

Purpose: append an event to this month's journal.

Args: \%event; a ts (now) is added.

Returns: 1.

Side Effects: appends one JSON line, creating the journal directory.

Usage:

$store->journal({ client => $id, action => 'push', path => $path, result => 'ok' });

EXAMPLE

$store->journal({ action => 'reindex', files => 10, result => 'ok' });

API SPECIFICATION

INPUT

[
	{ name => 'self',  type => 'object' },
	{ name => 'event', type => 'hashref' },
]

OUTPUT

{ type => 'integer', min => 1, max => 1 }

MESSAGES

Key         Meaning                        Resolution
io.append   the journal cannot be written  check space

FORMAL SPECIFICATION

FS'(journal_path) = FS(journal_path) ⁀ json({ts ↦ now} ⊕ event) ⁀ "\n"

LIMITATIONS

The index and every base are rewritten whole at the end of each session, so a session costs O(files) writes even when one file changed.
The symlink check walks the existing directories of each path on every write; a symlink created by an administrator between the check and the write is not caught.

AUTHOR

Nigel Horne, <njh at nigelhorne.com>

LICENCE AND COPYRIGHT

Usage is subject to the GPL2 licence terms. If you use it, please let me know.

To install App::FileSync, copy and paste the appropriate command in to your terminal.

cpanm

cpanm App::FileSync

CPAN shell

perl -MCPAN -e shell
install App::FileSync

For more information on module installation, please visit the detailed CPAN module installation guide.

	Global
`s`	Focus search bar
`?`	Bring up this help dialog

	GitHub
`g` `p`	Go to pull requests
`g` `i`	Go to GitHub issues (only if GitHub is preferred repository)

	POD
`g` `a`	Go to author
`g` `c`	Go to changes
`g` `i`	Go to issues
`g` `d`	Go to dist
`g` `r`	Go to repository/SCM
`g` `s`	Go to source
`g` `b`	Go to file browse

Search terms
module: (e.g. module:Plugin)
distribution: (e.g. distribution:Dancer auth)
author: (e.g. author:SONGMU Redis)
version: (e.g. version:1.00)