Name
App::FileSync - cron-driven file synchronisation without a daemon
Version
0.001.1
Synopsis
filesync init --repo documents --root ~/Documents \
--transport local --server-root /srv/filesync --remote-repo documents
filesync sync --repo documents --dry-run
filesync sync --repo documents
Description
App::FileSync keeps one directory per client in step with a master copy on a server, and keeps a separate machine holding a replica of that master. Every run is a short-lived CLI process suitable for cron or the Windows Task Scheduler: there is no daemon, no database and no web interface.
Only files whose content changed are transferred. Data frames are deflated when that
makes them smaller, and files of at least 64 KiB are sent as rsync-style block deltas
when the other side already holds an older copy, so an edit moves the changed blocks
rather than the whole file. filesync sync --no-delta forces whole files. Each run
reports the payload bytes it sent and received and how many files moved as a delta, and
filesync log repeats those figures for past runs.
Every message the tools print is looked up in a catalogue (App::FileSync::I18N), so a translation is one new catalogue.
This module holds the pieces shared by the client, the server and the admin tools: byte-transparent JSON, path validation, conflict-copy naming and atomic file writes.
Functions
Json_Encode
Purpose: encode a structure as one line of canonical JSON.
Args: $data, any JSON-encodable value.
Returns: the JSON text, keys sorted, with no trailing newline.
Side Effects: none.
Usage:
print {$fh} json_encode($event), "\n";
Example
print json_encode({ b => 1, a => [ 'x' ] }), "\n"; # {"a":["x"],"b":1}
Api Specification
Input
[ { name => 'data', type => 'scalar', optional => 1 } ]
Output
{ type => 'string', min => 1 }
Messages
JSON::PP's own error for a value it cannot encode (a code reference, say).
Formal Specification
json_encode : VALUE → TEXT
json_decode(json_encode(v)) = v
Json_Decode
Purpose: decode one JSON document, keeping strings as byte strings.
Args: $text.
Returns: the decoded structure. Strings that fit in bytes are downgraded, so a path read back from JSON names the same file as the bytes it was written from.
Side Effects: none.
Usage:
my $config = json_decode(read_text($path));
Example
my $entry = json_decode('{"p":"caf\u00e9.txt"}');
print length($entry->{p}), "\n"; # 8 (bytes), not 9
Api Specification
Input
[ { name => 'text', type => 'string' } ]
Output
{ type => 'scalar', optional => 1 }
Messages
JSON::PP's own error for malformed JSON.
Formal Specification
json_decode : TEXT ↛ VALUE
∀ s : STRING in result • ¬ utf8_flag(s) ∨ ∃ c ∈ s • ord c > 255
Validate_Relpath
Purpose: decide whether a path is safe to join to a sync root.
Args: $path, a relative path as it travels on the wire.
Returns: a two-element list (1, undef) for a usable path, or
(0, $reason) where $reason is a translated explanation. (A list rather
than a reference because every caller destructures it at once, and it is
called once per path in the tree.)
Side Effects: none.
Usage:
my ($ok, $reason) = validate_relpath($path);
Example
my ($ok, $why) = validate_relpath('notes/../secret');
print "$why\n" unless $ok; # parent path component
Api Specification
Input
[ { name => 'path', type => 'scalar', optional => 1 } ]
Output
{ type => 'arrayref', min => 2, max => 2 }
Messages
Key Meaning
path.missing no path was given
path.not_text the path is a reference
path.empty the path is empty
path.too_long longer than 4096 bytes
path.absolute starts with /
path.drive starts with a drive letter
path.backslash contains a backslash
path.nul contains a NUL byte
path.control contains a control character
path.empty_component contains //, or starts or ends with /
path.dot_component has a . component
path.parent_component has a .. component
path.component_too_long a component is longer than 255 bytes
Resolution: rename the file on the side that holds it.
Formal Specification
SafePath == { p : seq BYTE |
0 < #p ≤ 4096 ∧ head p ≠ '/' ∧ ¬ drive(p) ∧
'\' ∉ ran p ∧ ran p ∩ CONTROL = ∅ ∧
∀ c : split('/', p) • c ∉ {⟨⟩, ".", ".."} ∧ #c ≤ 255 }
validate_relpath(p) = (1, ⊥) if p ∈ SafePath
= (0, reason) otherwise
Pseudocode
reject undef, references and empty strings
reject paths over 4096 bytes
reject absolute, drive-relative, backslash, NUL and control characters
for each "/"-separated component:
reject empty, ".", ".." and components over 255 bytes
accept
Validate_Repo_Name
Documented with "validate_client_id" below.
Validate_Client_Id
Purpose: decide whether a repo name or client id is usable as a directory name.
Args: $value.
Returns: (1, undef), or (0, $reason) with a translated reason. The
alphabet is [A-Za-z0-9][A-Za-z0-9._-]*, at most 64 characters.
Side Effects: none.
Usage:
my ($ok, $reason) = validate_client_id($id);
Example
my ($ok, $why) = validate_repo_name('../etc');
print "$why\n"; # bad repo name '../etc'
Api Specification
Input
[ { name => 'value', type => 'scalar', optional => 1 } ]
Output
{ type => 'arrayref', min => 2, max => 2 }
Messages
Key Meaning Resolution
id.missing no value was given supply one
id.not_text the value is a reference pass a string
id.empty the value is empty supply one
id.too_long longer than 64 characters shorten it
id.bad outside the allowed alphabet use letters, digits, . _ -
Formal Specification
Id == { s : seq CHAR | 0 < #s ≤ 64 ∧ head s ∈ ALNUM ∧ ran s ⊆ ALNUM ∪ {'.','_','-'} }
validate(s) = (1, ⊥) if s ∈ Id, (0, reason) otherwise
Parse_Since
Purpose: the start of a time window.
Args: $value (YYYY-MM-DD or an epoch), optional $option naming the
command-line option for the error message (default --since).
Returns: the epoch, or undef when $value is undef. A date means
midnight UTC that day.
Side Effects: none.
Usage:
my $since = parse_since($opts{since});
Example
print parse_since('2026-10-22'), "\n"; # 1792627200
Api Specification
Input
[
{ name => 'value', type => 'string', optional => 1 },
{ name => 'option', type => 'string', optional => 1 },
]
Output
{ type => 'integer', optional => 1, min => 0 }
Messages
Key Meaning Resolution
when.bad neither a date nor an epoch use YYYY-MM-DD or digits
Formal Specification
parse_since : (DATE ∪ ℕ ∪ {⊥}) ↛ ℕ ∪ {⊥}
parse_since(n) = n ; parse_since(d) = midnight_utc(d) ; parse_since(⊥) = ⊥
Parse_Until
Purpose: the end of a time window.
Args: as for "parse_since" (default option name --until).
Returns: the epoch, or undef. A date means the last second of that day, so
--until 2026-10-22 includes that day's runs.
Side Effects: none.
Usage:
my $until = parse_until($opts{until});
Example
print parse_until('2026-10-22'), "\n"; # 1792713599
Api Specification
Input
[
{ name => 'value', type => 'string', optional => 1 },
{ name => 'option', type => 'string', optional => 1 },
]
Output
{ type => 'integer', optional => 1, min => 0 }
Messages
Key Meaning Resolution
when.bad neither a date nor an epoch use YYYY-MM-DD or digits
Formal Specification
parse_until(n) = n ; parse_until(d) = midnight_utc(d) + 86399 ; parse_until(⊥) = ⊥
Format_Utc
Purpose: a compact UTC timestamp.
Args: optional $epoch (default now).
Returns: YYYYMMDDTHHMMSSZ.
Side Effects: none.
Usage:
my $stamp = format_utc($run->{ts});
Example
print format_utc(0), "\n"; # 19700101T000000Z
Api Specification
Input
[ { name => 'epoch', type => 'number', optional => 1 } ]
Output
{ type => 'string', matches => qr/\A\d{8}T\d{6}Z\z/ }
Messages
None.
Formal Specification
format_utc : ℕ → STAMP
format_utc(e) = yyyy mm dd 'T' HH MM SS 'Z' of gmtime(e)
Split_Name
Purpose: split a relative path into directory, stem and extension.
Args: $path.
Returns: an arrayref [ $dir, $stem, $ext ]; $dir has no trailing slash
and is empty at the top level, $ext includes its dot and is empty when there
is none. A leading dot (.profile) is part of the stem.
Side Effects: none.
Usage:
my ($dir, $stem, $ext) = @{ split_name('a/b/report.txt') };
Example
my $parts = split_name('notes/plan.v2.txt'); # ['notes', 'plan.v2', '.txt']
Api Specification
Input
[ { name => 'path', type => 'string', min => 1 } ]
Output
{ type => 'arrayref', min => 3, max => 3 }
Messages
None.
Formal Specification
split_name(p) = (d, s, e) where p = d ⁀ "/" ⁀ s ⁀ e ∨ (d = ⟨⟩ ∧ p = s ⁀ e)
e = ⟨⟩ ∨ (head e = '.' ∧ '.' ∉ ran(tail e)) ; s ≠ ⟨⟩
Name_With_Suffix
Purpose: make a name unique by numbering it before its extension.
Args: $path, $suffix (a number).
Returns: report-2.txt for report.txt and 2, keeping the directory.
Side Effects: none.
Usage:
my $candidate = name_with_suffix($loser_name, $n);
Example
print name_with_suffix('a/b.txt', 3), "\n"; # a/b-3.txt
Api Specification
Input
[
{ name => 'path', type => 'string', min => 1 },
{ name => 'suffix', type => 'integer', min => 1 },
]
Output
{ type => 'string', min => 1 }
Messages
None.
Formal Specification
let (d, s, e) = split_name(p)
name_with_suffix(p, n) = join(d, s ⁀ "-" ⁀ n ⁀ e)
Conflict_Name
Purpose: the name a losing version is kept under.
Args: $path, $client_id (may be undef), $epoch (default now).
Returns: the original name with .conflict-CLIENT-TIMESTAMP inserted before
the extension, kept within 200 bytes so it fits common filesystems.
Side Effects: none.
Usage:
my $name = conflict_name('notes.txt', 'laptop-1', time);
Example
print conflict_name('report.txt', 'laptop-1', 1_792_664_100), "\n";
# report.conflict-laptop-1-20261022T101500Z.txt
Api Specification
Input
[
{ name => 'path', type => 'string', min => 1 },
{ name => 'client_id', type => 'string', optional => 1 },
{ name => 'epoch', type => 'number', optional => 1 },
]
Output
{ type => 'string', min => 1 }
Messages
None.
Formal Specification
let (d, s, e) = split_name(p) ; id = sanitise(c) ; t = format_utc(when)
let room = max(16, 200 − #(".conflict-" ⁀ id ⁀ "-" ⁀ t ⁀ e))
conflict_name(p, c, when) = join(d, s[1..room] ⁀ ".conflict-" ⁀ id ⁀ "-" ⁀ t ⁀ e)
Write_Atomic
Documented with "set_mtime" below.
Read_Text
Documented with "set_mtime" below.
Set_Mtime
Re-exported from App::FileSync::IO, where they are documented.
Examples
On the server, create the master copy and allow the client once. Then, on each client, register the directory and add a cron entry that runs every minute:
# once, on the client
filesync init --repo documents --root /home/foo/Documents --server srv
# crontab -e, for foo on each client
* * * * * /usr/local/bin/filesync sync --repo documents --quiet >/dev/null
The summary line is dropped so cron does not mail on every successful run;
diagnostics still go to standard error, so a failure is mailed. Every attempt,
successful or not, is also appended to ~/.filesync/logs/documents.jsonl, and
filesync log --repo documents --exit 1 lists the ones that failed.
Windows clients use a scheduled task rather than cron. With the script in
C:\filesync and Strawberry Perl installed, the same job is set up from a
command prompt:
REM once, on the client
C:\Strawberry\perl\bin\perl.exe C:\filesync\bin\filesync init ^
--repo documents --root C:\Users\foo\Documents --server srv
REM then create the task; it runs every minute as the current user
schtasks /Create /SC MINUTE /MO 1 /TN FileSync /F ^
/TR "C:\Strawberry\perl\bin\perl.exe C:\filesync\bin\filesync sync --repo documents --quiet"
The task runs only while that user is logged on, which suits a Documents
folder; its run log is still written under %USERPROFILE%\.filesync\logs, so
filesync log --repo documents --warnings works there too.
The master copy on the server is an ordinary directory tree, so it can be mirrored to a backup directory from the same crontab:
# crontab -e, for foo on the server
* * * * * rsync -a --delete --exclude '.filesync-tmp-*' \
/home/foo/FileSync/ /home/foo/backups/FileSync/
Filesync writes each file to a temporary name and renames it into place, so the
mirror never catches a half-written upload; the exclusion keeps the temporary
files themselves out of the backup. --delete makes the copy a mirror: a
deletion that reaches the master also removes the backed-up file.
To restore the master from that backup, stop the client jobs (comment out their cron lines, or disable the scheduled task) and the server's backup job, so the backup job cannot copy a half-restored tree back into the backup. Let any running sync finish, then mirror the backup back:
rsync -a --delete --checksum /home/foo/backups/FileSync/ /home/foo/FileSync/
--checksum compares the contents, so a restored file is not skipped just
because its size and modification time happen to match the newer copy. This is
a rollback rather than a merge: files created after the backup was taken
disappear from the master, and the next sync propagates that to the clients, so
copy anything you need to keep out of the tree first. The server
would rehash the restored files on the next session anyway, but
filesync-admin reindex rebuilds the index immediately, before any client
reconnects; clients then pull the restored versions, merging edits they have not
synced as usual.
Before re-enabling the jobs, check that the copy is exact with a read-only comparison against the backup:
rsync -a --delete --checksum --dry-run --itemize-changes \
/home/foo/backups/FileSync/ /home/foo/FileSync/
No output and exit 0 mean the master matches the backup. Any line beginning
>f names a path that still differs, and *deleting names one the
rollback should have removed. The index still describes the pre-rollback tree,
so filesync-admin verify may report drift at this point; that is expected.
Rebuild it and check it:
sudo -u filesync filesync-admin reindex documents
sudo -u filesync filesync-admin verify documents --deep
reindex hashes every file in the master and replaces the index atomically; it
refuses to run while a sync session holds the repo lock, and it names any paths
it could not index. clean and exit 0 from verify confirm that the master
and its index agree. Re-enable the jobs once that check passes.
To restore just one file or directory, copy that path back instead of mirroring the whole tree, with the jobs paused as above so nothing changes the tree while the copy runs:
rsync -a --checksum /home/foo/backups/FileSync/notes/plan.txt /home/foo/FileSync/notes/
rsync -a --checksum /home/foo/backups/FileSync/photos/ /home/foo/FileSync/photos/
The trailing slash on the second source copies the directory's contents; without
it rsync would put the directory itself inside the destination. Add --delete
only when that directory should match the backup exactly, since it removes files
created after the backup was taken. Everything outside the restored paths is
left alone; refresh the index with reindex as above, or let the next session
do it, and clients pull the restored versions. Check the restored paths with the
matching rsync --dry-run --itemize-changes comparison before re-enabling the
jobs; no output means they match the backup.
Adjust /usr/local/bin/filesync to where the script was installed, because
cron's default PATH rarely includes a per-user Perl bin directory.
Limitations
These are the design weaknesses a reviewer should weigh before relying on the tool. None of them loses data silently; most cost time or space.
- One master. The topology is a star around one server. The replica is a pull-only mirror, not a failover: if the server is lost, clients stop until an administrator points them at a rebuilt master.
- Every run exchanges the whole manifest. The client sends one entry per file and the server answers with the whole plan as a single JSON line, so a run costs O(files) in time, memory and bandwidth even when nothing changed. A tree of a million files means a plan message of the order of a hundred megabytes. A change journal or Merkle summary would remove that, at the cost of the "everything is inspectable with cat" property.
- No line-length cap on the wire. Because of the previous point, a control message cannot be bounded without breaking large trees. The peer is authenticated by ssh and pinned by the forced command, so this is a resource risk from a misbehaving client, not from the network.
- Lock-step protocol. One request is in flight at a time and each 256 KiB chunk waits for its acknowledgement, so throughput is bound by the round-trip time on a high-latency link.
- Clock-dependent tie-breaks. When both sides changed a file, the newer mtime wins (the master wins ties within 2 seconds). A client with a wrong clock picks the wrong winner; the loser is still kept as a conflict copy.
- No fsync. Renames are atomic between processes but not durable across a power cut; see "LIMITATIONS" in App::FileSync::IO.
- Pure-Perl deltas are bounded. Only the first 8 MiB after the unchanged prefix are scanned for moved blocks; later changes go as literals. See App::FileSync::Delta.
- What is not synced. Symlinks, devices, sockets, extended attributes, ACLs and ownership. Permission bits are carried only between POSIX systems.
- Names. Unicode normalisation is not reconciled, so the same name in NFC (Linux) and NFD (older macOS) is two different files. A case-insensitive client cannot hold two master paths that differ only in case.
- Encapsulation costs. Private methods use Sub::Private in
enforcemode, which costs about 1.4 microseconds per call (measured on 5.40). Helpers that run per file or per JSON value (the merge comparisons, the JSON byte downgrade, the rolling checksum) are therefore lexical code references: private by scope, with no wrapper. Both Sub::Private and Sub::Protected are bypassed whileHARNESS_ACTIVEis set. - Validation costs. Params::Validate::Strict costs about 11 microseconds per call, so it guards the public entry points that run a few times per command (constructors, admin and client operations). Functions that run per path or per protocol frame ("validate_relpath", the protocol methods, the scanner) check their arguments by hand.
- Errors carry locations. Library errors are raised with
croakand so end withat FILE line N.; the commands strip that before printing, andFILESYNC_TRACE=1turns every error into a full stack trace instead. - English only. The catalogue mechanism supports other languages, plural forms and gender, but only the English catalogue ships.
See Also
- App::FileSync::I18N - the message catalogue
- Test Dashboard
Author
Nigel Horne, <njh at nigelhorne.com>
Licence and Copyright
Copyright 2026 Nigel Horne.
Usage is subject to the GPL2 licence terms. If you use it, please let me know.