/ 
File-SOPS-0.003
/ File::SOPS::Encrypted

NAME

File::SOPS::Encrypted - Parse and generate SOPS encrypted values

VERSION

version 0.003

SYNOPSIS

use File::SOPS::Encrypted;

# Parse an encrypted value string
my $enc = File::SOPS::Encrypted->parse(
    'ENC[AES256_GCM,data:xyz,iv:abc,tag:def,type:str]'
);

# Check if a string is encrypted
if (File::SOPS::Encrypted->is_encrypted($string)) {
    my $decrypted = $enc->decrypt_value(key => $data_key, aad => $path);
}

# Encrypt a value
my $enc = File::SOPS::Encrypted->encrypt_value(
    value => 'secret',
    key   => $data_key,
    aad   => 'database:password',
);

# Get encrypted string representation
my $string = $enc->to_string;
# => ENC[AES256_GCM,data:...,iv:...,tag:...,type:str]

DESCRIPTION

File::SOPS::Encrypted handles parsing and generation of SOPS encrypted value strings. Each encrypted value in a SOPS file is represented as:

ENC[AES256_GCM,data:base64,iv:base64,tag:base64,type:str]

Values are encrypted with AES-256-GCM using:

  • A shared data key (32 bytes, encrypted separately for each recipient)

  • A random initialization vector (IV) per value (32 bytes -- see "iv")

  • Additional Authenticated Data (AAD) derived from the value's path

  • Type preservation (str, int, float, bool, bytes)

algorithm

Encryption algorithm. Currently only AES256_GCM is supported. Defaults to AES256_GCM.

data

Encrypted ciphertext as raw bytes. Required.

iv

Initialization vector (IV) as raw bytes, 32 bytes. Required.

Note that this is deliberately not the conventional 12-byte (96-bit) AES-GCM nonce. The SOPS reference implementation generates a 32-byte nonce and builds its cipher with Go's cipher.NewGCMWithNonceSize, so 32 bytes is part of the on-the-wire format rather than a choice this module is free to make. AES-GCM derives its initial counter block differently for non-96-bit nonces (the nonce is folded through GHASH instead of being used directly), which means an IV of a different length does not merely look unusual -- it produces a value the Go implementation cannot authenticate, and vice versa.

encrypt_value generates 32 random bytes accordingly. Do not "correct" this to 12.

tag

Authentication tag as raw bytes, 16 bytes for AES-GCM. Required.

type

Original value type for deserialization. One of str, int, float, bool, bytes, time or comment -- the seven the reference implementation writes. Anything else makes "decrypt_value" die with Unknown datatype, as Go does; until 0.003 it silently returned the raw plaintext, so a document this module could not actually interpret looked like it had been read.

time is read but never produced here: it is Go's time.Time, stored as RFC3339 and returned as that string, because Perl has no native date type.

comment is a YAML comment, and since 0.003 it is both read and written. sops attaches one to the node that follows it: above a mapping key it stays a #ENC[...] line, which YAML::XS discards, but above a sequence entry sops writes it as a real sequence element (- ENC[...,type:comment]) and every parser keeps it. Such a leaf is not a value, so it has a Perl value of its own -- File::SOPS::Comment -- which "decrypt_value" hands back, "detect_type" types, and the MAC digest leaves out. Before 0.003 the leaf was read as an ordinary string, which failed MAC verification and, under ignore_mac => 1, put a value in the caller's list that the file does not contain. See docs/adr/0041.

Defaults to str.

parse

my $enc = File::SOPS::Encrypted->parse($string);
# Returns undef if $string is not encrypted

Parses a SOPS encrypted value string.

Takes a string like ENC[AES256_GCM,data:...,iv:...,tag:...,type:str] and returns a File::SOPS::Encrypted object with decoded attributes.

Returns undef if the string is not in the encrypted format.

Dies if the string has the shape but not the content: a data, iv or tag field that is not valid standard base64 -- a character outside the alphabet, or a length that is not a multiple of four. MIME::Base64 silently drops what it cannot read, so such a field used to decode to something shorter and fail much later, as Authentication failed - data may be corrupted or as a cipher rejecting the nonce length. Go decodes with base64.StdEncoding, which fails immediately and names the problem.

"is_encrypted" is unaffected and never dies: it answers about the shape, which is what decides whether a value is a candidate for decryption at all.

is_encrypted

if (File::SOPS::Encrypted->is_encrypted($string)) {
    # It's an encrypted value
}

Class method to check if a string is in SOPS encrypted format.

Returns true if the string matches the ENC[...] pattern.

encrypted_type

my $type = File::SOPS::Encrypted->encrypted_type($string);
# 'str', 'int', 'comment', ... or undef

Class method. Returns the type: label of an ENC[...] string without decoding its base64 fields, or undef if the string is not an encrypted value.

It shares one anchored pattern with "is_encrypted" and "parse", and it answers for a value whose ciphertext is damaged -- which "parse" does not, because it decodes. That is the case it exists for: a type:comment leaf has to be recognised as a comment before anyone tries to decrypt it.

is_comment

if (File::SOPS::Encrypted->is_comment($value)) { ... }

Class method. True for a File::SOPS::Comment, which is what a plaintext tree holds where the document holds a comment. False for everything else, undef and every other reference included.

It is the predicate behind "detect_type"'s comment rung, kept as a method of its own because the MAC walk has to ask the same question without asking for a type. It is not true for an ENC[...,type:comment] string -- that is what the file says a leaf is, which "encrypted_type" answers, and on the encrypt side such a string is an ordinary string a caller passed.

See docs/adr/0041.

to_string

my $string = $enc->to_string;
# => ENC[AES256_GCM,data:xyz==,iv:abc==,tag:def==,type:str]

Serializes the encrypted value to SOPS string format.

Returns a string representation with base64-encoded components.

encrypt_value

my $enc = File::SOPS::Encrypted->encrypt_value(
    value => 'secret',
    key   => $data_key,        # 32 bytes
    aad   => 'database:password',  # Additional Authenticated Data
    type  => 'str',            # optional, auto-detected
);

Class method to encrypt a value.

Encrypts a scalar value using AES-256-GCM with a random IV. Returns a File::SOPS::Encrypted object.

The aad (Additional Authenticated Data) is typically the path to the value in the data structure (e.g., database:password:), used to prevent value substitution attacks.

Both value and aad are always encoded to UTF-8 before they reach the cipher, which is what the Go implementation authenticates against, and neither consults Perl's UTF-8 flag. For a string whose characters are all below U+0100 that flag is an internal storage detail rather than a statement about meaning: "caf\x{e9}" may be held as one byte or as two, Perl considers both the same string, and YAML::XS::Dump and the JSON emitter (built with utf8) write both to the file as caf\xc3\xa9. Anything that reads the flag therefore disagrees with the bytes our own emitter wrote, which is a document that fails its own MAC.

The exception is type => 'bytes', SOPS's binary type, which is passed through untouched -- and which is how a caller says that a scalar really is bytes rather than characters. See "value_to_bytes" and docs/adr/0003.

Known limitation as of sops 3.13.3: a document that contains a type:bytes cell cannot be opened by the reference implementation. sops -d exits 2 with panic: runtime error: hash of unhashable type []uint8 in aes/stashKey on the very first such leaf -- measured against sops 3.13.3 on YAML, dotenv and INI, on payloads of "hello" and a 4-byte blob, all identical. The label is what trips it: the same payload under type:str reads back at exit 0. No sops store produces a type:bytes cell (YAML's !!binary and the binary input store both surface as type:str), so this is a sops-side bug we can only warn about: a caller who uses the documented type => 'bytes' escape hatch above writes a document sops cannot open in any format, as of 3.13.3. The read path is unaffected -- a foreign type:bytes cell decrypts correctly through "decrypt_value" -- because the panic is sops's own aes cache, keyed by plaintext, refusing to index on a byte slice. See k136 and the "Known limitation" section of docs/adr/0003.

Type is auto-detected from the value if not specified, by "detect_type".

Passing type explicitly overrides the label only, never the bytes: those always come from what the value is. encrypt_value(value => '007', type => 'int') writes the label int and the plaintext 007, because a Perl string is written verbatim. That is how you reproduce a document some other producer wrote; it is not how you write 007 as an integer, which no SOPS implementation does (see "value_to_bytes").

Dies if the value is one no SOPS document can carry -- today, an integer wider than Go's int64. See "assert_representable".

A non-finite float (NaN, +Inf, -Inf) is written, in both formats, as type:float and the plaintext "value_to_bytes" derives from the number. That is what sops -e stores for a plain .inf, -.inf or .nan -- measured against sops 3.13.3, six documents, --output-type yaml and --output-type json alike, exit 0 with type:float on the wire in both. Every release before this one refused it here. See "assert_representable", k122 and docs/adr/0040.

This method passes encrypted => 1 to "assert_representable", which it may because it is the encrypted slot. The one non-finite shape still refused is a scalar that states a string half contradicting its number, which an encrypted slot would drop without a trace.

Dies if the value's plaintext is empty (which includes undef). GCM ciphertext is the length of its plaintext, so the result would be an ENC[AES256_GCM,data:,...] that neither "parse" nor sops accepts -- sops stops with Input string ENC[...] does not match sops' data format. Until 0.003 such a string was built and returned. Empty values belong in the document unencrypted, which is what sops does with them and what "encrypt" in File::SOPS does before calling this.

decrypt_bytes

my $bytes = $enc->decrypt_bytes(
    key => $data_key,  # 32 bytes
    aad => 'database:password:',
);

Decrypts the value and returns the authenticated plaintext as raw bytes, with no type conversion and no character decoding applied.

aad is UTF-8 encoded before verification by the same unconditional rule "encrypt_value" uses, so the two sides derive the same AAD from the same key path however Perl happens to be storing it.

This is what the MAC is computed over. "decrypt_value" runs the same bytes through a type conversion (0 + $x, unpack('d', pack('d', $x)), JSON::PP::Boolean) whose inverse is not exact in Perl: '007' comes back as 7, '1.50' as 1.5, and the 100000000000000000000 that Go writes for 1e20 restringifies as 1e+20. Feeding a re-serialization of the converted value to the digest therefore produces a different hash than the one the producer computed over the plaintext. Anything hashing a decrypted value must use this method, not "decrypt_value" -- the one exception being type:bool, where SOPS's ToBytes titlecases whatever spelling it parsed, so the caller normalises true/false to True/False.

Dies if authentication fails (wrong key, corrupted data, or mismatched AAD).

decrypt_value

my $value = $enc->decrypt_value(
    key => $data_key,  # 32 bytes
    aad => 'database:password',
);

Decrypts the encrypted value.

Returns the decrypted value with type conversion applied (int, float, bool are converted to appropriate Perl types) and, for textual types, decoded from UTF-8 into a character string -- the inverse of what "encrypt_value" accepted. This is the Perl-facing side of the boundary; "decrypt_bytes" is the wire side and is what anything feeding a digest must use.

type:bytes is the one exception: it is SOPS's binary type, so it is returned as raw bytes with nothing decoded. A type:str whose plaintext is not valid UTF-8 is also returned as bytes rather than being mangled.

type:comment comes back as a File::SOPS::Comment and not as its text, because a comment is not a value: an object cannot be mistaken for a string by a caller walking the tree, and it is what tells "encrypt" in File::SOPS to write the leaf back as a comment. Until 0.003 this returned the text.

A type:float value comes back as a scalar Perl calls a float even when its digits spell a whole number, so detect_type still says float and the next write keeps the type:float label the document already carried. The conversion is unpack('d', pack('d', $plaintext)) rather than + 0.0, because the addition sets the public SVf_IOK flag on an integral result and that flag is what "detect_type" reads: a whole: 2.0 written by sops came back out of our own rotate as type:int, in both formats, at exit 0 and without a word. See k73 and docs/adr/0009.

That conversion is a float64, as Go's is, so a plaintext carrying more integer precision than a double can hold comes back rounded to the double Go also reads -- 9007199254740993 as 9007199254740992. "decrypt_bytes" still returns the plaintext digits verbatim.

A type:float plaintext of -0 comes back as a negative zero, which is what Go's strconv.ParseFloat returns for it and what the document meant. Neither conversion gets there on its own: both settle that text as an integer zero, which has no sign to keep, and + 0.0 additionally lost it a second time to IEEE round-to-nearest (-0.0 + 0.0 is +0.0). Nothing in Perl shows the difference -- == and print cannot tell the two zeroes apart -- but "value_to_bytes" can, so the value writes back out as -0 where it used to write 0 and silently change a document sops itself had produced. See k72.

Dies if authentication fails (wrong key, corrupted data, or mismatched AAD).

Also dies, rather than guessing, on a plaintext that does not match its label:

  • a type:int plaintext that is not a decimal integer, or is one Go's strconv.Atoi would refuse -- that is, outside int64. Until 0.003 that went through Perl's int(), which is exact up to 2**64-1 on a 64-bit Perl and silently rounds above it, so the same document produced one answer here and value out of range in sops.

  • a type:float plaintext that is not a number, which Perl's numeric conversion turns into 0 without a word and strconv.ParseFloat rejects.

  • an unrecognised type: altogether -- Unknown datatype, as Go says.

"decrypt_bytes" still returns the authenticated plaintext in every one of those cases, which is how a value is recovered from such a file.

detect_type

my $type = File::SOPS::Encrypted->detect_type($value);
# => 'str' | 'int' | 'float' | 'bool' | 'comment'

Class method. Returns the SOPS type of a Perl scalar, from the scalar itself rather than from a pattern match on its text.

  • a File::SOPS::Comment is comment

  • a JSON::PP::Boolean (JSON->true, JSON->false, or a true/false loaded by YAML::XS or JSON::MaybeXS) is bool

  • a scalar Perl itself marks as a boolean -- !!1, !!0, $x > 3, builtin::true, any comparison's result -- is bool too, on perl 5.36 and newer. See below.

  • a scalar Perl holds as an integer is int

  • a scalar Perl holds as a floating point number is float

  • a negative zero is float, even where Perl also holds an integer for it -- the one place a scalar's two numeric halves contradict each other and the integer half is the wrong one. See below.

  • everything else, including every string, is str

This is the rule the Go implementation uses -- it takes the type from what the YAML/JSON parser returned -- and both parsers this distribution uses preserve the distinction, so a quoted scalar is a string end to end. Measured against sops 3.13.3: bare false is type:bool, but "false", "true", "1", "0", "007" and "1.50" are all type:str.

The corollary is that Perl's own literals decide the type for a structure passed straight to "encrypt" in File::SOPS: 5432 is int and '5432' is str. The string 'true' is a string.

Past Perl's own integer limit both parsers hand this ladder the same shape, and in a JSON document that is new. A bare integer literal too wide for an IV or a UV -- 100000000000000000000, -9223372036854775809 -- arrives as the double carrying its source spelling, a "dualvar" in Scalar::Util publishing NOK and POK, so the float rung answers it. YAML::XS has always returned that; until 0.003 Cpanel::JSON::XS returned a plain string SV instead, bit-identical to the same digits quoted, so one number was a float read out of a YAML document and a str read out of a JSON one -- and "rotate" in File::SOPS wrote a JSON string where the reference had written a number. There is no big integer in the SOPS data model: past int64 a JSON number is a float64 to Go, and sops 3.13.3 writes such a leaf as type:float in an encrypted slot and rounds it to its own double in an unencrypted one, so float is the reference's answer too. A quoted "100000000000000000000" is unaffected and stays a str; the two are told apart at parse time by the decoder's own type map, never by a pattern match on the text. See k63 and docs/adr/0020.

The same leaf reaches this ladder one magnitude lower, where the limit crossed is Go's and not Perl's. A bare JSON literal in [2**63 .. 2**64-1] is a UV to Perl and a float64 to Go, so sops -e normalises it before this library ever sees the file -- 9223372036854775808 is written back as 9223372036854776000 -- and marks it type:float in an encrypted slot and an unencrypted one alike (measured, sops 3.13.3, twelve literals across the window, both slots). File::SOPS::Format::JSON hands such a literal over as the same dualvar, so the float rung answers it and "assert_representable"'s int64 refusal -- which used to make "rotate" in File::SOPS croak on a document sops itself had written -- is not reached. The window is JSON's alone: yaml.v3 resolves those digits as a uint64 sops cannot walk at all, so no such YAML document exists to read, and one built by hand is still refused. A caller's own Perl UV in the window is still refused as well -- no parser has spoken for it, so calling it a float would be this library guessing. See k101 and docs/adr/0021.

Perl has no boolean type, but since 5.36 it has a boolean SV, and that is what type:bool reads on a plain scalar. !!1, !!0, $x > 3, 'a' eq 'a', defined $x and builtin::true all produce it, the mark survives assignment and storage in a hash, and both emitters write such an SV as a bare true/false. It is asked for through builtin::is_bool, which is the same predicate the emitters use -- a Scalar::Util/dualvar carrying 1 and '1' is indistinguishable from !!1 in every public flag, and is not a boolean to any of the three. On a perl older than 5.36 there is no such SV, and there type:bool still needs a JSON::PP::Boolean or an explicit type.

Until 0.003 such a scalar was an int: the digest covered 1/0 while the document said true/false, so File::SOPS->encrypt(data => { admin => ($user->{level} > 3) }) wrote a file that failed its own MAC. See k90 and docs/adr/0016.

Note that Perl marks a scalar as numeric in place the first time it is used in numeric context, so if ($cfg->{port} > 1024) before encrypting turns '8080' into an int. See docs/adr/0002.

A negative zero is the one exception to reading the integer flag first, and it exists because that in-place marking is not harmless for this value. Perl caches an integer on a float whenever the cast round-trips, and for -0.0 it decides the cast round-trips while the sign does not survive it: the cached integer is 0, which is a different number on the wire (0 against -0) and a different MAC digest. So a scalar publishing both halves is asked which half it is, and for this one value the answer is the float.

The route in is ordinary code, in both directions. YAML::XS caches that integer for an integral float written in exponent notation, so -0.0e0, -0e0 and -0.000e2 arrive here carrying it where -0.0 does not; and any $v > 1, $v == 0 or int($v) on a caller's own -0.0 sets it before encrypt ever sees the tree. Until 0.003 that turned the value into a 0 in the document -- and for the YAML spellings into a file sops -d rejected with MAC mismatch. See k89 and docs/adr/0015.

integer_fits_int64

File::SOPS::Encrypted->integer_fits_int64($value)   # 1 or 0

Class method. True where $value is inside Go's int64 range -- -9223372036854775808 .. 9223372036854775807 -- which is the only range the SOPS int type covers, and false where Perl holds an integer Go cannot. Perl's integers are unsigned-capable and reach 2**64-1, so that window is real: see "assert_representable" for what this distribution refuses to write there, and "Integers are Go's int64, and Perl's are wider" in File::SOPS for why.

It exists so that a format handler can ask the question without spelling the boundary a second time. A JSON document can carry a bare integer literal in that window -- sops -e writes one itself, having normalised it through the float64 Go reads it as -- and telling that leaf apart from an ordinary port: 5432 is a decision the JSON parser makes per leaf. Two copies of a constant are this distribution's signature defect: when they drift they are wrong together, so the wire bytes stay self-consistent and only the reference implementation disagrees.

$value must be a scalar Perl holds as an integer -- one "detect_type" calls an int. That is not checked here, deliberately: the caller's own flag test is the cheap fact and has to come first, and a second type ladder inside this method is exactly what ADR 0002 removed. Neither other kind is answered correctly. A float of exactly 2**63 answers true, because the comparison promotes 2**63-1 to a double and the two become the same number. A decimal string is worse -- '-9223372036854775809' numifies to exactly -2**63 and answers true to a value Go refuses -- which is why the text form of this test is a separate, private one that reads digits and never numifies.

assert_representable

File::SOPS::Encrypted->assert_representable($value);
File::SOPS::Encrypted->assert_representable($value, encrypted => 1);

Class method. Dies if $value is something no SOPS document can carry, and returns true otherwise. Called for every leaf "encrypt" in File::SOPS is about to write -- encrypted or not -- before anything is emitted.

encrypted says which slot the leaf is going into, and defaults to false, which is the stricter answer. It changes exactly one of the checks below, the non-finite float: an encrypted slot's wire form for one is type:float and the plaintext "value_to_bytes" writes, which is what sops -e stores in both formats, so the emitters' inability to spell the number says nothing about that slot. The two callers inside this distribution supply it from what they structurally know -- "encrypt_value" is the encrypted slot, and File::SOPS's MAC sweep asks "should_encrypt_path" in File::SOPS::Metadata. See docs/adr/0040.

There are three such values today:

An unblessed reference. \1, \$x, \@arr, \%hash, sub {}. detect_type calls a reference str, so "value_to_bytes" would digest its stringification -- SCALAR(0x...), ARRAY(0x...), HASH(0x...) or CODE(0x...), the ref's heap address. _encrypt_tree would feed that same address into the encrypted slot, and the document would verify against itself (doc and digest agree on the address) but store a value that differs per run and is meaningless on a later read. The pre-fix code wrote the file silently and the caller had no reason to notice; that is the defect k67 exists to close. A blessed object passes: ADR 0008 measured that an encrypted Math::BigFloat, an object overloading "", and a Regexp all round-trip correctly today -- they have a stringification the caller chose, which is exactly what an encrypted slot carries. The exception is the same class detect_type uses for the digest, which is blessed($v) and so covers any subclass: the encrypted-slot rule and the unencrypted-slot guard asked the same question in different ways and the encrypted slot is the looser one.
A non-finite float: NaN, +Inf, -Inf -- unless it carries go-yaml's own token for that double. "value_to_bytes" writes them as NaN / +Inf / -Inf -- the same text Go's strconv.FormatFloat produces -- but no emitter derives that text from the number: Cpanel's JSON encoder writes null (the document is silently rounded), JSON::XS writes bare inf (invalid JSON, sops refuses with invalid character i), and YAML::XS writes a bare Inf / -Inf / NaN, which Go's yaml.v3 resolves as a string. Measured against sops 3.13.3 in an unencrypted YAML slot, Inf is sops -d exit 51 and -Inf / NaN are exit 0 with the leaf silently retyped from a float to a string -- which is worse, not better. The pre-fix code wrote the file anyway, and the caller had no reason to notice; that is the defect k59 exists to close.

Narrowed since 0.003 (k113, docs/adr/0031): a scalar that also carries, as its own string half, one of the twelve plain tokens go-yaml resolves to exactly that double -- .inf, +.Inf, -.INF, .nan and their case variants -- is written, in an unencrypted YAML slot. That is the shape "parse" in File::SOPS::Format::YAML produces for a document sops wrote (docs/adr/0026), and refusing it left this library reading a document it could not write back: rotate croaked on a file sops -d reads at exit 0. Measured, all twelve tokens: sops -e > File::SOPS->rotate > sops -d exit 0, the wire byte-identical.

Both halves have to agree. dualvar(+Inf, '-.inf') is a contradiction and stays refused, the same answer docs/adr/0012 gives an integer whose halves disagree, and so does any other spelling -- .INf and +.nan are strings to sops, and a dualvar(+Inf, 'banana') would put a document on disk that fails its own MAC. This check is a gate: whether the document being written can really carry the token is answered where the format is known, by the foreign-resolution guard docs/adr/0013 installs in the YAML handler. A JSON document has no such spelling in an unencrypted slot (measured, sops -d exit 51) and is refused there.

Everything in this item is about the UNENCRYPTED slot, which is the only one whose leaf reaches the document as a token. With encrypted => 1 the value is written instead: the wire carries type:float and the plaintext +Inf / -Inf / NaN, in both formats, which is what sops -e writes for the same value (k122, docs/adr/0040). The one exception is a scalar that states a string half its number contradicts -- dualvar(+Inf, 'banana'), or the JSON literal of 400 zeros whose text is its digits -- because the encrypted slot is derived from the number and that text would be dropped without a trace.

Store the value as a string instead where none of that applies -- that is type:str, written verbatim, and the same number (or its deliberate stringification) is round-tripped exactly through both implementations. Reading is unaffected: a type:float plaintext of +Inf or NaN is accepted by "decrypt_value" today, and sops itself writes such a value, so the read path has to keep accepting it.

Since 0.003 a parsed document can reach this guard on its own, where the route in used to be Perl code or a YAML .inf: a bare JSON integer literal that overflows a double -- 1 followed by 400 zeros -- is a float ("detect_type") whose value is +Inf, and it used to be read as a string and written back as one. The croak is the closer answer, because sops refuses that document itself and one step earlier, at unmarshal time (measured, sops 3.13.3, exit 2, strconv.ParseFloat: value out of range).

An integer outside Go's int64 range. Perl's integers reach 2**64-1, Go's int stops at 2**63-1, and the reference implementation writes type:int only within that range. Measured against sops 3.13.3, a wider integer produces a document it will not read:

  • encrypted, it is type:int with a plaintext strconv.Atoi rejects -- sops -d stops with value out of range

  • unencrypted, it reaches the document verbatim, and sops -d either refuses to walk it (YAML: unknown type: uint64) or recomputes the digest from the float64 it parsed and reports MAC mismatch (JSON)

sops's own JSON store truncates such an integer to a float64 and writes the truncated value. File::SOPS does not: a value silently losing digits on its way through a library whose job is to preserve it is the defect this method exists to prevent. Store the digits as a string instead -- that is type:str, written verbatim, and it round-trips exactly through both implementations.

Since 0.003 the message offers a second answer, because since ADR 0021 there is one: hand the guard the double instead, unpack('d', pack('d', $value)), and the leaf takes the float rung and gets exactly what sops writes for the same digits -- type:float and the double's canonical decimal -- at the cost of the digits the double cannot hold. It is an answer and not a promise: 12 literals across the window in both formats and both slots gave 41 of 48 cells sops -d exit 0, and 7 -- all of them unencrypted YAML -- refused here by ADR 0013's guard instead, because the decimal "value_to_bytes" makes the emitter write there is one yaml.v3 resolves as a uint64. $value + 0.0 is not that conversion: Perl keeps the integer and the value lands back on this croak.

The window is positive-only, and structurally so: SVf_IOK means the SV carries an IV or a UV, an IV bottoms out at exactly int64min and a UV cannot be negative, so no integer SV exists below the range. Measured for k104 across 14 negative decimals bracketing int64min and uint64max and 13 construction routes each: 182 rows, 36 of them int, and not one below int64min. Below the range a value arrives as a float (or a string) and takes the float rung instead (ADR 0020).

This is a write-side rule only. Reading is unaffected: sops's truncated 12345678901234567000 parses back as a Perl integer above int64, and "value_to_bytes" must still hash it as those digits or a legitimate sops document would fail verification. An older file this library itself wrote under 0.003 with an unblessed ref in an encrypted slot is read back as the heap address that was stored, which is unrecognisable but verifiable; the guard refuses the new write and leaves the read path alone.

value_to_bytes

my $bytes = File::SOPS::Encrypted->value_to_bytes($value);
my $bytes = File::SOPS::Encrypted->value_to_bytes($value, $type);

Class method. Returns the wire plaintext for a value: the bytes that get encrypted, and the bytes the MAC digest is taken over. Those are the same bytes by definition, which is why there is one method and not two -- the ciphertext and the digest disagreeing is this distribution's signature defect, and it is invisible from inside Perl because both sides are then consistently wrong.

$type defaults to "detect_type". It selects the boolean spelling and nothing else; the bytes always come from the value:

  • a boolean is True or False (SOPS titlecases them)

  • a comment (File::SOPS::Comment) is its text, verbatim -- what sops puts after the #, leading space included

  • an integer is its canonical decimal form -- Perl's 0 + $value, which is what Go's strconv.Itoa writes

  • a float is strconv.FormatFloat($v, 'f', -1, 64): the shortest decimal that round-trips, in positional notation and never in exponent notation. Measured against sops 3.13.3, which stores 1.0 as 1, 1.50 as 1.5, .5 as 0.5 and 1e20 as 100000000000000000000

  • anything else, including every string, is written verbatim

type => 'bytes' is the one type that is not UTF-8 encoded on its way out, mirroring "decrypt_value", which does not decode it either. It is SOPS's binary type, so it is not text, and it is the only way to tell this module that an unflagged scalar really is a byte string rather than a Perl string that happens to be stored as bytes -- a distinction Perl itself does not make. Everything else is encoded unconditionally; see "encrypt_value".

Sops 3.13.3 cannot read a document that contains a type:bytes cell. The write-side escape hatch this paragraph describes therefore produces a document sops cannot open in any format -- exit 2 with panic: runtime error: hash of unhashable type []uint8 in aes/stashKey, on the first such leaf, regardless of payload. The label is what trips it. See "encrypt_value" for the full warning, and k136.

So a Perl string is never renormalised -- '007' stays 007 and '1.50' stays 1.50 -- while a Perl number always is. A document that gets this wrong is not merely odd-looking: Go recomputes the MAC by re-serializing the value it parsed out of the plaintext, so 007 stored under type:int digests as 7 on the reading side and sops -d rejects the whole file.

That renormalisation now reaches a wide JSON integer literal as well. Such a leaf carries both halves -- the double and its own digits, which is what either parser returns past Perl's integer limit (see "detect_type") -- and the bytes come from the number, so 99999999999999999999 is written as 100000000000000000000. Inside the range a sops-written document can carry, that costs nothing: the digits sops writes are already the double's canonical decimal, so the two texts are the same string and the digest does not move at all (measured against sops 3.13.3, every positional literal it writes). A hand-written literal that is not its double's canonical decimal does move -- to exactly the value sops -e rounds the identical document to itself.

Character strings are UTF-8 encoded on the way out, by the same unconditional rule "encrypt_value" documents.

In an untyped store these bytes are also what the emitter writes

For YAML and JSON the emitter and the digest legitimately write different text: Format::YAML writes a bare true where this method returns True, and that is correct, because go-yaml parses true back into a boolean and Go re-derives True from it.

The flat formats have no such reader. ENV and INI carry no type syntax, so sops hands the digest the literal text of the line -- and for an unencrypted leaf, where there is no type: label either, the document verifies only if the text written is the text this method returns. docs/adr/0035 therefore decides that the ENV and INI emitters (k36, k37) write exactly what this method returns, and refuse nothing for its type.

That is not a divergence dressed up as one. Measured against sops 3.13.3, one document per row, both formats, both slots: the sops_mac plaintext is SHA-512 of this method's output on every row of the ladder -- True, 1, -0, 100000000000000000000, the empty string for a null. sops's own emitter then writes a display form into the unencrypted slot for three of them and cannot read the file back:

value       sops wrote      digest covers   sops -d
true        true            True            MAC mismatch, exit 51
null        <nil>           (empty)         exit 51 -- and exit 25 in an
                                            encrypted slot, where the
                                            placeholder reaches the file raw
1.0         1.0             1               exit 51
1e20        1E+20           100000000000000000000   exit 51
-0.0        -0.0            -0              exit 51
42          42              42              exit 0
1.5         1.5             1.5             exit 0

Writing this method's output instead leaves the digest exactly where it is and repairs the one line that disagreed with it: each of those texts is a line sops itself writes for the corresponding string, and a document carrying it reads back at exit 0. The cost is that an unencrypted flat-format value loses its type on the way back -- a boolean returns as the string True, an integer as "42" -- which is what sops's own untyped reader does to every value in that slot. See k124 and k125, and docs/adr/0035.

The type label needs no format rule at all: an ENV or INI parser hands the walk plain string SVs, so "detect_type" answers str for the whole document by itself, which is what makes sops -e on a plaintext .env write type:str for NUM=5. A typed source keeps its types in a flat document in both implementations alike.

The return is a plain string -- a scalar carrying its text and not the number that text spells. This matters to a caller who feeds the result back in: "detect_type" reads the SV and not the characters (ADR 0002), so a return still carrying its numeric half went back onto the wire as type:float, or as type:int for the text -0, where the caller meant the type:str they were holding. k80; the bytes were never affected, only what the scalar says it is.

canonical_float_tree

my $tree = File::SOPS::Encrypted->canonical_float_tree(
    $data,
    roundtrips => sub { my ($value, $text) = @_; ... },
    carrier    => sub { my ($value, $text) = @_; ... },
);

Class method. Returns a copy of $data in which every float leaf that the caller's emitter cannot write faithfully has been replaced by a carrier holding the canonical decimal from "value_to_bytes" -- the same text the MAC digest covers.

This exists because the digest and the document have to state the same number. "value_to_bytes" writes a float as Go's strconv.FormatFloat($v, 'f', -1, 64), up to 17 significant digits, while every emitter in this distribution renders a double with 15. For a value that needs 16 or 17 the document said one number and the digest covered another, and the file failed its own MAC. See docs/adr/0006.

Two callbacks, both given the original scalar and its canonical text:

  • roundtrips answers whether the emitter's own rendering of this value parses back to the same value. It must measure that -- emit and reparse -- rather than model it, which is what keeps this correct if an emitter changes. Returning true leaves the scalar untouched, so a value the emitter already writes faithfully keeps exactly the bytes it has today.

    It is asked about two leaf classes. For a float a false answer selects the carrier below. For an integer that carries its own, different string form -- a "dualvar" in Scalar::Util, or a value a YAML parser kept the source spelling of -- a false answer is a refusal: "detect_type" calls such a leaf an int, so the digest covers the number, while both emitters write the string half, and the document then fails its own MAC (measured, sops -d exit 51 in both formats). It is refused rather than repaired because both halves are a candidate for what the caller meant and nothing measurable separates a spelling (007 for 7) from a contradiction (five for 5). The emitter is asked only where the two halves actually differ; an int whose string half is the digest's text costs one string comparison. See k84 and docs/adr/0012.

  • carrier returns the replacement, and is format-specific: YAML::XS emits a "dualvar" in Scalar::Util's string half verbatim and unquoted, while every JSON backend quotes it and needs a Math::BigFloat under allow_bignum instead.

  • reject is optional and is called as $reject->($leaf, $where) for every blessed or otherwise referenced leaf, so a handler can refuse one its emitter cannot write as the text the digest covers. $where is that leaf's key path, colon-joined, or the string (document root) -- the same shape the MAC walk's own messages use, with array indices carried because this is a diagnostic and not an AAD. Both handlers put it in front of their message, so one bad leaf in a large document is named rather than searched for (k68). "detect_type" calls every reference but a JSON::PP::Boolean str, so the digest covers its stringification, and an emitter that writes something else instead produces a document that fails its own MAC. Both handlers use it: File::SOPS::Format::YAML refuses every reference except an exact JSON::PP::Boolean, because YAML::XS writes the rest as !!perl/ tagged structures; File::SOPS::Format::JSON refuses every reference except an exact JSON::PP::Boolean, for the same reason -- under allow_bignum Cpanel::JSON::XS writes a Math::BigFloat or Math::BigInt as a bare number and an unblessed \1 / \0 as bare true / false. Both messages name the class or ref kind and never the value; the exception is the exact class (a JSON::PP::Boolean subclass is refused the same way detect_type accepts it -- the guard's question is what the emitter can write). See docs/adr/0008 (k65 on the YAML side, k66 closing the known gap on the JSON side).

  • reject_scalar is optional and is called as $reject_scalar->($leaf, $where, $path, $text) for every plain scalar leaf on its way out -- the leaf as it will be written, so a carried float arrives as the carrier rather than as the scalar it replaced. $where is the same key-path string reject is given; $path is that path as an array reference, for a handler that has to answer a question about where in the document the leaf sits rather than only report it. $text is the text the MAC digest covers -- the walk passes the one it already derived, which for a carried float is the original value's, and leaves it undef for a leaf it never converted, so a handler with a cheap gate of its own does not pay a conversion per string.

    This is the one hook that asks about a reader instead of about this distribution's own emitters. A document's leaves are resolved again by whoever opens the file, and where that resolver disagrees with "detect_type" the document and its own MAC state different things -- Go's yaml.v3 reads a leading-zero integer as octal and 0o10, 0x1f, 1_000, .inf, Null and 2015-01-01 as numbers, an infinity, a null and a timestamp, where YAML::XS reads all of them as this module's type. File::SOPS::Format::YAML installs it on the encrypt path only. See docs/adr/0013 (k86).

    Where $text is undef a handler takes it from "value_to_bytes", on the leaf it was handed. It must not render one itself: that is the second conversion this whole walk exists to avoid.

The canonical text is not recomputed by the caller and must not be: a second conversion is how the ciphertext and the digest came to disagree in the first place, and it is invisible from inside Perl because both sides are then consistently wrong.

Call this at emit time, on a copy, after the digest has been taken. The walk that computes the MAC must never see a carrier: the Math::BigFloat one is a blessed reference, which "detect_type" calls str, so the digest would cover its stringification instead of the number.

NaN, +Inf and -Inf never go through roundtrips. That callback reparses with this side's own parser, which reads .inf back as a string, so it answers no for a leaf that is perfectly writable, and the carrier would then be handed +Inf as the text and write a bare +Inf -- a document sops -d reads at exit 0 with the leaf silently retyped to a string.

What the walk does with one depends on whether anything can vouch for it. A non-finite leaf whose string half is a plain token go-yaml resolves back to that same double -- .inf, -.inf, .nan and their case variants, the shape a sops-written YAML document parses to (docs/adr/0026) -- is passed to reject_scalar, whose model of the foreign resolver is the one that decides. In a document that carries a MAC and whose handler installed no reject_scalar the leaf is refused, naming the key path: that is JSON, where a non-finite float has no spelling at all (measured, sops -d exit 51 unencrypted, exit 4 encrypted). See docs/adr/0031 (k113).

A non-finite leaf with no string half of its own -- the shape a decrypted type:float arrives in, and a bare 9**9**9 -- has stated no spelling, so one is asked of carrier: it is called with .inf, -.inf or .nan, and the leaf it returns is accepted only if it carries that token as its public string half and still renders as the same bytes. The YAML carrier answers with a dualvar, which YAML::XS writes as the bare token; the JSON carrier croaks, and the leaf is refused naming the key path, as sops -d --output-type json and sops edit refuse the same document (exit 4). So decrypt_file reproduces sops's own .inf byte for byte for an encrypted type:float as well as for an unencrypted slot, where it used to write Inf and turn the leaf into a string on the next read. A leaf that does publish a string half keeps it, whatever it says. See docs/adr/0037 (k134).

Whether the document carries a MAC is read off the tree this walk is handed: a handler's serialize puts the metadata under sops before calling its emitter, and the plaintext emitters hand over the tree without one. The YAML handler also says so by installing reject_scalar, but the JSON handler's emit is the identical call on both paths.

A negative zero is carried, and it is the one case where the carrier's text is not the canonical decimal: value_to_bytes writes -0, which Go's yaml.v3 resolves as an integer and digests as 0. The rule ADR 0006 states is that the emitted decimal has to parse back to the same double, not that it has to be spelled canonically, so the YAML carrier writes -0.0. See "emit" in File::SOPS::Format::YAML.

canonical_float_dualvar

my $value = File::SOPS::Encrypted->canonical_float_dualvar($value);

Class method. Returns $value with its stringification replaced by the canonical decimal from "value_to_bytes" -- numerically the same double, as a string the text the document actually contains. Anything that is not a plain float scalar comes back untouched: a reference, undef, a string, an int, a boolean, and a non-finite float, whose NaN / +Inf / -Inf are wire spellings rather than a number's decimal.

This exists because a decrypted float is a bare NV with no string form of its own, so Perl renders it at 15 significant digits: an encrypted 0.30000000000000004 printed as 0.3, where sops -d --extract prints all 17. See k61 and docs/adr/0010.

Only for a value on its way out to a caller. The result is a "dualvar" in Scalar::Util, and a dualvar inside a tree changes the bytes the emitters write. Not the value and not the type -- both formats write the canonical decimal as a number, measured, sops -d exit 0 reading the same double (YAML::XS writes the string half verbatim; Cpanel::JSON::XS quotes it, which sends the leaf to the Math::BigFloat carrier that writes it bare, k78 / ADR 0011). What changes is the spelling: this text is always positional, where an emitter's own rendering switches to an exponent at the extremes, so 1e300 is written as 301 digits and 1e-7 as 0.0000001 -- in both formats. Correct documents, different bytes from the same value passed as a plain number. "extract" in File::SOPS therefore calls this on the single leaf it returns and on nothing else; "decrypt" in File::SOPS and "decrypt_file" in File::SOPS do not call it at all.

The digest is unaffected either way: "detect_type" reads SVf_NOK before POK, so a dualvar is still a float, and "value_to_bytes" re-derives the identical text from its numeric half.

SEE ALSO

SUPPORT

Issues

Please report bugs and feature requests on GitHub at https://github.com/Getty/p5-file-sops/issues.

CONTRIBUTING

Contributions are welcome! Please fork the repository and submit a pull request.

AUTHOR

Torsten Raudssus <getty@cpan.org>

COPYRIGHT AND LICENSE

This software is copyright (c) 2026 by Torsten Raudssus.

This is free software; you can redistribute it and/or modify it under the same terms as the Perl 5 programming language system itself.