NAME
File::Raw::XML - an XML parser
VERSION
Version 0.02
SYNOPSIS
use File::Raw qw(slurp);
use File::Raw::XML qw(file_xml_decode);
my $doc = file_xml_decode($bytes, id_attrs => ['ID']);
my $doc = file_slurp('metadata.xml', plugin => 'xml');
my $root = $doc->root;
my ($assertion) = $root->find($SAML_NS, 'Assertion');
my ($signature) = $assertion->find($DS_NS, 'Signature');
my $bytes = $assertion->c14n(mode => 'exclusive',
without => [$signature]);
my $signed = $doc->by_id(ID => $reference_uri_without_the_hash);DESCRIPTION
An XML 1.0 and Namespaces 1.0 parser producing a read-only tree, and a canonicaliser for it in the three algorithms a signature names: Exclusive XML Canonicalization 1.0, Canonical XML 1.0 and Canonical XML 1.1. It is written for consumers that verify signatures over the canonical form, where one wrong byte is a signature that never verifies, and it is small enough to own rather than fence: the safe subset of XML is not large.
By default it is not a full XML processor. It refuses every document with a DOCTYPE and accepts UTF-8 only, and that one rule is what removes entity expansion, external entities, parameter entities, external DTD fetches, the billion laughs and XXE as a class. Three surfaces reach the same tree: a File::Raw plugin, a direct codec for bytes already in memory, and a C ABI for XS consumers.
profile => 'full' is a full XML 1.0 and 1.1 processor over the same core: document type declarations and their internal and external subsets, entity expansion under budgets, every encoding, DTD validation and XInclude. Nothing in that paragraph is reachable without asking for it, and what the default profile does is unchanged by any of it. See "PROFILES".
XPath 1.0, the writer and the mutable tree are over any document under either profile, since none of them reads a declaration.
PROFILES
my $doc = file_xml_decode($bytes); # strict
my $doc = file_xml_decode($bytes, profile => 'full'); # everythingstrict is the default everywhere: the codec, the plugin, the reader and the C ABI's parse entry. Under it a document type declaration is refused wherever it stands, the input is UTF-8 or it is refused, and the tree, the messages and the canonical bytes are what they are and will not move. A caller who wants the guarantee that no entity was expanded and nothing was fetched has it by doing nothing.
full reads what strict refuses. On its own it brings document type declarations, the internal subset, entity expansion under "BUDGETS", UTF-16 and ISO-8859-1 through "ENCODINGS", XML 1.1, and the record of where CDATA sections were. Anything that reaches outside the document is a further option on top of it and is off until named: resolve for an external subset or an external entity ("RESOLVERS"), validate for the validity constraints ("VALIDATION"), xinclude for inclusion ("XINCLUDE").
An option that only the full profile can honour is refused under strict rather than ignored, so a caller who asks for one and does not get it is told.
WHAT IS ACCEPTED
Under either profile: XML 1.0 with Namespaces 1.0, in UTF-8: elements, attributes, text, CDATA sections, comments, processing instructions and the XML declaration; the five predefined entities and numeric character references. Line ends are normalised as section 2.11 of XML 1.0 requires, and attribute values as section 3.3.3 does for an attribute with no declared type, which without a DOCTYPE is every attribute: a literal tab, line feed or carriage return in a value becomes a space, while a character reference to one of them stays the character.
Namespace declarations are resolved at parse time. The xml prefix is bound without a declaration and is never reported as one. Unprefixed attributes have no namespace, never the default.
Under profile => 'full', additionally: a document type declaration with its internal subset and, with a resolver, its external subset; element, attribute list, entity and notation declarations; general and parameter entities, expanded under "BUDGETS"; conditional sections in external markup; XML 1.1 and its line ends; and the encodings "ENCODINGS" lists. Attribute values are then normalised by their declared type rather than as CDATA, and declared defaults are materialised onto every element that lacked them.
WHAT IS REFUSED
Every refusal dies with a message naming the byte offset it was found at and up to sixteen bytes of what was there, with anything outside printable ASCII rendered as \xNN. In the order the parser meets them:
Input longer than
max_bytes, when one is given.Under
strict, a UTF-16 or UTF-32 byte order mark, or a first pair of bytes that is one; an XML declaration naming any encoding other than UTF-8; an XML version other than 1.0. "ENCODINGS" says whatfullreads instead.Under
strict, a DOCTYPE anywhere, and every other declaration beginning<!with it. This is the rule that removes entity expansion, external entities, parameter entities, external DTD fetches, the billion laughs and XXE as a class.profile => 'full'is the only thing that reads one, and under it the billion laughs is stopped by "BUDGETS" instead.A byte sequence that is not UTF-8: overlong forms, surrogates, anything above U+10FFFF. A character that is valid UTF-8 and not an XML character: a C0 control other than tab, line feed and carriage return, U+FFFE, U+FFFF.
A named entity other than the five predefined ones; a character reference to a non-character;
]]>in content; a literal<in an attribute value;--inside a comment; a processing instruction whose target isxmlin any case anywhere but offset 0.A name that is not a name; a qualified name with more than one colon, or a colon at either end.
An attribute given twice on one element, literally or through two prefixes bound to one namespace.
A prefix with no binding in scope;
xmlns:xmlbound to anything but the XML namespace, or another prefix bound to it;xmlns:xmlns; undeclaring a prefix with an empty value in a document that is not XML 1.1, that being Namespaces 1.1; a relative namespace URI, over which canonicalisation is undefined.Nesting deeper than
max_depth.More than one root element, none, or content outside it.
Two elements carrying the same value for an attribute named in
id_attrs. Which of two is "the" element is the whole of the signature wrapping attack, and a lookup that picked one would be picking for the attacker, so the document is refused instead.
DIRECT CODEC
file_xml_decode($bytes, %options)
Parse bytes into a File::Raw::XML::Document, or die with a message as above. A string with the character flag on is taken as its UTF-8; one without it is taken as bytes, and refused if they are not UTF-8. A string holding Latin-1 characters must be upgraded first.
Exported on request, by name or under :codec; :const exports the node kind constants FRX_ELEMENT, FRX_TEXT, FRX_COMMENT, FRX_PI and FRX_DOCUMENT and the event kinds FRX_START, FRX_END and FRX_DOCTYPE; :all exports both. A bare use exports nothing.
file_xml_events($bytes, %options)
The push form of File::Raw::XML::Reader: parses $bytes and calls, for each event, the callback given under start, end, text, comment, pi or doctype, with the reader positioned on the event as the one argument, so the reader's accessors describe it. A missing callback skips its events. Every other key is an option of the reader. Dies as the codec would, at the first refusal. Exported with file_xml_decode under :codec, or by name.
File::Raw::XML->new_document(%options)
An empty File::Raw::XML::Document with only its document node, to be built with "new_element" in File::Raw::XML::Document and the editing methods of File::Raw::XML::Node. The one option is version, 1.0 by default or 1.1.
OPTIONS
The same options for the codec, the plugin and the reader. An unknown one dies, and so does one that only profile => 'full' can honour when the profile is strict: an option that was silently ignored would be a caller believing something the parser is not doing.
Under either profile:
max_bytes- the largest input accepted, in bytes; 0, the default, is no cap. The ceiling is the caller's to set.max_depth- the deepest nesting accepted; the root is depth 1; the default is 256.id_attrs- an arrayref of attribute local names, in any namespace, whose values are indexed for "by_id" in File::Raw::XML::Document and refused when duplicated. With none, nothing is indexed and nothing is refused.
Under profile => 'full' only:
encoding- the input's encoding, overriding what the bytes and the declaration say. See "ENCODINGS".resolve- where an external subset, an external entity or an XInclude gets its bytes. See "RESOLVERS".base- the document's own URI, which relative system identifiers resolve against.validate- check the document against its document type declaration. See "VALIDATION".xinclude- processxi:include. See "XINCLUDE".max_entity_depth,max_expansion_bytes,max_expansion_ratio,max_fetches,max_xinclude_depth,max_token_bytes- see "BUDGETS".
ENCODINGS
Under strict the input is UTF-8, and anything else is refused at the byte order mark, at the first pair of bytes or at the declaration.
Under profile => 'full' the encoding is worked out the way section 4.3.3 and appendix F say: a byte order mark if there is one, then the first four bytes, then what the declaration names. UTF-8, UTF-16 in either byte order and ISO-8859-1 are read; anything else is refused by name. The encoding option overrides all of it, for the caller who knows what the bytes are because the transport said so.
Whatever came in, what comes out is characters, and every offset in a refusal is an offset into the bytes you passed rather than into the transcoded form, so a message points where you can look.
DOCUMENT TYPE DECLARATIONS
Under profile => 'full' a DOCTYPE is read: its name, its public and system identifiers and its internal subset, all of which "doctype" in File::Raw::XML::Document reports as they were written.
What the declarations say is applied to the tree before you see it. Attribute defaults are materialised onto every element that lacked them, in declaration order after the written attributes. Attribute values are normalised by their declared type rather than as CDATA. Entity references are replaced by their replacement text: there is no entity reference node, which is what Canonical XML requires and what makes the canonical form of a document with entities equal to the canonical form of the same document without them.
An external subset is fetched only with a resolve, and refused without one rather than skipped, because a document whose declarations were partly read would canonicalise differently from a processor that read them all.
RESOLVERS
The core opens no file and no socket. An external subset, an external entity and an XInclude reach it as bytes through resolve and no other way, so nothing leaves the process that the caller did not agree to.
my $doc = file_xml_decode($bytes, profile => 'full', resolve => sub {
my %r = @_; # kind, public_id, system_id, base
return $store{ $r{system_id} };
});kind is subset, entity or xinclude. Returning undef is a refusal and so is dying, and either way the message names the reference's offset. The system identifier arrives already resolved against the document's base and the base of whatever declared it.
my $doc = file_slurp($path, plugin => 'xml', profile => 'full',
resolve => 'file');is the one built-in resolver: it reads files, and only files whose real path, symbolic links followed, lies under the real path of the document's own directory. It needs a document with a path, so the codec refuses it.
BUDGETS
Every expansion has one, each refused with the offset it was crossed at. A budget of 0 means the default and never means unlimited.
max_entity_depth- how deep entity references may nest; 16.max_expansion_bytes- how much text expansion may produce in all; 16 MiB.max_expansion_ratio- how many times the input's own length that may be; 100.max_fetches- how many times the resolver may be called; 32.max_xinclude_depth- how deep inclusion may nest; 8.max_token_bytes- the largest single token a reader will hold while waiting for its end; 16 MiB.
Together they are what stops the billion laughs under full, where the DOCTYPE rule that stops it under strict no longer applies. A document that would expand to more than any of them is refused rather than truncated.
THE TREE
A File::Raw::XML::Document owns the tree; every File::Raw::XML::Node handed out keeps its document alive for as long as the node exists, so there is nothing the caller has to do about lifetime. The tree is read-only, and it is not shareable across interpreter threads.
Names, attribute values and text come back as character strings. Where a method takes a namespace, undef means any namespace and the empty string means no namespace. One class serves every node kind, with kind telling them apart; the document node is a node too, whose children are the top-level comments and processing instructions with the root element among them.
CANONICALISATION
my $bytes = $node->c14n(mode => 'exclusive',
comments => 0,
prefix_list => ['xs', '#default'],
without => [$signature]);
my $bytes = $doc->c14n(%same);The canonical form of the node's subtree, or of the whole document, as bytes with no character flag, because a signature is over bytes.
mode names the algorithm: exclusive, the default, is Exclusive XML Canonicalization Version 1.0 (W3C Recommendation, 18 July 2002); inclusive is Canonical XML Version 1.0 (15 March 2001); inclusive-1.1 is Canonical XML Version 1.1 (2 May 2008). Comments are rendered when comments is true, the WithComments variant of each.
without names subtrees left out of the node set: the element named, its attributes and everything below it. This is the enveloped-signature transform, and it is the only subset the tree expresses; a node's ancestors are always in the set when the node is, so nothing here can orphan a node. A node from another document is an error.
The node whose ancestors lie outside the set is the one you called c14n on, and each algorithm treats it as its Recommendation says: Canonical XML 1.0 copies onto it the nearest occurrence of every attribute in the xml namespace it does not itself carry; Canonical XML 1.1 copies xml:lang and xml:space that way, joins xml:base through its section 2.4, and never copies xml:id; the exclusive algorithm copies nothing. The exclusive algorithm renders a namespace declaration only where it is visibly used and not already rendered by an ancestor in the output; prefix_list is its InclusiveNamespaces PrefixList, with #default naming the default namespace, and the inclusive modes ignore it.
EVENTS AND STREAMING
File::Raw::XML::Reader is a pull reader over the same parser: bytes in through feed, one event at a time out through next, and the memory held is the memory of one record, so a stream of a million records costs what one costs. "file_xml_events" is the push form of it.
The xml plugin streams too:
file_each_line($path, $cb, plugin => 'xml', record => [$ns, $local]);yields one File::Raw::XML::Document per matching element, built from that element's start event, with the namespace bindings in scope carried into it so each record canonicalises as it would have in place.
A reader enforces every well-formedness constraint the codec does, at the same offsets. It does not validate: a validity constraint is about a finished document and an event stream has none.
WRITING
my $bytes = $doc->to_string(indent => 2);
file_spew($path, $doc, plugin => 'xml', encoding => 'UTF-16');"to_string" in File::Raw::XML::Document writes a document or a subtree back as markup: an encoding, a declaration, indentation, <a/> or <a></a>, the quote, and the recorded DOCTYPE. It is a second serialiser beside the canonical one and shares only the escapers with it, so no option here can move a canonical byte.
Parsing what it wrote gives a document equal to the one it was given, which "equals" in File::Raw::XML::Document is for and which the test suite asserts over every document it has.
EDITING
my $doc = File::Raw::XML->new_document;
my $root = $doc->new_element($NS, 'p:Envelope');
$doc->document->append($root);
$root->set_attr('', 'ID', 'x1');A document can be built from nothing or edited in place: see "BUILDING" in File::Raw::XML::Document and "EDITING" in File::Raw::XML::Node. A node is never moved or freed by an edit, so every node you already hold stays valid, and a prefix that would mean something else where a subtree lands gets a declaration rather than changing meaning.
XPATH
my @nodes = $doc->xpath('//item[@sku = $sku]', vars => { sku => $s });
my $n = $doc->xpath('count(//item)');XPath 1.0 in full, over a document of either profile: the four value types, the thirteen axes, the 27 core functions. See File::Raw::XML::XPath, which also compiles an expression once for evaluating many times.
VALIDATION
my $doc = file_xml_decode($bytes, profile => 'full', validate => 1);
my $doc = file_xml_decode($bytes, profile => 'full', validate => 'collect');
warn "$_\n" for $doc->errors;validate => 1 checks every validity constraint of XML 1.0 and dies at the first one broken, with the constraint named and the byte offset it was found at, in the shape every other refusal uses. validate => 'collect' checks them all, returns the document, and puts the list on it for "errors" in File::Raw::XML::Document, in document order. There is no warning channel and no third state: the specification's errors that are raised "at user option" are collected.
Validating is not the same as parsing. A document with no document type declaration is well-formed and cannot be valid, and one whose declaration declares nothing leaves every element type undeclared, so validate refuses both.
A namespace declaration is validated as an attribute. Namespaces in XML is a layer above XML 1.0, so xmlns:foo="..." is an Attribute to the grammar and has to be declared for its element type like any other. A document type declaration written without the xmlns attributes its markup uses therefore makes every document drawn from it invalid, which is a real friction between document type declarations and namespaces and not a quirk of this parser. It costs nothing unless validate is asked for. Content models are compiled to an automaton, and one that is not deterministic is itself a violation, reported against the declaration it is in rather than against the document.
A validated document gets its ID index from the attributes the document type declaration says are IDs, which is what "by_id" in File::Raw::XML::Document and XPath's id() then answer over. A document parsed with id_attrs keeps that index instead: the two are different questions and never both apply.
File::Raw::XML::Reader does not validate. A validity constraint is about a finished document, and an event stream has none.
XINCLUDE
Under profile => 'full' and xinclude => 1, every xi:include element in the namespace http://www.w3.org/2001/XInclude is replaced, after the document is parsed, by what it names. The bytes arrive through the same resolve option external entities use, called with a kind of xinclude, so an href reaches no file and no socket the caller has not agreed to.
href is resolved against the base URI in force at the include element, which is the document's base option with any xml:base above it applied. parse is xml, the default, or text, which reads the bytes as one text node under encoding and never as markup. xpointer takes a bare name, resolved through the included document's ID index, or the element() scheme with a child sequence; the other XPointer schemes are refused by name. An xi:include with no href includes from the document itself.
When the resolver refuses, or the bytes do not parse, the xi:fallback children stand in the include's place; with no xi:fallback that is fatal. An inclusion loop, and a chain deeper than max_xinclude_depth (8 by default), are fatal whatever fallback says. Every fetch counts against max_fetches.
The included subtree is copied into the including document, so what comes back is one document, and section 4.7.5's fixup puts an xml:base on each included root naming where it came from. That attribute is part of the document from then on: it is written by to_string and it appears in the canonical form, which is correct and worth knowing before you canonicalise an included document.
C ABI
File::Raw::XML exposes a small C ABI so that other XS modules can parse, walk and canonicalise entirely in C, with no per-call Perl dispatch. The motivating consumer is Punk::SAML, which verifies XML signatures over the canonical form.
The header is distributed through ExtUtils::Depends - a consumer does not copy it. Building this dist installs frx_abi.h and writes File::Raw::XML::Install::Files, so a dependent's Makefile.PL that says
my $pkg = ExtUtils::Depends->new('My::Consumer', 'File::Raw::XML');
WriteMakefile( ..., $pkg->get_makefile_vars );picks up frx_abi.h on its include path automatically.
This is an integration surface for XS authors, not part of the Perl API. Perl callers should use "DIRECT CODEC" and the two classes above.
The table
The contract lives in include/frx_abi.h:
#define FRX_ABI_VERSION 1
typedef struct frx_doc frx_doc; /* opaque; owns every node and string */
typedef struct frx_node frx_node; /* opaque; borrowed from its doc */
enum { FRX_ELEMENT = 1, FRX_TEXT, FRX_COMMENT, FRX_PI, FRX_DOCUMENT };
enum { FRX_C14N_EXC = 0, FRX_C14N_INC10, FRX_C14N_INC11 };
typedef struct frx_abi {
int abi_version; /* consumers compare >=, never == */
/* the strict profile, and all `parse` will ever be */
void (*opts_init)(frx_opts *o);
frx_doc *(*parse)(pTHX_ const char *bytes, STRLEN len,
const frx_opts *o, SV **err);
void (*doc_free)(pTHX_ frx_doc *d);
root, document, kind, ns, local, prefix, parent, first_child, next,
attr_count, attr, attr_value, find, by_id, text, c14n
/* the full profile */
parse_ex, err_format
version, standalone
doctype_name, doctype_public, doctype_system, doctype_subset
cdata_span_count, cdata_span
error_count, error_at
write_opts_init, write, tree_equal
reader_new, reader_free, reader_feed, reader_next, reader_str,
reader_int, reader_offset, reader_attr_count, reader_attr,
reader_subtree, reader_error
new_document, new_element, new_text, new_comment, new_pi,
append_child, insert_before, remove_node, set_attr, remove_attr,
set_text, declare_ns, set_name, import_node
xpath_compile, xpath_free, xpath_var_count, xpath_var_name,
xpath_bind_str, xpath_bind_num, xpath_bind_bool,
xpath_result_new, xpath_result_free, xpath_eval,
xpath_result_kind, xpath_result_number, xpath_result_string,
xpath_result_count, xpath_result_node
} frx_abi;The header is the contract and carries every signature; the names above are the shape of it. parse takes frx_opts and means the strict profile, and that is all it will ever mean: a consumer that calls it can never be handed an entity-expanded or transcoded document, whatever a later version adds. parse_ex takes frx_opts_ex, which begins with its own sizeof so that later versions can add options without a new entry.
Everything that can fail after parse takes an frx_err * rather than a message SV, and err_format renders one, so no entry needs a wrapper to translate a refusal.
The table is append-only from the first release onwards: new entries go at the end, FRX_ABI_VERSION bumps, existing offsets never move. A consumer written against version N keeps working against every later version.
File::Raw::XML::_abi_ptr
my $iv = File::Raw::XML::_abi_ptr;Returns the address of the process-wide frx_abi table as an integer (an IV). A consumer calls this once at BOOT, INT2PTRs it to a const frx_abi *, and checks ->abi_version >= the version whose entries it calls. At least, never exactly: the table is append-only from the first release onwards, so a provider newer than the consumer is always safe, and an equality check would turn every append into a breaking change. Not intended to be called from Perl for any other purpose. File::Raw::XML::_abi_selftest and File::Raw::XML::_abi_selftest_full walk every entry end to end in C and are what t/20-abi.t runs.
Functions and ownership
parse returns a document the consumer frees with doc_free, or NULL with a mortal message SV in *err; it never croaks. Every string the tree accessors return is borrowed from the document, NUL-terminated, and valid until doc_free; ns and prefix return "" when there is none, and local on a processing instruction is its target. Every node is borrowed from its document: a consumer that holds a node holds the document. attr_value and find take NULL for any namespace and "" for none; find iterates by passing the previous match as after. by_id answers exactly one element or NULL, over the attribute names given in id_attrs at parse. text and c14n return an SV with a reference count of one owned by the caller, or NULL on an allocation failure; c14n's bytes carry no character flag.
Two lifetimes are shorter than the document's, and both are the reader's: a string from reader_str or reader_attr is valid until the next reader_next and not after, because the reader releases what a record held when the record ends, and the frx_err * from reader_error is valid while the reader is. A message never is borrowed: frx_err.what and the refusal from xpath_compile are static strings, so a consumer can free what failed and then report it.
reader_new, xpath_compile and xpath_result_new each hand out something the consumer frees with the matching entry. new_document and reader_subtree hand out documents, freed with doc_free.
Example: a consumer walking an Assertion
Vendor nothing - add File::Raw::XML via ExtUtils::Depends (above), resolve the table at boot, then use it wherever needed:
#include "frx_abi.h" /* found via ExtUtils::Depends, not copied */
static const frx_abi *FRX = NULL;
/* at BOOT */
{
dSP; IV p;
ENTER; SAVETMPS; PUSHMARK(SP); PUTBACK;
call_pv("File::Raw::XML::_abi_ptr", G_SCALAR);
SPAGAIN; p = POPi; PUTBACK; FREETMPS; LEAVE;
FRX = INT2PTR(const frx_abi *, p);
if (!FRX || FRX->abi_version < 1)
croak("File::Raw::XML with a compatible C ABI is required");
}
/* later: the signed element, canonicalised without its signature */
static SV *
signed_bytes(pTHX_ const char *xml, STRLEN len, const char *id)
{
static const char *const ids[] = { "ID" };
frx_opts o; frx_c14n c; SV *err = NULL, *out;
frx_doc *d;
const frx_node *elem, *sig;
FRX->opts_init(&o);
o.id_attrs = ids; o.n_id_attrs = 1;
d = FRX->parse(aTHX_ xml, len, &o, &err);
if (!d) croak_sv(err);
elem = FRX->by_id(d, "ID", id, strlen(id));
sig = elem ? FRX->find(elem, "http://www.w3.org/2000/09/xmldsig#",
"Signature", NULL) : NULL;
if (!elem || !sig || FRX->parent(sig) != elem) {
FRX->doc_free(aTHX_ d);
croak("no signed element with that ID");
}
c.mode = FRX_C14N_EXC; c.comments = 0;
c.prefix_list = NULL; c.n_prefix = 0;
c.without = &sig; c.n_without = 1;
out = FRX->c14n(aTHX_ elem, &c);
FRX->doc_free(aTHX_ d);
return out; /* +1, the caller's */
}SEE ALSO
File::Raw, File::Raw::XML::Document, File::Raw::XML::Node, Punk::SAML.
AUTHOR
LNATION <email@lnation.org>
LICENSE AND COPYRIGHT
This software is Copyright (c) 2026 by LNATION <email@lnation.org>.
This is free software, licensed under:
The Artistic License 2.0 (GPL Compatible)