Repository navigation
Add three data formats: XML, JSON and CBOR, with their judges and libraries - #15
Merged
Merged
Conversation
… libcbor for CBOR, built to be counted Each is pinned by the commit of its tag (libxml2 2.15.4, Expat 2.10.0, json-c 0.19, libcbor 0.14.0), built with counting, and counted for its reading code only: libxml2's parser and the tree, entity, DTD and encoding code it uses; Expat's parser and tokenizer; json-c's tokener and the objects it builds; libcbor's decoder, not its encoders. Each has a small program that reads a file as an application would: libxml2 as xmllint does (no network, no entity substitution), Expat as xmlwf does, json-c strictly with UTF-8 checked and nothing but white space after the value, libcbor with nothing after the item. The real files are XML written by hand, and the SVG drawings, JSON made by Python's json, and CBOR made by cbor2 with the examples of RFC 8949's appendix A: 16, 9 and 37 of them. The image also gets the packages of the new judges, and its workflow a step that checks every new reader counts.
…CBOR decoders XML: libxml2 (xmllint, not let at the network), Expat (xmlwf) and Xerces-C++ (SAX2Count). Xerces loads the external DTD a document names, from the network too, so its wrapper points every proxy at a port where nothing listens. JSON: Python's json, jq, YAJL (json_verify), Perl's JSON::PP (json_pp) and V8 (JSON.parse in Node.js); jq reads a stream of values, so its wrapper accepts only one, and json_verify and json_pp read standard input. CBOR: cbor2, which also checks what tags mean, and the Rust crate cbor-diag through its Python package, for well-formedness; what the crate has not implemented counts as unsupported. The wrappers leave file names, positions and quoted names out of the reasons. The pull-request and main workflows install them.
8 specs. xml (declarations, a DOCTYPE with an internal subset, nesting elements, CDATA, references), xml-boundary (the edges of well-formedness: external identifiers, character references at the ends of the ranges, namespaces, unusual names and white space) and xml-valid (documents valid against their own DTD); json (objects, arrays and scalars nested to fixed depths, every escape, the whole number syntax) and json-boundary (numbers past double and 64-bit precision, lone surrogates, repeated keys, deep nesting); cbor (every major type in preferred serialization, with common tags), cbor-boundary (overlong heads, the largest arguments, every length width, indefinite lengths, all simple values, float specials) and cbor-tags (each registered tag with content valid for it). Fixed in xml: the internal subset could not be reached (the DOCTYPE was a fixed string), a content model could be a bare name, names in declarations could have colons, and system identifiers a fragment; in xml-boundary, a content model could mix "," and "|". decodes is the band of the best judge at 1000 inputs, seed 1. Each format has its format.yml naming its parsers. index.yml is regenerated (fanbase 0.5.0, so that the specs that did not change are not touched).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
8 specs in three data formats, and what the registry needs around them so that they are judged and measured like the others. The CBOR specs follow RFC 8949 and its tag registry.
Three commits:
coverage/: the four libraries pinned by their tags' commits, each with a program that reads a file as an application would, real files (XML written by hand and the SVG drawings, JSON by Python'sjson, CBOR by cbor2 and RFC 8949's appendix A), and a smoke-test step incoverage-image.yml.targets/README.md; the pull-request and main workflows install them (Node.js is already on the runners).format.ymlfiles, andindex.yml(reindexed with fanbase 0.5.0, so that no other spec is touched).How it was tried
fanbase check --stricton the 8 specs, in anubuntu:24.04container with Fandango 1.3.1: all produce inputs.fanbase evaluateof the 8 specs asevaluate.ymlruns it (100 inputs, budget 40 s,--require-targets), in the same container with the packages it installs: every target ready, everydecodesmet. The next three weekly seeds make all 100 inputs of every spec.fanbase evaluate --coverageof the 8 specs with the quality run's settings (1000 inputs, seed 1, budget 100 s, curve 1/10/100/1000), which is wheredecodescomes from.For the review
The numbers (1000 inputs, seed 1, budget 100 s, in the coverage image built from this branch). The last column is how much of the counted reader's lines the 1000 files reach, against the real files of
make-seeds.sh.All eight say
decodes: always(the best judge accepts 95% or more), as for the earlier formats. Every spec reaches about as much of its reader as the real files do, or more, exceptcbor-tagsand the plaincbor, whose items are small.xml: its internal subset could not be reached (the DOCTYPE was a fixed string, so every declaration rule was dead), a content model could be a bare name, names in declarations could have colons (refused by namespace-aware parsers), and system identifiers a fragment (refused by libxml2); with these, all three XML parsers accept all of its files and 4 in 10 have an internal subset.xml-boundary: a content model could mix,and|in one group.xml-boundarydocument whose DOCTYPE names an external DTD (it loads it) and prefixed entity and notation names; jq and JSON::PP refuse lone surrogates injson-boundary; cbor2 refuses tag content that does not fit its tag incbor-boundary, and libcbor unassigned simple values; json-c refuses integers beyond 64 bits and nesting deeper than 32 (counted as a resource limit).--nonet), xmlwf does not load external entities, and Ubuntu's Xerces, which is built with libcurl and does fetch an external DTD given as a URL, has its proxies pointed at a port where nothing listens.source, and the checklist items about it and about ETHICS.md are for whoever wrote them.coverage-image.ymlrebuilds the image (wait for it before a release). The release measures the 8 new specs; the others keep their results unless the rebuilt image brings parsers of other versions.index.yml.signeeds signing again.Checklist
fanbase checkpasses (it also runs here)version(none is changed)description,authorsandlicensesourcesays whose and thelicenseallows it