decisions.md

Design decisions

Every entry here is a trade made on purpose: what it costs, and what it buys. Changing one means re-arguing the trade, not editing the code. The README states what stdx does; this file states why.

Entries marked owner wait on a ruling and are not settled. Everything else is settled and is re-argued, not edited. Entries 1 to 10 record the rules the owner set in the brief that started stdx on 2026-09-25. Entries 11 to 18 were proposed the same day, as the decision records the brief asked for before any codec code, and the owner ruled on each after reviewing it. Entries 19 and 20 came out of that review, entry 21 out of design §8 step 2, entries 22 to 25 out of step 11, entry 26 out of the owner's review of CI, entry 27 out of the owner's request for JSON, entry 28 out of the owner's request that its state machines be proved, entry 29 out of issue 13's DEFLATE decoder, entry 30 out of the JSON baselines' numbers, entry 31 out of design §8 step 17's profile, entry 32 out of step 12's brotli decoder, entry 33 out of the owner's ruling on what step 17's profile left, entry 34 out of the losing files of issue 13, entry 35 out of decision 17's json measurement, entry 36 out of design §8 step 9's comparison with libdeflate's output, entry 37 out of design §8 step 18's non-ASCII rows, entry 38 out of the question entry 37 left, how fast the check runs alone, entry 39 out of the owner's ruling on the gap entry 38 measured on x86-64, entry 40 out of issue 15's request for a probe of the CPU, entry 41 out of the owner's ruling once the benchmarks carried a memset of their own, entry 42 out of design §8 step 9's look at E.coli's parse beside libdeflate's, entry 44 out of the blocks that look counted, and entry 45 out of a probe of how the benchmarks time a small file.

Scope and shape

1. stdx is a library of compression codecs, and it stands alone. Ruled by the owner on 2026-09-25. It holds DEFLATE with its zlib and gzip containers, Zstandard and brotli, each with an encoder and a decoder. colibri's decision 90 moved the DEFLATE decoder here from colibri, so that another project can take a codec without taking an HTTP library.

   • stdx never depends on colibri and never names a consumer in its source. tools/lint/denied_words.zig refuses the names in every .zig file (invariant 15).
   • colibri is the first consumer: its h11 decodes the gzip and deflate transfer codings of RFC 9112 §7.2 with stdx. Other projects want the encoders and decoders for the HTTP content codings of RFC 9110 §8.4: gzip, deflate, br and zstd.
   • Entry 27 amends this item: stdx also holds a JSON encoder and decoder, the json module.

   The alternatives refused, in colibri's decision 90: the decoder inside colibri, which makes every other project take HTTP to get a codec; and stdx holding colibri's core as well, which would have made stdx carry limits that belong to HTTP.

2. No I/O. Ruled by the owner on 2026-09-25. A codec reads octets the caller already has and writes into storage the caller owns. It opens no file or socket, starts no thread, and makes no syscall. Cost: a caller that has a file or a socket writes the loop that feeds the codec. Gain: the codec runs under any I/O model the caller has (blocking, io_uring, an event loop, a kernel bypass), and a test drives it with nothing but slices. tools/lint/io.zig refuses the syscall, filesystem, network, thread, process and logging surfaces of std under src/ (invariant 2). Entry 40 names the one exception: platform, whose probe a program calls once when it starts, may read macOS's sysctlbyname, and no other module may import it.

   The alternative refused: the std.Io.Reader and std.Io.Writer interfaces of Zig 0.16, which std.compress.flate takes. They put a vtable call on every refill and a blocking model in the API, and colibri's survey found that std.compress.flate cannot resume when its input runs out.

3. No heap. Ruled by the owner on 2026-09-25, as colibri's decision 35 ruled it for colibri. There is no Allocator anywhere in src/: no parameter, no field, no std.heap, and no test that allocates. The caller owns every state, window, table and hash chain, and places each one where it chooses: static storage, its own arena, or memory it mapped. Each size is a comptime constant the codec exports, per codec and per encoder level (decision 12).

   Cost: a stream that asks for more than the size the caller chose is refused, never accommodated, and a decoder of a format whose window the stream chooses (brotli, Zstandard) holds the largest window it accepts. Gain: nothing to allocate, fail or leak; no error.OutOfMemory in any signature; and @sizeOf accounts for all of a codec's memory.

   The alternatives refused: an allocating constructor, which puts error.OutOfMemory into the API and makes stdx decide where memory lives; and allocation through callbacks the caller supplies (zlib's zalloc, libzstd's customMem), which keeps a failure path in every call that no format requires.

4. Streaming, and able to resume. Ruled by the owner on 2026-09-25. Every call takes whatever input the caller has and writes what fits. It ends by saying whether it needs more input, needs more room, or is done, and the next call resumes where it stopped. Work per call is bounded by the octets it consumed and wrote. A whole-buffer helper is built on the streaming call, never the reverse. Decision 11 proposes the call.

   The alternative refused: a whole-buffer core with a streaming wrapper, the shape of libdeflate, tinf and puff. It needs the whole coded body in memory at once, which a caller receiving a body in pieces cannot give. colibri's survey found that Zig's std.compress.flate cannot resume when its input runs out and never compares the checksums it reads, so a caller would hold the whole body and check the checksums itself.

5. Deterministic. Ruled by the owner on 2026-09-25. No clock, no PRNG, and no pointer value in any output. An encoder's output is a pure function of its input and its parameters, byte-identical across hosts and build modes. A decoder's output and verdict are a pure function of its input. Decision 11 adds that the way a caller splits the input and the output across calls is not a parameter: it changes no octet of the output (invariant 5).

   Cost: an encoder cannot adapt to elapsed time, and a heuristic that would read a pointer's alignment reads an offset instead. Gain: a failure found on one seed reproduces on every host, and a cache or a content hash over encoded output stays valid across hosts and releases of the same version.

6. One module per codec, plus a checksum module, each exported by name. Ruled by the owner on 2026-09-25, as colibri's decision 86 exports colibri's modules. build/modules.zig creates checksum, deflate, zlib, gzip, zstd and brotli with b.addModule, so a dependent reaches one with dependency.module("gzip") and compiles no other codec. The library keeps no process-wide mutable state (invariant 4), so a dependent runs codecs on its own threads. Decision 11 proposes a seventh module, codec, for what the six share.

   The alternative refused: a dependent that imports stdx's source files by path. It would bypass the module graph that keeps each codec from reaching another (invariant 14).

   Entry 27 adds an eighth module, json, exported the same way, and entry 40 a ninth, platform, which no other module imports.

Tooling and checks

7. stdx's developer tooling comes from pepegrillo, a lazy Zig package pinned by hash. Ruled by the owner on 2026-09-25, as colibri's decision 36 ruled it. The lint driver and its generic rules, the cognitive-complexity scorer, the commit-message linter and the pre-push hook come from github.com/c4milo/pepegrillo, pinned at 6fcb273, its main branch on that date. build.zig requests it only when stdx is the root build, so a project that depends on stdx never fetches it. tools/ keeps stdx's configuration of each rule with the fixtures that pin it, and the two rules only stdx has: module-graph and rfc-citation. .githooks/pre-push is a copy of pepegrillo's hook, and zig build test compares the two byte for byte.

   The alternatives refused are colibri's: a copy of the tooling per repository, which drifts within a day; a vendored copy with a sync step per change; a git submodule; and a .path dependency, which resolves wrong inside a worktree.

8. Other implementations are test oracles and benchmark baselines, compiled in tools/ and bench/ only. Ruled by the owner on 2026-09-25.

   • zlib and Wuffs are ruled in for the DEFLATE family (stdx issue 1, 2026-09-25). zlib is fetched from its release archive, and Wuffs as the single file wuffs-v0.4.c from github.com/google/wuffs-mirror-release-c. Each is a lazy package pinned by hash.
   • libzstd (github.com/facebook/zstd), Google's brotli (github.com/google/brotli), zlib-ng and libdeflate were proposed, and the owner ruled all four in on 2026-09-25. Each is a lazy package pinned by hash, and joins CLAUDE.md's list of ruled dependencies in the commit that adds it.
   • Each does two jobs. In tools/, it is an oracle: the same inputs through stdx and through it give the same octets, and stdx refuses what it refuses (decision 15). In bench/, it is a baseline, run in the same harness in the same run as stdx (decision 10).
   • Nobody working on stdx reads an oracle's source (decision 9). An oracle is a black box that takes octets and returns octets or a refusal.

   The alternative refused: linking any of them into the library. colibri's decision 89 weighed Wuffs, zlib, zlib-ng, zlib-rs and miniz_oxide as the library itself and refused each: C that every consumer would compile, Rust that every consumer would need a toolchain for, or four CVEs in zlib's decoder and checksums.

9. The RFCs, never another implementation's source. Ruled by the owner on 2026-09-25. stdx is written from RFC 1950, 1951, 1952, 7932, 8878 and 9659, copied unmodified into docs/rfcs/ with docs/rfcs/SHA256SUMS. RFC 9841 is copied so stdx can refuse what it adds (decision 13). Each was checked for a later revision on 2026-09-25, and docs/rfcs/README.md lists the errata that change what a codec does. Every check that exists because an RFC demands it cites the RFC and the section on the line that does the checking (invariant 16).

   Cost: stdx cannot copy a known-good trick from a baseline, and must find each one again from the format and from first principles, then prove it with a measurement. Gain: every rule in the code traces to a sentence in a document the owner can read, and stdx inherits no defect and no licence from another codebase. Decision 18 asks about the one place an RFC defines a rule by reference to a document that is not an RFC.

10. Real numbers come from Linux, measured one way. Ruled by the owner on 2026-09-25, as colibri's decision 32 ruled it for colibri. Entry 20 amends it: the Linux hosts are GitHub's hosted runners, and a result is a ratio within one job. Entry 25 adds a literal-heavy text to its corpora. Entry 45 has the benchmarks time the 1 KiB and 16 KiB payloads as slices of the 1 MiB payload.

    • Benchmarks run on Linux alone, with the machine written down beside the numbers: CPU model, core count, kernel, compiler versions. macOS publishes no number.
    • Each result is the median of five runs, with the spread.
    • Every candidate runs in the same harness in the same run, at a pinned version: zlib, zlib-ng, libdeflate, Wuffs, libzstd and brotli, as far as decision 8's rulings admit them.
    • The corpora are Silesia, Canterbury, and HTTP-shaped payloads (HTML, JSON, JavaScript and CSS) at 1 KiB, 16 KiB and 1 MiB (decision 15).
    • Each report gives decoding and encoding throughput, and the ratio per level, and includes the runs where stdx loses.
    • docs/costs.md holds the measured costs that every speed claim is priced against (decision 14).
    • Anything under about 5% is noise until shown otherwise, as colibri's decision 33 states.

    The alternatives refused: best-of-N, which reports the favourable tail of the noise; and a number from the development Mac, which predicts nothing about the Linux hosts stdx's consumers run on.

The decision records

11. The streaming contract every codec shares. Proposed on 2026-09-25, the first decision record the owner asked for. Ruled by the owner on 2026-09-25, after a review of the proposal. The owner accepted the codec module, the overrun into the caller's output past written, and an assertion for a call after done. Amended by the owner on 2026-09-26, at design §8 step 9: an encoder's init clears its hash heads (under State and sizes).

    A seventh module, codec. Decision 6 names six modules. The status a call ends with, the counts it reports and the checked reader and writer are the same for all five codecs, so they need a home that every codec imports and that imports nothing. The proposal is a module named codec. The alternatives refused:

    • A copy of the types in each codec. A caller that dispatches on the HTTP content coding would switch over five Status types that only look alike, and the copies would drift.
    • The types in checksum or in deflate. zstd would then import a module whose concern it does not share, and deflate would stop being the smallest thing a DEFLATE user compiles.
    • The same shape in each codec with a comptime test that the shapes match. The types would still be distinct, so the dispatching caller keeps its five switches.

    The call. Every decoder and every encoder is a struct the caller places. One call:

    // In codec:
    pub const Status = enum { needs_input, needs_room, done };
    pub const Progress = struct { consumed: usize, written: usize, status: Status };
    pub const Flush = enum { none, flush, finish };
    
    // In each codec, for example gzip:
    pub fn init(decoder: *Decoder) void;
    pub fn decode(decoder: *Decoder, input: []const u8, output: []u8) Error!Progress;
    
    pub fn init(encoder: *Encoder) void;
    pub fn encode(encoder: *Encoder, input: []const u8, output: []u8, flush: codec.Flush) codec.Progress;

    • consumed counts the octets of input the call took. The caller never presents them again: a call that returns needs_input took every octet of input, even when they end in the middle of a code, and the state holds the partial code.
    • written counts the octets of output that hold decoded or encoded data, from output[0]. A call may write any octet of output while it works, so the octets past written are scratch (decision 16); only output[0..written] carries meaning.
    • needs_input: the call took all of input and the stream is not finished. A caller with no more input has a truncated stream.
    • needs_room: the call filled all of output and has more to write, or input left to process. The caller gives more room, and may give more input.
    • done: the stream ended, and every check its format carries has passed (invariant 11). consumed stops at the stream's last octet. Octets after it stay in input[consumed..] for the caller: trailing data, the next gzip member, or the next Zstandard frame.
    • Read-ahead: a decoder may take up to 8 octets into its bit buffer before it knows it needs them. At the end of every call but one that returns needs_input, it hands back every whole octet it has not used, by leaving it out of consumed, so the caller presents it again. A call that returns needs_input keeps only the octets of the code it could not finish, and all of them belong to the stream. So the octets after the end of a stream are always in the input of the call that returns done, and never in the state.
    • Progress: a call either consumes or writes at least one octet, or returns done, or returns the status an empty slice explains (needs_input with empty input, needs_room with empty output). A caller loop therefore never spins (invariant 8).
    • An encoder cannot fail: every input is valid, and misuse is a programmer error that an assertion catches. Flush.none lets it hold input back to find matches; Flush.flush makes it write everything taken so far so that a decoder can produce all of it, and the stream continues; Flush.finish ends the stream. Once a call passes finish, every later call passes it too, with the input the earlier call did not consume.
    • After done, the caller calls init before the next stream. A decode or encode call after done without init is a programmer error, and an assertion catches it.
    • The call takes no flag for the end of input. Every format marks its own end: the last DEFLATE block, the zlib and gzip trailers, a Zstandard frame's last block, brotli's ISLAST.
    • input and output must not overlap, and an assertion checks it at entry.

    Errors. Each codec has its own error set, with one value per cause, each cited to the sentence that requires the refusal. The set is the union of two named sets, so a caller can tell the two kinds of refusal apart without listing names:

    // In codec:
    pub const Refusal = enum { corrupt, unsupported };
    
    // In each codec, for example zlib:
    pub const Corrupt = error{ InvalidHeaderCheck, InvalidWindowSize, ChecksumMismatch, ... };
    pub const Unsupported = error{PresetDictionary};
    pub const Error = Corrupt || Unsupported;
    pub fn refusal(err: Error) codec.Refusal;

    • Corrupt holds every way the input breaks its RFC: a bad header check, a malformed block, a wrong checksum, a distance past the output.
    • Unsupported holds every valid feature stdx refuses: error.PresetDictionary (RFC 1950 §2.3), error.DictionaryUnsupported (RFC 8878 §3.1.1.1.3), error.WindowTooLarge (decision 12) and error.LargeWindow (RFC 9841 §6).
    • A refusal leaves the state unusable until init.

    State and sizes. The state is a plain value. It holds no pointer, not even into itself, so the caller may move or copy it between calls and continue from the copy (invariant 12). Its size is @sizeOf, known at comptime. Where a format leaves a size to the decoder or to the encoder's level, the type is a comptime function of the choice, with a named instance for the usual one:

    const Http = zstd.Decoder(.{ .window_len_max = zstd.constants.http_window_len });
    const Fast = deflate.Encoder(.{ .level = 1 });
    comptime std.debug.assert(@sizeOf(Http) <= 9 * 1024 * 1024);

    A decoder's init costs a constant. It writes the few dozen octets of state a stream starts from and does not clear the window or any table. No octet of history is read before this stream writes it (invariant 10), so neither uninitialised memory nor the octets of a previous stream in the same state is ever observed. That comparison is what makes a pool of decoders safe to share between messages from different peers, so invariant 10 tests it on every copy path.

    An encoder's init also clears its hash heads: 1 << hash_bits positions of 2 octets, 32 KiB at level 1 and 64 KiB at levels 6 and 9. A head holds the last position with its hash, and each search for a match starts there. A head left by a previous stream, or never written, can name a position this stream has written, so the search would try it, and the output would depend on more than this stream's input (invariant 5). The clear is estimated at about one copy of 32 KiB in docs/costs.md, 568 ns on x86-64 and 393 ns on aarch64, and has not been measured on its own. The alternatives it beat:

    • A sparse set recording which heads this stream wrote, so that init clears nothing: a second dependent load for each search, and a second table as large as the heads.
    • A head checked against the current position alone: a stale head below the position passes the check, so the search still depends on it.

    Whole-buffer helpers. Each codec builds these on the streaming call:

    pub const Whole = struct { consumed: usize, written: usize };
    pub fn decode_all(decoder: *Decoder, input: []const u8, output: []u8) (Error || error{ Truncated, NoSpaceLeft })!Whole;
    pub fn encode_all(encoder: *Encoder, input: []const u8, output: []u8) error{NoSpaceLeft}!usize;
    pub fn encoded_len_max(input_len: usize) usize;

    decode_all makes one streaming call with all the input and all the output, and turns needs_input into error.Truncated and needs_room into error.NoSpaceLeft, the two operational errors CLAUDE.md names. For gzip and Zstandard it calls init and continues while input remains, because a gzip file is a series of members (RFC 1952 §2.2) and Zstandard data is a series of frames (RFC 8878 §3). encoded_len_max bounds the encoded size of any input, so a caller sizes the output of encode_all once.

    What colibri's h11 needs. colibri's owner ruled on it on 2026-09-25, in colibri's decision 91 (colibri commit f925839), after the colibri session read these questions. Each ruling, and what in this contract serves it:

    • At most one compression coding per body, so h11 holds one decoder per message. Instances still compose, one's output feeding another's input, because they share no state (invariant 4).
    • deflate is the zlib container alone, and a raw stream fails its header check. The raw deflate module stays for other callers.
    • No cap on the decoded size. h11 counts written per call, and the output goes into the application's buffers.
    • A gzip body may hold several members, each checked. The decoder reports done at the end of each member, with the octets after it left in input. h11 calls init and continues with them, as decode_all does. Octets that do not start a member fail the next member's header check.
    • Octets after the coded stream make the message malformed. done with consumed shows h11 exactly where the stream ended and what input remains; stdx never swallows it.
    • Two verdicts: a corrupt body, and a feature refused. refusal gives the class of every error.
    • Decoders live in a pool the caller owns, taken per message. The state is a plain struct of comptime size that holds no pointer, so the caller places it in any slot, moves it, and resets it with init (invariant 12).
    • Earlier, the colibri session asked for input split anywhere down to one octet and output room down to one octet. consumed takes every octet and a partial code waits in the state, and the checked path writes one octet at a time when that is all the room there is (decision 16).

    The alternatives refused:

    • Cursor structs in the shape of Wuffs's I/O buffers, where history is read from the part of the output buffer the caller kept. It saves the copy into the window when a caller decodes into one large buffer. It costs a second API shape, unlike the slice-and-counts contract colibri's design §4.1 uses, and a caller must keep old output unchanged. The copy it saves is at most one copy per output octet, and at most 32 KiB per call for DEFLATE; decision 14 prices it. Reopen if step 7's benchmark shows the copy above the noise floor.
    • A decoder that pulls input through a callback, as uzlib and puff do. It inverts control and blocks, which decision 4 forbids.
    • A flag for the end of input on every decoder call. Every format marks its own end, so the flag would carry nothing a decoder uses, and a caller could set it wrong.
    • A window the caller hands to init as a separate slice, so one type serves every window size. The state would then hold a pointer and could move only with its window, and the ring buffer's mask would stop being a comptime constant.
    • An encoder that returns an error union. No input is invalid to an encoder, so the error set would be empty and every caller would handle it anyway.
    • One flat error set per codec, with each caller keeping its own list of which names mean a refused feature. Every caller would copy the list, and a name stdx adds would land in the wrong class until each caller noticed.
    • A fourth status for the end of a gzip member, so the decoder crosses members by itself. The caller would still need to know whether the body may end there, which done already says, and the owner fixed three statuses.

12. The memory each decoder and each encoder level takes, and what happens when a stream asks for more. Proposed on 2026-09-25, the second decision record the owner asked for. Ruled by the owner on 2026-09-25, after a review of the proposal. The owner accepted a Zstandard HTTP window of 2^23 for decoding and at most 8,000,000 octets for encoding, a brotli default of WBITS 24 with every tree's table held, and the refusal of a distance past the window a zlib header declares.

    Decoders. The window is the history a back-reference reaches. The other state is tables, counters, and the buffers a format needs between calls. Budgets are upper bounds. The step that writes each decoder pins its exact @sizeOf with a comptime assert, and bench/ measures each baseline's memory through its allocator hook, never an estimate.

    Decoder	Window	Other state, budget	What a stream that asks for more gets
    deflate	32 KiB, fixed by RFC 1951 §3.2.5	16 KiB of tables and state	It cannot ask for more: no distance exceeds 32,768. A distance past the start of the output is error.DistanceTooFar (RFC 1951 §3.2.3)
    zlib	deflate's	deflate's plus 16 octets	CINFO above 7 is error.InvalidWindowSize (RFC 1950 §2.2). FDICT is error.PresetDictionary (RFC 1950 §2.3)
    gzip	deflate's	deflate's plus 32 octets	Nothing: the gzip header states no window
    zstd, HTTP instance	8 MiB, 2^23 octets (RFC 9659 §3)	128 KiB for a block that spans calls, 128 KiB of literals, 32 KiB of tables	A Window_Size over the instance's limit, or a single-segment frame whose Frame_Content_Size is over it, is error.WindowTooLarge (RFC 8878 §3.1.1.1.2 allows the refusal)
    brotli	16 MiB, 2^24 octets for WBITS 24 (RFC 7932 §9.1)	3 MiB of prefix-code tables for 256 trees of each kind; 17 KiB of context maps	WBITS over the instance's window_bits_max is error.WindowTooLarge (RFC 7932 §12 advises a constrained decoder to check the window against its limit). RFC 9841's large-window signature is error.LargeWindow (RFC 7932 §9.1 calls the pattern invalid; RFC 9841 §6 gives it its meaning)

    What each row rests on:

    • DEFLATE. RFC 1951 §3.3 requires a decoder to accept every distance up to 32,768, so the window is fixed, and §3.2.3 forbids a distance past the start of the output. zlib's CINFO states the window the encoder used (RFC 1950 §2.2), and allows no value above 7, a window of 32 KiB. The decoder keeps 32 KiB whatever CINFO says. A distance past the window CINFO declares is refused as error.DistanceTooFar: RFC 1950 states no decoder rule for it, so stdx fails closed (decision 15), and step 6 records the oracles' verdicts on it.
    • Zstandard, the HTTP instance. RFC 9659 §3 says decoders of the zstd content coding MUST support a Window_Size up to and including 8 MB. It does not say whether a MB is 10^6 or 2^20 octets. The instance holds 2^23 octets, which covers both readings. A Window_Size is 2^(10+E) plus a multiple of 2^(7+E) (RFC 8878 §3.1.1.1.2), so the representable values nearest 8,000,000 are 7.5 MiB and 8 MiB, and a limit of 8,000,000 would refuse a frame that RFC 9659's other reading requires. The encoder takes the other side: its HTTP levels never require more than 8,000,000 octets.
    • Zstandard, a larger instance. zstd.Decoder(.{ .window_len_max = ... }) takes any power of two, for a caller outside HTTP that wants more. The window and the size of the type grow with it. Nothing else changes.
    • Zstandard block buffer. A compressed block's sequences are read backward from the block's last octet (RFC 8878 §3.1.1.3.2.1.2), so a block must be whole before its sequences decode. When the caller's input holds the whole block, the decoder reads it in place; when a block spans calls, the decoder copies it into a buffer of Block_Maximum_Size, 128 KiB (RFC 8878 §3.1.1.2.4).
    • brotli. RFC 7932 §1.4 and §9.1 require a decoder to accept WBITS up to 24, so the default instance holds 16 MiB. A caller may build a smaller instance with brotli.Decoder(.{ .window_bits_max = 22 }), which refuses larger streams and no longer decodes every compliant stream. RFC 7932 §12 advises a decoder to grow its window as the stream grows. Without a heap, stdx cannot grow, but init touches no window octet, so a caller that maps the window lazily commits only the pages a stream writes.
    • brotli tables. A meta-block may carry up to 256 prefix codes each for literals, insert-and-copy lengths and distances (RFC 7932 §9.2, NTREESL, NBLTYPESI and NTREESD). A decoder that builds tables only for the tree a block switch selects would rebuild one per block switch, and a stream could switch every symbol, so the budget holds all of them. The exact worst-case table size for an alphabet, a root width and RFC 7932's canonical codes is computed by a tool at the brotli decoder's step and pinned there. The 3 MiB above assumes 8 root bits, about 2.5 KiB per 256-symbol table, and is the number that step must beat.

    Encoders. Every encoder keeps a window of input (the history its matches reach plus the input it has not yet encoded), its match finder's tables, and the symbols of the block it is building. It writes a block into the caller's output as it goes, resuming where it stopped, so it holds no copy of its own output, except where a format needs the block's size before its content (Zstandard's Block_Size, RFC 8878 §3.1.1.2.3). Decision 13 proposes the levels.

    Encoder level	Window of input	Match finder	Block state	Budget
    deflate 1: greedy, one probe	64 KiB: 32 KiB history, 32 KiB ahead	16,384 hash heads of 2 octets: 32 KiB	16,384 symbols of 4 octets: 64 KiB, and 3 KiB of counts and codes	163 KiB
    deflate 6: lazy, hash chains	64 KiB	32,768 heads and 32,768 chain links of 2 octets: 128 KiB	67 KiB	259 KiB
    deflate 9: lazy, longer chains	64 KiB	128 KiB	67 KiB	259 KiB
    zstd 1: one hash table	512 KiB window plus 128 KiB ahead	2^16 entries of 4 octets: 256 KiB	128 KiB of literals, 512 KiB of sequences, 128 KiB for the block's compressed octets	1.6 MiB
    zstd 3: two hash tables	2 MiB plus 128 KiB	2^17 and 2^16 entries: 768 KiB	768 KiB	3.6 MiB
    brotli 1: one pass, no context modeling	256 KiB	2^15 entries of 4 octets: 128 KiB	256 KiB of commands	0.6 MiB
    brotli 5: hash chains and literal context modeling	4 MiB	2 MiB	2 MiB of commands and histograms	8 MiB

    Each level also takes a smaller comptime window, zstd.Encoder(.{ .level = 3, .window_log = 17 }), which shrinks the window and the tables with it for a caller whose messages are small. The brotli rows are the least certain: the brotli encoder's step writes its own record before any code, and may change them.

    The alternatives refused:

    • A decoder that sizes its window to what each stream declares. With no heap, that means a caller-supplied window per stream, which is the separate-slice API decision 11 refused.
    • A Zstandard HTTP instance of 8,000,000 octets. It refuses frames of 8 MiB, which one reading of RFC 9659 §3 requires a decoder to accept.
    • A brotli default below WBITS 24, which saves memory and fails RFC 7932's compliance clause.
    • A brotli decoder that rebuilds a table on each block switch, with a cache of built tables. It saves most of 3 MiB, and a stream that switches on every symbol makes it rebuild a table per output octet.

13. The scope and order of version one. Proposed on 2026-09-25, the third decision record the owner asked for. Ruled by the owner on 2026-09-25, after a review of the proposal: the DEFLATE encoder second, the levels as proposed, and nothing further into version one. The owner's brief proposed: the DEFLATE family decoder, then the Zstandard decoder, the brotli decoder, the DEFLATE encoder, the Zstandard encoder and the brotli encoder. This entry proposed one change: the DEFLATE encoder moves to second.

    The order.

    1. The gzip, zlib and DEFLATE decoder: stdx issue 1, which colibri waits on.
    2. The DEFLATE encoder, levels 1, 6 and 9, in all three containers.
    3. The Zstandard decoder.
    4. The brotli decoder.
    5. The Zstandard encoder, levels 1 and 3.
    6. The brotli encoder, levels 1 and 5.

    Why the DEFLATE encoder moves to second.

    • It completes one coding on both sides of HTTP. The HTTP clients in wide use accept gzip, so a server with a gzip encoder compresses for nearly all of them. A client needs a decoder only for the codings it lists in Accept-Encoding, so it can list gzip alone until the other decoders land.
    • It is the cheapest encoder to get right. It reuses what the decoder builds: the canonical codes of RFC 1951 §3.2.2, the bit order, the containers, the checksums, the oracles and the corpora.
    • It is where the encoder checks get built: the round trip through the reference decoders, the output hashes across hosts and modes, and the split independence of invariant 5. Every later encoder reuses them.
    • It widens the decoder's differential inputs. zlib's encoder makes one family of block splits, code lengths and match choices. A second encoder makes another, and stdx's decoder and both oracles must agree on both.
    • The cost: the Zstandard decoder waits one step, and a client that wants zstd responses waits with it.

    Why the rest keep the owner's order. Decoders come before their encoders because a decoder has a fixed finish line: every valid stream decodes, and every invalid one is refused. An encoder's finish line is a ratio and a speed, which the decoders are needed to check. Zstandard's decoder comes before brotli's because it is smaller: no 122,784-octet dictionary and no context modeling, and RFC 9659 fixed its memory for HTTP. The Zstandard encoder comes before brotli's because its fast levels are what a server uses for content it compresses per response, while brotli's strongest qualities are slow and mostly used ahead of time, where any tool will do. If a consumer needs br responses decoded before zstd, steps 3 and 4 swap without touching anything else.

    Inside version one, because the formats require it.

    • Multi-member gzip (RFC 1952 §2.2), multi-frame Zstandard and skippable frames (RFC 8878 §3 and §3.1.2).
    • Flush.flush in every encoder, which a streamed HTTP response needs.
    • encoded_len_max for every encoder.

    Out of version one.

    • RFC 9841: shared dictionaries, large windows and the framing format. The br content coding is RFC 7932, and shared dictionaries serve Compression Dictionary Transport, which the owner left out. The decoder refuses the large-window signature by name.
    • Zstandard dictionaries (RFC 8878 §5). RFC 8878 §6 says content of the registered media type should not use one. A frame that names one is refused.
    • zlib preset dictionaries. RFC 1950 §2.3 requires a decoder to refuse FDICT when the enclosing format defines no dictionary, and HTTP defines none.
    • Dictionary training, multithreaded compression and Compression Dictionary Transport, which the owner left out. Threads also contradict decision 2.
    • Telling raw DEFLATE from zlib in the HTTP deflate coding. RFC 9110 §8.4.1.2 names the zlib format, and stdx fails closed. A caller that wants to accept raw streams can try both.
    • Zstandard's long-distance matching, negative levels, and brotli qualities above 5.
    • Formats that are not the four HTTP codings: Deflate64, LZW (compress), LZ4, Snappy, xz.

    Nothing else is argued in. Entry 27 argues JSON in, at the owner's request.

14. Where the speed comes from. Proposed on 2026-09-25, the fourth decision record the owner asked for. Ruled by the owner on 2026-09-25, after a review of the proposal. The owner chose checksum paths per architecture (x86-64 and aarch64, register operands only) with a table path for every other target and as the oracle, and the predictions stated before any code. Each claim below names the cost it removes, priced against a row of docs/costs.md, and the check that tests it. Each fast path is an A/B against the checked path on the Linux runners, five runs, median and spread (decision 10), and stays only when it wins by more than the noise. The last column names the baselines whose own documentation or published write-ups describe the same technique. It is a list of what to compare against, and nobody confirms it by reading their source (decision 9). A dash means none of them documents it that stdx found; the benchmark compares against all of them either way.

    DEFLATE decoder:

    Claim	Cost it removes (docs/costs.md row)	Test	Baselines documented to do it
    S1. A 64-bit bit buffer, refilled with one unaligned 8-octet little-endian load and no loop	A branch per octet refilled; one refill per several symbols (64-bit refill)	A/B against the octet-at-a-time refill of the checked path	libdeflate, Wuffs
    S2. Primary tables of 11 bits for literal/length and 8 for distance, whose entries carry the base and the count of extra bits, so most symbols take one lookup	A second lookup and its branch per symbol (L1 hit, branch mispredict)	The fraction of one-lookup symbols per corpus, counted in a test build; A/B against 9 and 6 bits	libdeflate; zlib documents 9-bit root tables
    S3. Two literals from one lookup when both codes fit the table width	One lookup and one data-dependent branch per literal pair (branch mispredict)	A/B with the pairing off	None of the baselines documents it
    S4. Match copies of 8 or 16 octets at a time, overrunning into the output's scratch room; a fill for distance 1 and a repeated pattern for distances under 8	A branch per copied octet (memcpy of 64 octets)	A/B against the octet copy of the checked path	libdeflate, zlib-ng (chunk copy), Wuffs
    S5. Decoding straight into the caller's output, with one copy of the call's last 32 KiB into the window at the end of the call, and none at the end of the call that ends the stream	A second write of every output octet (memcpy of 32 KiB per call at most), and the one copy too of a stream that a call decodes to its end	Copies counted per call in a test build; A/B against decoding into the window and copying out	Wuffs
    S6. init writes a few dozen octets and clears no window or table	A 32 KiB clear per stream, which dominates a 1 KiB body (memcpy of 32 KiB)	Start-of-stream cost on the 1 KiB HTTP corpus against each baseline's reset	zlib.h: inflate defers allocating its window to the first call that needs it
    S7. The fixed codes of RFC 1951 §3.2.6 as comptime tables	A table build per fixed block; a small body is often one fixed block	The fraction of fixed blocks in the 1 KiB corpus from zlib's encoder; A/B with the tables built per block	—
    S8. A table build that writes only the entries the code uses	Clearing unused entries; bounds the cost of a stream of tiny dynamic blocks (invariant 17)	Entries written per octet consumed, counted in a test build, on the worst-case generators	—
    S9. CRC-32 by carry-less multiplication (x86 PCLMULQDQ, Arm PMULL) or Arm's CRC32 instructions; Adler-32 with vectors and a deferred modulo	A table-driven CRC-32 runs at about the speed of a fast decode, so it comes close to doubling the cost per output octet; step 4 measures both	Checksum throughput against each baseline; the checked table path is the oracle	zlib-ng, libdeflate, Wuffs
    S10. The checksum runs over each call's output once, while it is still in cache	A second pass over output that left L1 (cache miss)	A/B against checksumming after the whole stream	—
    S11. A length and its distance's code in one literal/length entry, when both fit the table, so one lookup decodes both	The distance's lookup, a second load each match waits on (L1 hit)	A/B with the entries of a length and its distance apart	—
    S12. A length's extra bits in its literal/length entry, when its code and the extra bits fit the table	Reading the extra bits of most matches: a shift and a mask each	A/B with every table built plain	—
    S13. A dynamic block's code lengths (RFC 1951 §3.2.7) read by a loop of their own: the bits from a 64-bit buffer refilled with one 8-octet load, and the code length code decoded by one lookup in a table of 7 bits, its longest code	For each code length symbol, 87 to 143 a block at the mean of entry 45's rows, the canonical code's branch a bit and the checked reader's branch an octet (branch mispredict), and 157 to 165 instructions where a lookup and a shift take a few	A/B with the loop off, on entry 45's rows of slices	—
    S14. A dynamic block's two codes built from a tally its lengths' read keeps: the count of each length in each alphabet, and each length that is not zero with its place. A build places the symbols that have a code and reads no length	Two reads of every code length a build, 258 to 318 lengths a block whatever codes it has, about 20 instructions a length, and the branch on each length of zero (branch mispredict): on the N2 the literal/length code's build takes 5,267 to 6,037 instructions and 1,964 to 2,310 cycles of a member of entry 45's rows	A/B with the tally off, on entry 45's rows of slices	—
    S15. The tail loop, which decodes what the symbol loop's margins leave out, refills with one 8-octet load while 8 octets of input remain, and copies a match in chunks while the room holds what the copy stores (entry 16)	A branch an octet refilled and an octet copied over a call's last 274 octets of room (branch mispredict): with S13 a buffer of the slice's length costs a 1 KiB member 1,210 of its 11,152 cycles on the N2, and libdeflate 447	A/B with the tail's refills and copies an octet at a time, on entry 45's rows of slices	—

    Zstandard and brotli decoders:

    Claim	Cost it removes	Test	Baselines documented to do it
    Z1. The four Huffman-coded literal streams of RFC 8878 §4.2.2 decoded in one interleaved loop	A serial dependency between lookups (L1 hit latency)	A/B against one stream at a time	libzstd
    Z2. A two-symbol table when the literal codes are short	One lookup per two literals	A/B	libzstd
    Z3. The default distributions of RFC 8878 §3.1.1.3.2.2 as comptime tables	A table build per block that uses them	A/B	—
    Z4. Sequence execution with 16-octet copies and overrun, as S4	As S4	As S4	—
    Z5. Sequences read from the caller's input when the whole block is there	A copy of up to 128 KiB per block	Copies counted in a test build	—
    B1. RFC 7932's 122,784-octet dictionary and its transforms as comptime data, checked against Appendix A's CRC-32 0x5136cb04	Generating or loading the dictionary per process	A comptime assert on the CRC-32	Every brotli decoder carries the dictionary
    B2. The context lookup tables of RFC 7932 §7.1 as comptime data	A computation per literal	A/B	—
    B3. Every tree's table built once per meta-block (decision 12)	A table build per block switch	Entries written per octet consumed on a switch-per-symbol stream	—

    Encoders:

    Claim	Cost it removes	Test	Baselines documented to do it
    E1. A 4-octet hash by one multiply and a shift; match length by 8-octet XOR and a count of trailing zeros	A compare and a branch per octet of every candidate	A/B against octet compares	zlib-ng (compare256)
    E2. One code path per level, chosen at comptime	A runtime switch on the level in the inner loop (branch mispredict)	The level's code size and speed against a runtime-dispatched build	—
    E3. The exact bit cost of a stored, fixed and dynamic block computed from the symbol counts before writing, so each block takes the cheapest (RFC 1951 §3.2.3)	Output larger than the stored form of the same input	Ratio per level against zlib at the same level	zlib
    E4. When the first call passes finish with all the input, matches are found in the caller's input with no copy into the window	A copy of every input octet	A/B on the 1 MiB corpora	libdeflate takes whole buffers only
    E5. A 64-bit bit writer that stores 8 octets at a time when the output has the room, resuming octet by octet when it does not	A branch per output octet	A/B against the checked writer	—

    Where stdx expects to win, match and lose, reported all three ways, as colibri's decision 31 does:

    • Win, plausibly: 1 KiB and 16 KiB messages, where starting a stream and crossing the call boundary dominate (S6, S7), and memory per stream, which is fixed, known and placed by the caller. Entry 45 changes the rows those messages are read on: slices in rotation.
    • Match, at best: 1 MiB DEFLATE decoding against libdeflate, which is years deep and needs the whole buffer; checksums against zlib-ng's and libdeflate's vector paths; Zstandard decoding against libzstd.
    • Lose, probably: every encoder's strongest levels against libzstd, Google's brotli and libdeflate's level 12; and anything a baseline does with AVX-512 that stdx has not written.

    The alternative refused: tuning before the checked path exists and is proved. Every claim above is measured against a correct path, never against a guess.

    What design §8 step 7 found. Recorded on 2026-09-26; step 7's entry holds the runs.

    • S1, S2, S4, S5 and S7 beat the noise on both runners and stay. S3 did not: with the pairs off, the median was 1.00 on both runners, and the 1 KiB bodies ran faster, so the pairs left with their code.
    • S10 beats the noise on the N2 and ties on the EPYC 7763. It stays, since a container sees one call's output at a time, and a checksum after the stream is the caller's to run.
    • S7's test as written finds nothing to time: zlib at level 6 writes no fixed block for the HTTP corpus. Step 7 times S7 over zlib's fixed strategy instead.
    • S4's repeated pattern for distances under 8 is not written: few matches take such a distance. Since decision 29, the assembly loops copy a match whose distance is below 16 with one table lookup (TBL on aarch64, PSHUFB on x86-64); the Zig loop still leaves it to the step out of line.
    • The predictions: the 1 KiB bodies beat zlib, libdeflate and Wuffs on the N2 but lose to zlib-ng, and 1 MiB decoding stays behind libdeflate, at a median of 0.69 of its speed on the N2 and 0.66 on the EPYC 7763.

    What issue 13 found. Recorded on 2026-09-28; design §8 step 7 holds the runs.

    • S12 beats the noise. With it off, the N2 runs at a median of 0.97 of all on and the EPYC 7763 at 0.99, and all on wins by more than 5% on 14 and 12 files (run 36380436703). It stays.
    • S11 tied with S12 alone in the Zig loop: 1.00 at the median on both runners. A table that mixes combined and plain entries makes the branch between the two mispredict, and building plain each block whose matches mostly would not combine removed those mispredictions and none of the cycles (runs 36412089015 and 36412091480). The owner kept S11 until the assembly loops of decision 29 measured it again.
    • They did, and it stays. The combination read and branched on every entry of a block's table, in an order no predictor follows, and each extra pass cost the M1 2 to 4% of a text decode, so the tie had hidden S11's worth. Once it visits only the pairs it joins (dac6cd9), S11 off runs at a median of 0.96 of all on on both runners, and all on wins by more than 5% on 16 of the N2's files and 18 of the EPYC 9V74's, off on none (run 36478169575). S12 off runs at 0.97 and 0.92.

    What the small bodies' split found. Recorded on 2026-10-04; design §8 step 7 holds the runs and the parts.

    • The prediction above, a win on 1 KiB and 16 KiB messages from S6 and S7, does not hold on entry 45's rows. stdx's gzip decoder stands at 0.67 to 0.78 of the fastest baseline over 1 KiB slices on the N2 and at 0.82 to 0.85 over 16 KiB slices. zlib writes one dynamic block for all but 381 of the 4,096 slices of 1 KiB and for every slice of 16 KiB, and S6 and S7 take nothing off a dynamic block's header.
    • On the N2 that header takes 60% to 69% of a 1 KiB member's cycles and 19% to 28% of a 16 KiB member's. Reading its code lengths takes 157 to 165 instructions and 0.44 to 0.88 branch misses a code length symbol: about a third of the instructions are the canonical decode a bit at a time, and about a quarter the checked reader's.
    • S13 is the owner's ruling of 2026-10-04 on that read. Design §8 step 7 had measured a lookup table for the code length code on the M1's 1 KiB bodies, found it 5% to 7% slower, and not kept it. Those rows repeated one body, and a CPU's branch predictor learns a header that repeats, as entry 45 found of every small input a row repeated. So that measurement priced the table's build against a decode whose branches cost nothing. The owner ruled that the table be measured again on entry 45's rows, inside a loop of its own under entry 16. The alternatives refused: the table alone, read through the checked reader, which keeps the reader's and the step's own instructions, about 100 a symbol; and the read left as it is.
    • The prediction, stated before the code. On the N2 the code lengths' read falls from 4,339 cycles of one of html's 1 KiB members to about 1,500, and that member's decode from 15,083 to about 12,300; a 16 KiB member's decode falls by about 7%. On x86-64 the read is 30% to 34% of a 1 KiB member's time and 6% to 10% of a 16 KiB member's, and falls as far. No file taken whole moves past entry 20's floor: a block reads its header once in thousands of symbols.
    • S13 measured, the same day; design §8 step 7 holds the pairs under "The code lengths' loop". On the N2 a 1 KiB member of html takes 10.94 cycles an octet where it took 15.39, past the prediction. The rows of 1 KiB slices run 1.41 to 1.46 times as fast in both of the N2's jobs and the rows of 16 KiB slices 1.10 to 1.17; on an EPYC 9V74 and an EPYC 7763, 1.38 to 1.46 and 1.10 to 1.16. The prediction for the files taken whole was wrong: 23 of 26 rose on the N2, by 1.8% to 6.3%, on 2% to 12% fewer instructions. No row fell on any of the three CPUs. stdx over the fastest baseline stands at 0.94 to 1.10 over 1 KiB slices on the N2 and at 0.95 to 0.97 over 16 KiB slices, and on x86-64 at 0.90 to 1.06 and 0.67 to 0.89.
    • The owner ruled three things on those numbers, on 2026-10-04. Invariant 17's count for a dynamic block's header, block_table_work_max, rises from 4,530 to 4,696 for S13's table, and S13 lands. The tail takes entry 16's amendment as written, under S15, to be paired on its own. And the two thresholds that keep S11 and S12 from a short stream, combine_bits_min and combine_input_min, may be measured at other values on branches that never land: with S12 off, html-1m, js-1m and css-1m run at 0.83 to 0.87 of all on on both EPYCs, and no 16 KiB slice ever takes S12 or S11. Neither limit changes on main without the numbers and a ruling, and the rows of 1 KiB slices and the N2 must not lose.
    • S14 is the owner's ruling of 2026-10-04 on the codes' builds, to be paired apart from S13. With S13 the two builds are the largest part left of a 1 KiB member: 6,022 of its 26,721 instructions on the N2. A build reads every length twice, to count it and to place its symbol, and few lengths have a code: at the mean of entry 45's rows, 55 to 71 of a 1 KiB member's 275 literal/length lengths and 80 to 112 of a 16 KiB member's 282. The read of the lengths now counts each one as it writes it and lists those that are not zero, with no branch on the length, and the two builds place the listed symbols. Invariant 17's bound stands: the tally of a length stands for a build's two reads of it. The alternatives refused: both changes in one, whose loss could not be told from the loop's; and the builds left as they are.
    • S14's prediction, stated before its pairs. Counted on the owner's M1, a 1 KiB member of html takes 23,060 instructions with the tally and 25,801 without, and a 16 KiB member 91,048 and 93,040. So the N2's rows of 1 KiB slices run about 8% faster and its rows of 16 KiB slices about 2%, and the files taken whole gain less than entry 20's floor.
    • S14 measured, the same day; design §8 step 7 holds the pairs under "The codes built from a tally of the lengths". After the loop in the same job, the N2's rows of 1 KiB slices run 1.18 to 1.25 times as fast in both jobs and its rows of 16 KiB slices 1.03 to 1.06, past the prediction: over 1 KiB slices the N2 misses 16% to 19% fewer branches, which a count of instructions on the M1 does not show. A Xeon 6973P-C runs the 1 KiB rows 1.17 to 1.27 times as fast, and an EPYC 9V45 1.28 to 1.33 where its baselines' same rows rose 1.03 to 1.08. No row fell on any of the three. The files taken whole gain nothing, as predicted: five of them take 0.3% to 0.5% more instructions, since a long block gives a code to nearly every length. stdx over the fastest baseline stands at 1.10 to 1.34 over 1 KiB slices on the N2 and at 0.98 to 1.02 over 16 KiB slices.
    • The thresholds measured, the same day; design §8 step 7 holds the runs under "The thresholds of S11 and S12 on a short stream". S12 from a stream's first block runs three of x86-64's four rows of 16 KiB slices 2.0% to 5.9% faster and its rows of 1 KiB slices 2.6% to 5.1% slower, and all eight rows 2.8% to 5.8% slower on the N2. S11 and S12 in a call of 1,024 octets of input run x86-64's 16 KiB rows 1.0% to 14.3% slower. Neither is kept, and both limits stand. What holds x86-64's 16 KiB rows at 0.76 to 0.93 of the fastest baseline is still open.
    • S15 measured; design §8 step 7 holds the pairs under "The tail's words and chunks". After main in the same job, the rows of 1 KiB slices run 1.06 to 1.07 times as fast on the N2 in three jobs, on an EPYC 9V74 and on an EPYC 7763, and the rows of 16 KiB slices 1.01 to 1.02. The N2 takes 5.6% to 6.4% fewer cycles an octet over 1 KiB slices. One file ran slower in one job, E.coli at 0.952 on the 9V74, on functions the change leaves instruction for instruction as they were: placement, by the comparison of the two programs' objects.
    • Where the four leave entry 45's rows, by one pair of main with S14 and S15 against main. stdx over the fastest baseline stands at 1.20 to 1.47 over 1 KiB slices on the N2 and at 1.02 to 1.07 over 16 KiB slices, where it stood at 0.67 to 0.78 and 0.82 to 0.85. On an EPYC 9V45 it stands at 1.22 to 1.35 and 0.77 to 0.96, and on an EPYC 7763, in two pairs of the tip that lands against main, at 1.16 to 1.40 and 0.885 to 0.968, where it stood at 0.68 to 0.75 and 0.71 to 0.76. No row fell in both jobs of a CPU. x86-64's rows of 16 KiB slices stay open: the header takes 9% of such a member there, and what follows it takes longer than libdeflate's whole decode.
    • The owner ruled on 2026-10-11 that S14 and S15 land. The reports of the two threshold probes and of one noisy job of S15's pairs stay with their runs, which design §8 step 7 links, and are not filed in bench/results/.
    • S5 changed by the owner's ruling of the same day: the call that ends the stream copies nothing into the window, which no call reads after done. Called apart on the N2, that copy takes 2.3% to 3.6% of a 16 KiB member's cycles and under 1% of a 1 KiB member's. Measured the same day in two paired jobs a runner (design §8 step 7): over 16 KiB slices stdx's gzip decoder ran at 1.021 to 1.035 of main's speed on the N2 and at 1.016 to 1.027 on an EPYC 7763, and no row ran slower in both jobs.

15. The checks. Proposed on 2026-09-25, the fifth decision record the owner asked for. Ruled by the owner on 2026-09-25, after a review of the proposal. The owner chose to fail closed where an RFC lets a decoder choose, and let a verdict entry land when it cites its RFC section and carries its mutation results, with no separate ruling per entry. The owner chose the payload sources below.

    The corpora. Each is a lazy package pinned by hash, fetched by tools/ and bench/ and never by the library:

    • Silesia, the 12-file corpus at https://sun.aei.polsl.pl/~sdeor/corpus/silesia.zip.
    • Canterbury: cantrbry.tar.gz and large.tar.gz from https://corpus.canterbury.ac.nz; since 2026-09-29 from the corpora release of this repository, a copy of the two tarballs that the pinned hashes verify, after canterbury.ac.nz put its site behind a bot-management rule that turns Zig's fetch away (every run failed at the fetch from 03:18 UTC that day; the copies came through the Internet Archive). The owner ruled the mirror the same day.
    • HTTP-shaped payloads: HTML, JSON, JavaScript and CSS, each cut to 1 KiB, 16 KiB and 1 MiB by a tool, from files whose licences let stdx fetch and run them. The owner chose the sources on 2026-09-25: the WHATWG HTML Standard's single page (CC-BY 4.0, about 13 MB); the Unicode CLDR JSON data (Unicode licence); three.js's three.module.js (MIT); and Bootstrap's CSS files (MIT), concatenated and repeated to reach 1 MiB. Each comes from its npm tarball where it has one, so the hash pins an archive. Bootstrap's 1 MiB piece repeats about 500 KB, a distance DEFLATE's 32 KiB window cannot reach and Zstandard's and brotli's can, so its ratios favour those two, and each report says so beside the number.
    • Entry 25 adds a literal-heavy text, derived from Silesia's dickens rather than fetched.
    • Entry 45 amends how the benchmarks take the HTTP payloads: a 1 KiB or 16 KiB piece is timed as the slices of its 1 MiB piece, in rotation. The checks read the pieces as before.

    Decoders against the oracles. tools/oracle/ runs the same inputs through stdx and through every ruled oracle, and requires byte-identical output.

    • Inputs, DEFLATE family: every corpus file, encoded by zlib at every level (0 to 9) and every strategy (default, filtered, Huffman-only, RLE, fixed), in all three containers. A seed draws windowBits (9 to 15), memLevel (1 to 9), and flush points of every kind zlib offers, which put empty stored blocks and odd bit positions in the stream. Once stdx's encoder exists, its outputs join the inputs.
    • Inputs, Zstandard and brotli: every corpus file, encoded by the reference encoder at every level or quality, with the window, checksum and content-size options a seed draws.
    • Splits: a seed draws the size of each input piece and each output room from a mix of 0, 1, 2 to 16, 17 to 4096 and the rest, so every stream is decoded one octet at a time somewhere, and with empty calls between. At a seeded call, the harness copies the state to another address and continues from the copy (invariant 12).
    • Each stream is decoded three ways: stdx's streaming call under the seeded split, stdx's whole-buffer helper, and each oracle. Output, verdict and the consumed count at done must agree.

    Encoders against the reference decoders. Every stdx encoder output decodes to its input through each ruled reference decoder and through stdx's own. Every level, every corpus file, a seed's flush points and a seed's splits. Beside the round trip:

    • The same input at the same level and flush points gives the same octets under every split (invariant 5).
    • The SHA-256 of each encoder output over the corpora is committed, and must match on macOS arm64 and Linux x86_64, in Debug and in ReleaseSafe.
    • encoded_len_max bounds every output, incompressible input included.
    • After Flush.flush, decoding the output so far gives exactly the input so far.

    Corruptions. A seed turns valid streams into invalid ones: flipped bits, from 1 to 8, at offsets weighted toward headers and trailers; a cut at every offset for streams under 4 KiB and at seeded offsets above; wrong checksums and sizes; header fields set to values the RFC forbids (CM, CINFO, FDICT, reserved flags, BTYPE 11, extreme HLIT and HDIST, the reserved bit of a Zstandard frame header, WBITS patterns); and octets appended after a valid stream. Each input gets a verdict from stdx and from every oracle: accepted with its output, refused, or incomplete (input ended first). Then:

    1. stdx accepts and an oracle refuses: a failure, unless a verdict entry allows it.
    2. stdx refuses and every oracle accepts: a failure, unless a verdict entry allows it.
    3. stdx and every oracle accept: the outputs must be identical.
    4. The oracles disagree: the RFC decides, not a vote. Where the RFC says must, stdx does what it says. Where the RFC lets a decoder choose, stdx refuses, because it fails closed, and a verdict entry records the case.

    A verdict entry lives in tools/oracle/verdicts.zig. It names the codec, the shape of the input, stdx's verdict, each oracle's verdict, the RFC section, and the decision behind stdx's choice. A disagreement with no entry fails the run until an entry is added, and adding one is a change to a check, so it carries its mutation results.

    stdx's verdicts where an RFC lets a decoder choose. These are the entries known before any code:

    • gzip: stdx checks CRC32 and ISIZE, which RFC 1952 §2.3.1.2 lets a decoder skip. It checks the header's CRC16 when FHCRC is set (RFC 1952 §2.3.1), which Wuffs skips. It refuses a reserved FLG bit, as the same section requires. It ignores FTEXT, MTIME, XFL and OS, and skips FEXTRA, FNAME and FCOMMENT without storing them, with no limit on their length but the input's.
    • gzip, whole-buffer helper: it decodes every member (RFC 1952 §2.2) and refuses octets after the last member that do not start another.
    • zlib: stdx checks FCHECK, CM, CINFO and ADLER32, as RFC 1950 §2.3 requires, refuses FDICT, and ignores FLEVEL, which §2.3 allows. It refuses a distance past the window CINFO declares (decision 12), on which RFC 1950 is silent.
    • DEFLATE: an over-subscribed code, a code with no end-of-block symbol, and a code-length repeat with nothing to repeat are refused. An incomplete code is refused, except the single one-bit distance code and the single zero-length distance code that RFC 1951 §3.2.7 describes. Distance codes 30 and 31 and length codes 286 and 287 are refused when they appear, because RFC 1951 §3.2.6 says they never occur.
    • Zstandard: the reserved bit is refused (RFC 8878 §3.1.1.1.1.4) and the unused bit ignored (§3.1.1.1.1.3). A frame whose decoded size differs from its Frame_Content_Size (§3.1.1.1.4) is refused, as §8 warns. An offset of 0, which a repeat offset can produce (§3.1.1.5) and which names no decoded octet (§3.1.1.4), is refused; the RFC does not name the case. Content_Checksum is compared when present, and skippable frames are skipped (§3.1.2).
    • brotli: every "should be rejected as invalid" of RFC 7932 is a refusal, nonzero padding bits included.

    Fuzzing, with Zig's fuzzer.

    • In each codec's module, with no oracle: arbitrary input under a split the fuzzer draws. It checks that nothing panics, that the invariants' assertions hold, that the fast path writes what the checked path writes (decision 16), and, once the encoder exists, that decode(encode(x)) is x at every level and flush pattern.
    • In tools/oracle/, with the oracles linked: arbitrary input through stdx and every oracle, judged by the rules above.
    • The seed corpus is the committed fixtures of each module. A crash or a disagreement the fuzzer finds becomes a fixture and a named test.
    • Zig's fuzzer runs on Linux, so fuzzing runs there, and each step's entry in design §8 records how long it ran and what it found. Entry 26 amends this item: fuzzing runs on macOS arm64 too.

    Worst cases (invariant 17). Generators build valid and invalid streams that maximize work per octet: a DEFLATE stream of minimal dynamic blocks, each forcing a full table build; a Zstandard frame of tiny blocks with new FSE tables; a brotli stream that switches block type every symbol. A test build counts table entries written and symbols decoded per octet consumed, which is deterministic, and zig build test bounds the count. bench/ reports the octets per second on the same streams.

    Mutations. Every check lands with its mutations reported as CAUGHT or NOT CAUGHT (CLAUDE.md).

    The alternatives refused:

    • Majority vote among the oracles. Two oracles that share a lenience would outvote the RFC.
    • Following one oracle's verdict. zlib and Wuffs already disagree on the gzip header's CRC16, and stdx would inherit whichever lenience it followed.
    • Fuzzing without oracles. It finds crashes, and misses a decoder that accepts what it should refuse.

16. Where a hot loop may leave the checked reader and writer. Proposed on 2026-09-25, and ruled by the owner on 2026-09-25, after a review of the proposal: no safety turned off without a measurement and a ruling, and the input-index rule at step 3. It resolves the first conflict the owner named: fast decoders use table-driven multi-symbol lookups, wide bit buffers, word-at-a-time copies that overrun on purpose, and SIMD, while CLAUDE.md sends all parsing through a checked reader and all output through a checked writer.

    The rule.

    • The checked codec reader, writer and bit reader are the reference path. Every codec decodes and encodes through them alone, and each codec's checked path lands and passes every check of decision 15 before any fast path exists.
    • A fast path is a function this entry names, in the table below. Its caller enters it only after checking, once, that at least input_slack octets of input and output_slack octets of output remain. Both are named constants of the codec.
    • Inside, the loop checks the same two margins at the top of every iteration, one compare each, and returns to the checked path when either fails. The margins bound the most one iteration reads (a refill of 8 octets) and writes (a longest match plus its overrun), so every access in an iteration is in bounds by construction.
    • Zig's safety checks stay on (ReleaseSafe). Every slice access is still checked by the compiler, and a wide access is written input[position..][0..8], so one check covers 8 octets. The margin makes those checks predictable branches, and the checks turn a wrong margin into a panic instead of a write past the buffer.
    • A fast path writes the octets the checked path writes, on every input. A comptime field of the codec's options turns the fast paths off, the tests and the fuzzer build both, and the fuzzer runs the two against each other.
    • A fast path copies a long literal run or a long match in chunks of at most chunk_len_max octets, a named constant, so one iteration's writes stay bounded whatever length the stream asks for.
    • Overrun writes stay inside output. A call may write any octet of output, and only output[0..written] means anything (decision 11). Reads stay inside input and inside the history already written (invariant 10).
    • SIMD uses @Vector through the same slices. Inline assembly, for carry-less multiplication or the CRC32 instructions, takes register operands only, loaded through checked slices, and never an address.
    • @setRuntimeSafety(false) appears in one place, the brotli command loop, ruled below on 2026-09-29; entries 35 and 49 let a caller choose it for the json encoder's token loop and for the DEFLATE encoder's loops. An exception needs an A/B of the same function with safety on and off on the Linux runners, five runs each over all three corpora, a gain above the 5% noise floor, a new row in this entry with the numbers, and the owner's ruling.
    • A lint rule, input-index, lands with the codec module (design §8 step 3). Outside the reader, the writer and the functions below, it refuses an index or a slice bound derived from a value the reader produced, as colibri's peer-index rule does for peer input.

    The brotli command loop without Zig's checks. Ruled by the owner on 2026-09-29, after the A/B the rule above asks for: the claim then named loop_checks, off against on in one job on both runners (runs 36542507833 and 36542514694, at 0957f69). The N2 decodes the command-heavy files 1.10 to 1.22 times as fast without the checks, 1.11 and 1.12 at the corpus's median, the 1 KiB bodies 1.03 to 1.06, whose time is the header's, and the all-literal text 1.04; an EPYC 9V74 1.10 at the median, css-1m 1.31 and E.coli 0.90. The proposal came after the by-construction work decision 17 puts first (design §8 step 12): the straight loop in a frame of its own and the cuts around it.

    • Every function under decoder_fast.run, the straight loop, the chain and what they inline, runs with @setRuntimeSafety(false): no bounds, overflow or unreachable check, the loop's own assertions among them. The margins and the per-write checks of this entry and decision 32 stay, as does the checked path, which every refusal still reaches.
    • The claim unchecked_loop of src/brotli/claims.zig holds it, on by default. Off, the checks return: bench-brotli prices them in each run, and the tests and the fuzzer compare both settings with the checked path.
    • Decision 24's measures, once ruled, cover this loop as they cover the assembly loops.

    DEFLATE's tail takes words and chunks while the input and the room hold them. Proposed on 2026-10-04, after the split of design §8 step 7, and ruled by the owner the same day, who read this text before any code and approved it as written. It amends this entry for one function, the DEFLATE tail loop (fast_step.decode_tail), as decision 32 amended it for the brotli command loop.

    • The cost it removes. The symbol loop's output margin, 258 plus 16, leaves the last 274 octets of a call's room to the tail: 27% of a 1 KiB body decoded into a buffer of its own length. The tail refills its bit buffer an octet at a time and copies each match an octet at a time. With S13, on the N2, one dynamic member of html's 1 KiB slices takes 11,152 cycles into a buffer of its own length and 9,943 into one with room past it, where libdeflate takes 10,801 and 10,353. valgrind counts 2,273 of the member's 26,721 instructions in the tail's copies and 771 in its refills.
    • The rule, for the DEFLATE tail loop alone:
      • A refill takes one 8-octet load while at least input_slack octets of input remain, which one compare a refill checks, and whole octets one at a time after that, as now.
      • A match copies in chunks (S4) while the room holds what the copy stores: its length rounded up to a whole chunk, and at least the two chunks it writes first. One compare a match checks that. With less room the match copies an octet at a time, as now, and a match longer than the room goes to the checked path, as now.
      • A literal stores one octet and checks the room for one, as now.
      • Zig's safety checks stay on. Each wide access is a slice of the length it reads or stores, taken after the compare that admits it.
      • The symbol loop keeps its margins and its code, the assembly of decision 29 included.
      • A claim of decision 14 holds it, S15, on by default. Off, the tail refills and copies an octet at a time; the tests and the fuzzer compare both settings with the checked path.
      • Each compare has a test that decodes a stream into every room up to its length, and from every input length.
    • The measurement that admits it: two paired bench-deflate jobs a runner. Entry 45's rows of 1 KiB slices run ahead by more than their spread in both jobs, and no file runs behind by more than entry 20's floor.
    • The alternatives refused: the tail as it is; a smaller output margin for the symbol loop, which would put checks at each write into the loop every large file runs and into the assembly; and slack the caller provides, which this entry refused.

    The fast paths, with the measurement each must show. The DEFLATE rows exist since design §8 step 7, which records their A/Bs. Each row's numbers are filled in by the step that writes it, and a row whose A/B does not beat the checked path by more than the noise is deleted with its code.

    Function	Step	Input slack	Output slack	Measurement that admits it
    DEFLATE symbol loop (S1, S2)	7	8 octets, one refill per iteration	258 plus 16	Throughput against the checked path on all three corpora: a median of 11.68 times on the N2 and 9.29 on the EPYC 7763 in step 7
    DEFLATE and brotli match copy (S4)	7, 12	none	258 plus 16 for DEFLATE; chunk_len_max plus 16 for brotli	As above
    DEFLATE code lengths, in a dynamic block's header (S13)	7	8 octets per refill	none: the lengths go to the state's array, whose size is fixed	Throughput with the loop over main's without it, the change after its base in one job (entry 20), at cbb1353 against 9f728fe (runs 37244993630 and 37244996296): entry 45's rows of 1 KiB slices 1.41 to 1.46 on the N2 in both jobs, 1.38 to 1.44 on an EPYC 9V74 and 1.39 to 1.46 on an EPYC 7763; its rows of 16 KiB slices 1.10 to 1.17, 1.10 to 1.15 and 1.11 to 1.16; and none of the 39 rows slower on any of the three
    DEFLATE tail loop (S15)	7	none: a refill takes 8 octets while 8 remain, and one at a time after	none: each write checks the room it stores into	Throughput with it over main's without it, the change after its base in one job (entry 20), at d8eb2e8 against c33aab1 (runs 37254086506, 37254088899 and 37257457811): entry 45's rows of 1 KiB slices 1.06 to 1.07 on the N2 in three jobs, on an EPYC 9V74 and on an EPYC 7763; its rows of 16 KiB slices 1.01 to 1.02; and one file slower in one job, E.coli at 0.95 on the 9V74, which the objects' comparison of design §8 step 7 shows as placement
    Zstandard literal decoding, four streams (Z1, Z2)	11	8 octets before each stream's position, read backward	none: literals go to the state's literal buffer, whose size is fixed	As above
    Zstandard sequence execution (Z4)	11	none	chunk_len_max plus 16	As above
    brotli command loop	12	8 octets per refill	chunk_len_max plus 16	As above
    brotli code lengths, in the header	12	8 octets per refill	none: the lengths go to the state's array, whose size is fixed	Throughput against the loop off, two N2 jobs a side at 442ea0c against d478f94 (runs 36503381259 and 36503386029 against 36501231721 and 36501237307): the 1 KiB HTTP bodies +2.7 to +4.0%, json-16k +2%, xargs.1 +3 to +5%, no file behind by more than 1% in both; the x86-64 jobs drew different CPUs. The experiment the owner allowed on 2026-09-28; the owner approved the row on the N2 numbers the same day, and the x86-64 pair follows from the workflow's base input
    Every encoder's bit writer (E5)	9, 13, 14	none	8 octets	Encode throughput against the checked writer
    JSON encoder token loop (J11)	18	none: an item's octets are its own slice, read inside it	none: it checks the room an item takes before it writes the item	Throughput of batches against every item through Encoder.run, over bench-json's workloads
    JSON encoder token loop with runtime safety off, at the caller's choice (J11, decision 35)	18	as J11	as J11	Throughput with safety off over on in the loop's two files, run 36515654643: CLDR's texts 1.071 and qlog's records 1.053 on the N2, 1.057 and 1.089 on the EPYC 9V45
    DEFLATE encoder's match finder, appender and symbol writer with runtime safety off, at the caller's choice (decision 49)	9	none: the matcher reads its own window	none: the appender's block, and the writer's 8 octets (E5)	Throughput with safety off over on in the encoder's five inner files, the probe after its base in one job, run 38172161400: levels 1, 6 and 9 at 1.049, 1.044 and 1.067 at the median on the N2 and at 1.044, 1.037 and 1.050 on a Xeon 8573C; since then each bench-deflate report's stdx unchecked column beside stdx
    JSON decoder token loop (J10)	18	none: it checks a token's octets are there before it reads them, and a string's a block of 16 at a time, or of 32 in the AVX2 variant object's walk (entry 47)	16 octets, and 32 in that walk: a string's last block, stored past its end	Throughput of batches (decision 33) against every token through Decoder.run, over bench-json's workloads
    DEFLATE encoder's level 6 lazy loop in aarch64 assembly (decision 53)	9	none: every position lies below the window's end less the lookahead, and a match's compare reads inside the window	none: each symbol goes below the block's room, which the loop tests at each position	Throughput with the loop over main's without it, the change after its base in one job (entry 20), at be101d8 against 5fd03aa (runs 40261381739 and 40261383705): level 6 at 1.064 and 1.068 at the median on the N2, 36 rows faster in both jobs and none slower

    The alternatives refused:

    • The checked reader and writer everywhere, with no fast path. It is the simplest, and colibri's survey shows what it gives up: Wuffs, which has a fast path, decodes at 1.5 to 1.9 times zlib's speed, and the decoders without one run at zlib's speed or below.
    • Raw pointers with safety off in every hot loop, as C decoders do. It is the fastest, and a wrong margin becomes memory corruption. That is the class of CVE-2016-9841 and CVE-2022-37434 in zlib's decoder, which colibri's survey lists.
    • Slack the caller provides: buffers that carry extra octets past the length they report. It changes every caller's contract, and it breaks the output room of one octet h11 asked for.

17. Assertions in production, and what they cost in the inner loops. Proposed on 2026-09-25, to resolve the second conflict the owner named. Ruled by the owner on 2026-09-25, after a review of the proposal: no assertion per symbol in a fast path, and a proposal whenever the safety checks cost more than 5% on any corpus.

    What stays on. The build offers Debug and ReleaseSafe only. In ReleaseSafe the compiler keeps three kinds of check: slice bounds, integer overflow, and unreachable, which is what std.debug.assert compiles to. stdx adds its own assertions, about two per function. One exception stands, ruled under decision 16 on 2026-09-29: the brotli command loop runs without them, its margins and per-write checks in force.

    Where assertions live.

    • At every public call's entry and exit: the state is valid, input and output do not overlap, the counts are inside the slices, and the status matches the counts (invariants 7 and 8).
    • At every fast path's entry and exit: the margins hold, and the bit buffer holds at most 64 bits.
    • Once per block: a built table's counts sum to what the code lengths say, and a block's decoded length fits what its header declared.
    • None per symbol and none per copied octet inside a fast path. The checked path may assert per symbol, because it runs only for the last octets of a call, within the fast path's margins.
    • Arithmetic in the inner loops uses usize positions and a u64 bit buffer, so the overflow checks the compiler keeps are branches that never fire. The wrapping operators appear only where the format itself wraps: CRC and Adler arithmetic, and hash multiplies.

    The measurement, made by each codec's fast-path step on the Linux runners, and written into this entry:

    1. The same benchmark built ReleaseSafe and ReleaseFast. The difference bounds the cost of every safety check together. ReleaseFast is a measuring device only, and never offered to a caller.
    2. A test build that counts the assertions executed per octet decoded. The count times the docs/costs.md row for a predicted branch prices stdx's own assertions apart from the compiler's checks.
    3. When the first measurement shows more than 5% on any corpus, the entry says which checks cost it, and proposes either moving assertions out of a loop or an exception under decision 16. The owner rules on either.

    brotli's measurement, 2026-09-28, at 7520761 in run 36487926168: built ReleaseFast, the decoder runs at a median of 1.17 of its ReleaseSafe speed on the N2 and 1.23 on the EPYC 9V74, and at up to 2.40 on the 1 KiB HTTP bodies, past this entry's 5% (design §8 step 12). The owner ruled the same day on the route: remove the cost by construction first, rewriting the hot code so the compiler proves its bounds and writes nothing it need not, with every check still on; measure on the runners; and propose an exception under decision 16 only for what remains.

    json's measurement, 2026-09-28, counted with perf's counters on the N2 in one run of each build, not decision 20's five (design §8 step 18). Built ReleaseFast, bench_json's stdx takes 16% to 18% fewer cycles a token decoding CLDR's texts and qlog's records, and 29% to 36% fewer encoding them (runs 36508599164 and 36508601413), past this entry's 5%. With stdx's own assertions turned off and the compiler's checks on, it takes 1% to 3% fewer decoding and 6% to 10% fewer encoding (runs 36509679849 and 36509681736): the rest is the compiler's bounds and overflow checks. With safety off in the files of claims J10's and J11's token loops alone, it takes 13% fewer cycles decoding and 11% to 12% fewer encoding (runs 36511601584 and 36511603467). The json fast paths take the route the owner ruled for brotli's. Construction took the decoder loop's share first: it reads and writes slices whose lengths the compiler knows, where indices into the whole input and output cost each access a check (design §8 step 18). The owner ruled on 2026-09-28: once construction has taken what it can, propose an exception under decision 16 for the token loops of claims J10 and J11, with the A/B that entry asks for.

    The DEFLATE encoder's measurement, 2026-10-11, at 0b53229 in run 38170242086, both builds in one job: built ReleaseFast, the gzip encoder runs at a median of 1.095, 1.052 and 1.114 of its ReleaseSafe speed at levels 1, 6 and 9 on the N2, and of 1.077, 1.039 and 1.055 on an EPYC 9V45, past this entry's 5% at levels 1 and 9 on both and at level 6 on the N2. Construction came first there too: level 6's search (entry 48) took a quarter of its instructions with every check on. Entry 49 holds the A/B entry 16 asks for and the owner's ruling.

    The DEFLATE encoder's writer, ruled by the owner on 2026-10-17. The store loop of write_symbols_stored asserted each pair's two code entries, against the rule above. The checked path keeps those assertions. Once a coded block, check_entries requires code bits for each length and distance code the block counts, and as many lengths as distances. The store loop no longer checks that a value lies in its entry's range. A test of encoder_block.zig proves that each value gets the code whose range holds it. Design §8 step 9 holds the pairs.

    Amended by the owner on 2026-10-12, on the wrapping operators. The rule above kept them for where the format itself wraps. The DEFLATE encoder holds others, which landed with its steps (design §8 step 9), each where a bound shows the sum cannot overflow and a checked add costs a branch that never fires. The rule now reads: a wrapping operator may stand where the format wraps, or where a bound stated beside it shows the sum cannot overflow, asserted at comptime where a constant gives the bound. The ones that stand so:

    • The appender's three counts of a block's symbols (Appender.literal, Appender.pair): a block holds at most block_symbols_max symbols, and a comptime assert requires that many and end-of-block's one to fit a count's 16 bits.
    • The chain walk's count of candidates (Walk.step): the walk stops at its budget, at most candidates_max, which fits the count's 16 bits.
    • The tail walk's sum of a link and its shift (best_by_tail): a link is a position of the window, 16 bits, and the shift is less than a match's length.
    • A block's coded bits (coded_bits): each count times its code's length and extra bits, summed over a block's symbols, which a comptime assert requires to fit 32 bits.
    • The block checks' vector sums (encoder_split_price.zig): a block's counts sum to total_max at most, so no lane wraps, neither a count's 16 bits nor a product's 32, and each difference is of counts that are part of the counts they are taken from.
    • The counts of the 3-octet matches a block takes (Three.tally, entry 50): a block holds at most block_symbols_max symbols, each a match of three octets at most, and a comptime assert requires three times that many to fit a count's 16 bits.
    • The two differences of positions that tell how far a 3-octet candidate lies (probe, entry 50): the table names a position before the one asked about, or 0, and that one is not 0, as a block that takes these matches follows a block with a literal and a slide leaves the position past window_len less the lookahead.
    • Made plain adds, the five of the first three cost the default build 1.2% to 4.9% of level 1's instructions on the M1 over seven rows, up to 2.5% of level 6's and up to 2.8% of level 9's, x-ray the most at each. With entry 49's choice off they cost nothing.
    • The alternative refused: the five as plain adds, which follows the rule as it was written and pays those instructions in every default build.

    The alternatives refused:

    • Safety off in the hot functions only, with @setRuntimeSafety(false). It turns every assertion there into an assumption the optimizer relies on, which is the opposite of a check. Decision 16 admits it only function by function, with a measurement and a ruling.
    • A build option that turns assertions off for callers. A caller would ship with it off, and the invariants would stop being checked where it matters.
    • Assertions per symbol in the fast paths. On a 1 MiB body, that is millions of branches per call to check what the margin already proves.

18. XXH64, which RFC 8878 defines by reference. Proposed on 2026-09-25. Ruled by the owner on 2026-09-25, after a review of the proposal: read xxHash's specification document. A Zstandard frame's Content_Checksum is the low 32 bits of XXH64 with seed 0 (RFC 8878 §3.1.1). RFC 8878 cites xxHash by a URL and gives no algorithm, and no RFC defines it. Decision 9 allows the RFCs alone.

    Proposal: read xxHash's own specification document (doc/xxhash_spec.md in github.com/Cyan4973/xxHash), which is a specification in prose and not an implementation's source. Copy it unmodified into docs/specs/, pinned by commit and SHA-256 like the RFCs, and check stdx's XXH64 against libzstd's checksums through the oracle.

    The alternatives refused:

    • Skipping the check. It is what decision 4 names as a defect of Zig's DEFLATE decoder.
    • Reading libzstd's or xxHash's source. Decision 9 forbids it.

19. Every check that needs no fixed machine runs on each push to main. Ruled by the owner on 2026-09-25, during the review of entries 11 to 18, as colibri's decision 47 rules it for colibri.

    • tools/ci.sh runs the lint, the tests, the graph check and the oracle checks, and writes a report. .github/workflows/main.yml runs it on a hosted runner on each push to main. A new check joins tools/ci.sh, never the workflow file, so CI and a person run the same thing.
    • Entry 20 amends this item. Published numbers were to come from a machine of the owner's; the owner then ruled that costs, benchmarks and fuzzing run on the hosted runners too, in jobs of their own.
    • Both land with design §8 step 2.
    • Entry 26 amends this item: main.yml runs tools/ci.sh on three runners, macOS arm64 among them.

    The alternative refused: every check run by hand, with each step's entry recording what was run. A check nobody runs between steps drifts, and the owner asked for CI.

20. Costs, benchmarks and fuzzing run on GitHub's hosted Linux runners. Ruled by the owner on 2026-09-25, when asked which Linux machine decision 10 needs. It amends entry 10, which asked for a machine written down beside the numbers, and entry 19, which kept published numbers off the hosted runners.

    • The jobs run on pinned runner labels, never ubuntu-latest: ubuntu-24.04 for x86-64 and ubuntu-24.04-arm for aarch64, both free for a public repository, with 4 virtual CPUs and 16 GB each. Both architectures are measured, so both checksum paths of decision 14 are.
    • A hosted runner is a virtual machine on shared hardware. Its CPU model can change from one run to the next, other tenants load the host, nobody pins a governor, and it may expose no performance counters. So the method changes in four ways:
      • Every comparison happens inside one job. stdx and each baseline run on the same virtual machine in the same run, interleaved candidate by candidate, so a slowdown lands on all of them alike. A result is a ratio against a baseline in that job. An absolute number from one run is never compared with another run's.
      • Each report records the runner label, the image version, the CPU model and core count from /proc/cpuinfo, the kernel, the Zig version and the run's URL, and is committed under bench/ with that record.
      • The noise floor is the larger of 5% and the spread the job measured for that candidate. A claim of decision 14 must beat it, in the same job, on both architectures it applies to.
      • docs/costs.md is filled from one named run per architecture, in nanoseconds. Cycles are filled only where the runner exposes the counters, and marked unavailable otherwise.
    • Fuzzing runs in scheduled jobs of fixed length per target, within the runner's job time limit. The corpus a run grows is kept between runs, and a finding becomes a committed fixture and a named test (decision 15).
    • Entry 26 amends this item: fuzzing runs on macOS arm64 too. The costs and the benchmarks stay on Linux.
    • Amended by the owner on 2026-09-28, for issue 13's losing DEFLATE files and again for the brotli decoder, in two ways. A change may stay when it beats the target files' own run-to-run spread in two jobs and costs no other file more than the larger of its spread and 1% in both, in place of the 5% floor above; the 1% is for the shifts of code placement, which measured 0.6 to 1.0% on 1 MiB files whose decode never ran the changed code. And a temporary perf-* branch may run the bench workflow for a change's A/B, deleted once its numbers are recorded. The x86-64 runner draws a different CPU from run to run, four models on 2026-09-28, and a ratio against Google's brotli differs by 12% between two of them, so the workflow's base input benchmarks the change's base first in the same job, and an x86-64 pair counts only when one job holds both.
    • Amended by the owner on 2026-09-29, after design §8 step 18's UTF-8 run: a file that moves past the larger of its spread and 1% in both jobs counts as placement, neither the change's loss nor its gain, when the bench program built at both commits runs that file on code identical apart from its addresses. The change is then judged on the code it touches. The proof builds the benchmark for the runner's target at both commits with zig build-exe and the module flags build/ gives it, the C baselines replaced by stand-ins, and compares every function with its addresses masked; a file whose path runs a function that differs still counts. There, content_len_vector shrank by 52 octets and moved the 167 functions after it, the walks among them, by 48 or 52 octets: on the N2, five text files lost 1.2% to 4.3% in both runs and bible.txt encoded 12% to 15% faster, on the walks' unchanged instructions. The alternatives refused: counting such a move as a loss on its architecture, which blocks a change for where the linker placed code the change did not touch; and reading "every job" across both architectures alone, which lets a real loss on one of them pass.
    • Amended by the owner on 2026-09-30, after design §8 step 12's header cuts for the small bodies: a file whose ratio to the baseline moves past the larger of its spread and 1% in both jobs is not the change's loss when stdx's own speed on that file stays within its spread in every job and the baseline's speed moved instead. The proof splits the move into both decoders' speeds, the change phase's over the base phase's, in every table of the job that times them. There, E.coli's ratio to Google's brotli fell 1.4% and 1.5% in both N2 runs of the cuts (36648491985 and 36648500622), and 1.8% and 2.0% in the fill's own pair, because Google's decoder ran 1.4% to 2.3% faster in the job's second phase; stdx's E.coli speed moved under 0.4% in the decoding table, the fast path's and the ReleaseFast table, where Google's own speed held within 0.3%. The alternatives refused: a one-time exception, which leaves the next change to argue the same case; and holding the change for a third pair, which would time Google's second phase again.
    • Amended by the owner on 2026-09-30, after the brotli benchmark's detection fix: a change to a benchmark's harness alone, with the library's functions identical apart from their addresses on a runner, is judged there on the library, so the harness's own functions are not the function that differs of the placement amendment above. There, the harness stopped running codec.Features.detect() for each decode (13dc23a): x86-64's small bodies gained 7% to 62% in both jobs of runs 36676450570 and 36676457924, while on the N2 xargs.1 fell 1.5% and 1.6%, its library functions identical, stdx's unchanged checked path moving the same 1.1% to 1.3%, and only run_once, two instructions shorter, differing. The alternatives refused: landing it as a one-off, which leaves the next harness change to argue the same case; and holding it, which keeps charging stdx a cost a caller pays once.
    • Amended by the owner on 2026-09-30, after the brotli decoder's context map loop: a change that only moves code, every function's body the same apart from its addresses, need not win; it lands when its pair shows no loss past placement, and a change measured on top of it is judged on its own code. There, the context map loop won on both runners and lost world192.txt on the N2 on hot code it had only moved; 3abeeda started the decoder's hot functions on 64-octet lines, as the json module's kernels are, and its pair (runs 36733153860 and 36733165581) moved no file in both jobs. The alternatives refused: amending the placement rule for files whose changed function runs only in the header, which leaves every later change moving the hot loops; and holding the map loop.
    • Amended by the owner on 2026-09-30, after design §8 step 11's fill of repeated blocks: a change whose code no corpus file runs is judged on a probe, inputs that run that code, which a branch that never lands adds to the benchmark beside the corpus, in the base and the change alike. The change stays when the probe's inputs win past the larger of their spread and 1% in two paired jobs on each architecture, and no corpus file loses by its ratio to the baseline in both. There, libzstd writes no repeated block (RFC 8878 §3.1.1.2.2) for any corpus file at level 3, so no file ran the fill that codec.fill took over; the probe's 1 MiB of zeros decoded at 3.40 and 3.04 times its speed on the N2 and at 1.79 and 1.78 on an EPYC 7763, in runs 36669197603 and 36669203656. The alternatives refused: keeping the probe's inputs in the benchmark for good, which times inputs outside decision 15's corpora in every run; and leaving the fill as it was, an octet at a time on Linux for every repeated block.
    • Amended by the owner on 2026-10-03, after design §8 step 18's walk of the grammar: a file that moves past the larger of its spread and 1% in both jobs also counts as placement when callgrind counts the functions that differ at under 1% of the file's instructions at both commits, and every other function the file runs is identical apart from its addresses. There, the walk rewrote take, which every decode runs, so the amendment of 2026-09-29 exempted no file. On the N2 three hex strings of 512 KiB lost 1.2% to 3.3%, and alice29.txt as a string 2.1% and 2.2%, in both runs (37162989866 and 37162995161). callgrind counted 688,756 and 688,757 instructions a hex decode at the two commits, 316 and 424 of them in take, and 528,014 and 528,042 for alice29.txt, 268 and 308 in take and 527,440 in copy_rest, identical. In the same jobs the candidate with the token loop off, whose code is unchanged, lost 1.9% to 2.9% on the three hex rows, and the 39 hex rows' geometric mean held at 0.999 and 0.995. The alternatives refused: a third pair, which this binary's layout would repeat; and holding the walk for a layout that keeps those four rows.
    • Amended by the owner on 2026-10-04, after design §8 step 18's stack on an escape's instructions: a row that moves past the larger of its spread and 1% in both jobs counts as the branch predictor's, neither the change's loss nor its gain, when the runner's counters put the move in branch misses and the change altered no branch of the code the row runs. The proof counts the row's cycles, instructions and branch misses at both commits with bench-json's --profile on the runner, a probe branch adding the row to the profile's list where it is not there, and compares the branch instructions of each function that differs in the bench program built at both commits. There, 3b2105b took two vector instructions out of the block loop of the encoder's walk and left the function's 177 branch instructions as they were. On the N2 bible.txt as a string encoded at 0.938 to 0.958 of main's speed in four jobs: on 7% fewer instructions, an encode took 10,220 branch misses against 7,093 and 649,002 cycles against 621,586, while the same commit took the decoder's misses on that text from 12,389 to 7,826. The x86-64 runners expose no counters, so the rule exempts no row there: the owner landed the stack with an AMD EPYC 7763's rows below main recorded in the design, beside the baselines, which moved past their own floors on up to 41 of 60 rows in the same jobs. The alternatives refused: keeping the compare on aarch64 alone, which gives up an AMD EPYC 9V45's encoding gains of 4% to 18% and leaves the N2's row as it is; and holding the stack for a form that loses no row on any CPU drawn, which no measurement found, since each change to these loops moved the predictor on some row.
    • Amended by the owner on 2026-10-04, after design §8 step 9's read of short chains: a row that moves past the larger of its spread and 1% in both jobs counts as placement when a null build of the change moves that row as far on the same CPU model. A null build holds the change's code behind a test that never holds, so every file runs the base's instructions, and the code beside them sits where the change puts it. It is paired with the base as the change is, on a branch that never lands. There, the read added 20 instructions on aarch64 to level 6's lazy loop, the one function that differs, and on the N2 five rows ran 1.0% to 2.8% slower in both runs (37243295479 and 37243297116): css's 16 KiB slices at 0.982 and 0.989, fields.c at 0.974 and 0.981, grammar.lsp at 0.972 and 0.983, kennedy.xls at 0.985 and 0.980, and nci at 0.990 and 0.989. The null build ran those rows at 0.994 and 0.982, 0.992 and 0.984, 1.009 and 0.981, 0.990 and 0.985, and 0.992 and 0.989 (runs 37242315624 and 37242317768), level 6 at 0.993 and 0.991 of main's speed at the median, and five other rows more than 1% slower in both runs; at level 1, whose code no build changed, bible.txt ran at 0.981 and 0.982. The owner ruled the five rows placement on those numbers. The alternatives refused: landing the five rows as losses the owner accepts, which leaves the next change to that function the same question; and holding the read for a form that loses no row on the N2, when a build that changes no instruction a file runs moves the rows about as far.
    • Entry 45, ruled by the owner on 2026-10-04, changes the rows the rule reads: a 1 KiB or 16 KiB HTTP body is timed as the slices of its 1 MiB payload, in rotation, where a row repeated one file. A small-body row of a report published before it compares with none after it, and the rule still reads the rows of the seven files of 256 KiB or less that a row repeats.

    Cost: no absolute number is stable from run to run, and the noise floor can sit above 5% on a busy host, so a small gain may not be provable. Gain: no machine to keep, both architectures, and a run anyone can repeat from the workflow file.

    The alternatives refused: a machine of the owner's, which the owner declined; and a bare-metal cloud instance, which is steadier and costs money per run.

21. SIMD wherever it measurably speeds a loop up. Ruled by the owner on 2026-09-25, during design §8 step 2: "use SIMD to accelerate anything that can be accelerated". It amends entry 14, which named SIMD for the checksums alone.

    • Every hot loop gets a SIMD path beside its scalar one. The candidates known now: CRC-32 and Adler-32 (claim S9); match copies, runs of one octet and short repeated patterns (S4, Z4); the copy into the window at the end of a call (S5); stored blocks and raw Zstandard blocks; the replicated entries of a Huffman or FSE table fill (S8, Z3); match-length compares 16 or 32 octets at a time (E1); hashing several positions at once in the encoders' match finders; Zstandard's literal copies; and brotli's word transforms, which change the case of up to 24 octets (RFC 7932 Appendix B).
    • A SIMD path is written with @Vector wherever Zig can express it, so one source serves x86-64 and aarch64. Instructions @Vector cannot express, carry-less multiplication and Arm's CRC32, use inline assembly with register operands only (decision 16).
    • The scalar path stays. It is the oracle the fuzzer runs the SIMD path against, and the path on a target without the instructions.
    • "Anything that can be accelerated" is read as anything measurably accelerated. A SIMD path must beat its scalar path by more than the noise, in the same job, on each architecture it applies to (decision 20), or it is removed with its code, as every claim of entry 14 is.
    • docs/costs.md prices a 32-octet vector compare beside the scalar costs, so a SIMD claim states what it replaces in the same units.

    How the instructions are chosen. Ruled by the owner on 2026-09-25: at run time, from what the hardware supports, and not from the build target alone. Two rules shape how, and the owner ruled on each:

    • The caller detects, and passes the result in. codec.Features.detect() reads the CPU once, and the caller hands the value to each codec's init, which keeps it in the codec's state. Invariant 4 forbids a process-wide cache, and detecting in every init would pay for it on every message a pooled decoder starts: CPUID leaves a virtual machine on a cloud host, about half a microsecond. A caller may pass codec.Features.target() instead, what the build target guarantees with no detection, and a test may pass any set, so one machine exercises every path its CPU supports.
    • One object per feature level. Zig 0.16 cannot compile one function for more CPU features than the module's target. So the build compiles each SIMD path's source once per feature level, as a separate object linked into the module, and a codec calls the variant its features allow, once per call and never per octet. The SIMD stays in @Vector code at every level. The levels are x86-64 with SSE4.1 and carry-less multiplication, x86-64 with AVX2, x86-64 with AVX-512, and aarch64 with the CRC32 and PMULL instructions; the target's own level is the module itself.
    • Detection reads no file and makes no syscall. On x86-64 it runs the CPUID and XGETBV instructions. On aarch64 Linux it reads the kernel's hardware capability words with getauxval, which reads the auxiliary vector the kernel wrote into the process's memory at start: Zig's std.os.linux.getauxval, or libc's std.c.getauxval in a program that links libc, where Zig's finds nothing. tools/lint/io.zig allows those two calls and nothing else under std.os or std.c, and src/codec/features.zig is their one caller. On macOS, every aarch64 machine has CRC32 and PMULL. Entry 40 makes src/platform/ a second caller, and allows it sysctlbyname besides. Elsewhere, detect() gives the build target's features.

    The alternatives refused:

    • Comptime from the target alone, which stdx proposed. A consumer shipping one generic x86-64 binary would get SSE2 paths where zlib-ng and libdeflate use AVX2.
    • A write-once global cache of the features, which invariant 4 forbids.
    • Detection in every init, which pays a virtual machine exit per message.
    • Inline assembly for every path above the baseline. It needs no build change, but each SIMD path becomes hand-written assembly per architecture, which is more code to review and test.
    • @Vector at the baseline alone, with assembly only for carry-less multiplication and CRC32. It needs the least machinery and gives up AVX2 and AVX-512.

    What design §8 step 4 found. Recorded on 2026-09-25; the rulings above stand, and these are what applying them to the checksums took. Step 4's entry holds the measurements.

    • Seven levels, not four: x86-64 with PCLMULQDQ; with AVX2; with AVX2 and VPCLMULQDQ; with AVX-512 and VPCLMULQDQ; with AVX-512 and VNNI; aarch64 with CRC32 and PMULL; and aarch64 with DotProd. Each exists because a path on it measured faster on a hosted runner than the level below it.
    • Inline assembly also carries the dot products of Adler-32: UDOT, VPDPBUSD, VPMADDUBSW with VPMADDWD, and VPSADBW. @Vector cannot write a sum of products into wider lanes, and the shuffles that describe a pairwise sum measured slower than the scalar-reduction path they were meant to replace.
    • The level objects are compiled by LLVM in every build mode: Zig's own x86-64 backend, which Debug builds use on Linux, cannot place a 512-bit operand of inline assembly.
    • The checksum module imports nothing (design §3), so it names the fields of codec.Features it reads in a Features of its own, and the caller copies them.
    • A path is chosen by length as well as by features where the measurement asks for it: CRC-32 below one AVX-512 step takes the VPCLMULQDQ object.
    • Detection reads libc's getauxval in a program that links libc; Zig's reads a vector only Zig's own start code fills.

    What design §8 step 10 found. Ruled by the owner on 2026-09-26: where the same instructions measure faster on one vendor's cores and slower on another's, a path is chosen by the vendor as well as the features. Step 10's entry holds the measurements.

    • XXH64's AVX-512 path holds its four accumulators in one 256-bit register and multiplies them with VPMULLQ. From 16 KiB it ran at 1.27 times the scalar path on an AMD EPYC 9V74 runner and at 0.47 on an Intel Xeon 6973P-C, whose VPMULLQ takes about 15 cycles to AMD's 3. Both CPUs report the same AVX-512 features.
    • codec.Features.vpmullq_fast holds on an AMD CPU with AVX-512, which detection reads from CPUID leaf 0's vendor string. Xxh64Path.fastest takes the AVX-512 path only with it, and only for 1 KiB of stripes or more. runs_on holds on Intel as well, so the differential check and the benchmark run the path there.
    • codec.Features.lazy_loop_fast holds on an Intel CPU, from the same vendor string, and chooses level 6's x86-64 assembly loop (decision 54).
    • avx512 now requires DQ with F, BW and VL, as the AVX-512 level objects are compiled for it.
    • NEON has no multiply of 64-bit lanes, so XXH64 stays scalar on aarch64.

    The alternatives refused:

    • XXH64 scalar everywhere, which gives up the 27% on AMD.
    • The AVX-512 path kept for callers that name it, with fastest always scalar.

22. An offset equal to Window_Size is accepted. Ruled by the owner on 2026-09-26, during design §8 step 11. RFC 8878 §3.1.1.4 says that "all offsets leading to previously decoded data must be smaller than Window_Size". libzstd 1.5.7 writes offsets equal to Window_Size. The differential check found 38 such frames among 669. Each was encoded at level 9 or 19 with a window log of 10, and each held an offset of 1024 in a window of 1024.

    • stdx's decoder refuses an offset larger than Window_Size as error.OffsetTooFar, and accepts one equal to it.
    • Invariant 10 holds. An offset still reaches no octet written before the frame's first. codec.Window keeps the last capacity octets, and Window_Size is at most capacity, so an offset equal to Window_Size reads the oldest octet the window holds.
    • The evidence the owner weighed:
      • libzstd's maintainers say that its decoder accepts any offset inside its history buffer, even past Window_Size, because the check costs speed (facebook/zstd#3482, facebook/zstd#3151). In the first of the two, they also say that a conforming compressor sends only what the format allows. By that rule, these frames come from a defect in libzstd's encoder.
      • Since facebook/zstd#1624, libzstd's strategies from greedy up take match candidates up to the maximum window size. Levels 9 and 19 use those strategies, and levels 1 and 3 do not, which fits where the frames came from.
      • RFC 8878 §5 lets a frame reach its dictionary "as long as the amount of data decoded from this frame is less than or equal to Window_Size". That sentence treats Window_Size as inclusive.

    Cost: stdx accepts frames that RFC 8878 §3.1.1.4 calls invalid, by one octet of distance. Gain: stdx decodes what libzstd writes at level 9 and above for any input longer than its window.

    The alternatives refused:

    • Refusing an offset equal to Window_Size, as RFC 8878 §3.1.1.4 says. stdx would refuse frames the reference encoder writes, which libzstd's own decoder accepts.
    • Accepting any offset inside the window's capacity, as libzstd's decoder does. It decodes no more of the frames libzstd writes, and it lets a frame reach octets past the Window_Size it declared.

23. An assembly sequence loop for the Zstandard decoder, speed first. Ruled by the owner on 2026-09-27, during design §8 step 11, in the owner's words: "let's try assembly, once we match or exceed performance, we can see how to make it safe." The runners had shown the Zig sequence loop at 0.72 of libzstd's speed on the N2 (run 36323029517), at the same instructions per cycle as libzstd and 24 to 67% more instructions (run 36323476089), and three changes to its source that each dropped instructions ran 4% slower.

    • It amends decision 16 for one function: the Zstandard sequence execution fast path (Z4) may run as hand-written assembly that reads and writes memory by address, without the bounds checks Zig adds, inside the margins the checked Zig code sets up. aarch64 comes first, then x86-64.
    • It is an experiment. The Zig fast path stays for every other target and every other claim setting, as the reference the assembly must match: the tests and the fuzzer run the assembly wherever the CPU does, and the differential check compares it with libzstd.
    • Safety comes after speed. Once the loop matches or beats libzstd on the runners, a proposal says how it is made safe, and the owner rules on it. Until then, decision 16's refusal of raw memory access stands for every other loop.
    • Extended by the owner the same day, after the N2 reached 0.94 of libzstd's speed: "use assembly where you can't get the zig compiler to do better", in the Zstandard decoder's hot paths. The target is to beat libzstd on every corpus file on an aarch64 Mac first, the owner's M1 Pro, and port to the other architectures after. Published numbers still come from the runners (decision 20).
    • Extended by the owner to the brotli decoder on 2026-09-28, during design §8 step 12, in the owner's words: "Remember that you can write assembly once you cannot get the zig compiler to do better." Asked when, the owner ruled the same day to record it now and keep to Zig first: the brotli command loop moves to aarch64 assembly only once changes in Zig stop gaining on the runners. The Zig fast path stays the reference, as it does for Zstandard.
    • The condition met for brotli on 2026-09-29, and the loop begun on the owner's ruling that day: the header's cuts left the HTTP bodies at 0.77 to 0.90 of Google's speed on the N2 for 1 KiB, 0.83 to 0.88 for 16 KiB and 0.91 to 0.97 for 1 MiB, with the straight command loop at about 255 instructions per command against Google's about 175 on html-16k (M1 Pro, literals and words excluded), and every Zig cut left in it worth 1 to 3%. The loop follows decision 29's shape: the Zig loop stays the reference and runs on x86-64 and under every other claim setting; words and block switches leave the assembly for Zig at first.
    • The x86-64 port ruled by the owner on 2026-09-30, once the header's cuts left every corpus file at or above Google's speed on the N2 and 16 of 39 below it on an EPYC 9V45 (json-1k 0.57, js-1k 0.61, html-1k 0.65, css-1k 0.69, kennedy.xls 0.72): the straight command loop in x86-64 assembly, in files of its own beside the aarch64 loop's, which stay as they are. As decision 29's x86-64 DEFLATE loop does, it runs where LLVM assembles it and on a CPU with BMI2, which the caller's codec.Features names at init; the Zig loop stays the reference and runs everywhere else. The x86-64 runner exposes no hardware counters, so paired runs alone judge it. The alternatives refused: chasing the small bodies on x86-64 in Zig first, and leaving x86-64 as it stands.

24. How the assembly loops are shown safe. owner Proposed on 2026-09-27, as decision 23 asks once its loops match or beat libzstd on the runners: at 3a45936 the decoder runs at a median of 1.13 to 1.19 of libzstd's speed on the x86-64 runners and 1.01 on the N2 (design §8 step 11).

    What runs without Zig's checks. Five loops read and write memory by address:

    • The Zstandard sequence loop, in aarch64 and in x86-64 assembly.
    • The four-stream literal loops, in aarch64 and in x86-64 assembly.
    • XXH64's stripe loop, on Apple's cores.
    • The brotli straight command loop, in aarch64 assembly (decision 23's brotli extension, landed 2026-09-29) and in x86-64 assembly (its port, landed 2026-09-30), whose access tables their first template files carry.

    Each loop checks, once a sequence or a pass, the conditions that keep its accesses inside the stream, the tables, the literals and the output: the checked path's checks, and the margins the Zig code around it sets up (decision 16). Nothing checks each access. Only the tests check the checks: the loops must write what the Zig fast path and the checked path write, and libzstd's verdicts must hold.

    The proposal. Four measures, none of which adds an instruction to a loop:

    1. An access table in each loop's file: every load and store, the range it touches, and the check or entry condition that bounds it. A commit that changes a loop changes its table, and review compares the two.
    2. Guard pages, in a tool under tools/, where mapping memory is allowed. It decodes the differential check's frames and corruptions with every buffer a loop touches placed between two pages that fault on any access: the input, the output, the window, the literals buffer and the tables. An access past a bound then stops the tool, where octets a loop read or wrote past a bound could otherwise go unnoticed. It joins tools/ci.sh (decision 19) and runs on every runner of decision 26.
    3. Boundary tests: one per check of each loop, with the checked value on each side of its bound, as c6197bf's test does for an output below Window_Size.
    4. The Zig fast path stays the reference on every target: each test and the fuzzer compare the loops' octets and verdicts with it and with the checked path.

    The alternatives refused:

    • Dropping the assembly. Before it, the x86-64 runners measured stdx at 0.69 and 0.80 of libzstd's speed (runs 36328791718 and 36347478310), and the N2 at 0.72 with the Zig sequence loop (run 36323029517).
    • A bounds check on each access in the assembly. It repeats what the per-sequence checks prove, at a cost per octet.
    • A proof of each loop. No tool here reads Zig's inline assembly, and a proof by hand would be the access table with less to check it.

25. A literal-heavy text joins the corpora. Ruled by the owner on 2026-09-27, during design §8 step 11. It amends entries 10 and 15.

    • tools/corpus/shuffle.zig writes the first MiB of Silesia's dickens in the order a Fisher-Yates shuffle draws from a fixed seed, and zig build corpus installs it as shuffled/dickens-1m. Every check and benchmark over the corpora takes it.
    • codec.split.Generator draws the order, so every host writes the same octets. A host that wrote others would fail differential-encode, whose recorded hashes cover the file.
    • The shuffle keeps the text's octet frequencies and breaks its repeated strings: libzstd at level 3 leaves 868,620 of its 1,048,576 octets as literals, with Huffman codes of many lengths.
    • The reason: matches cover most of every other file at level 3, so no file measured claim Z2 of decision 14, which decodes two literals a lookup. The owner keeps Z2 for text that compresses poorly, as long as it costs nothing where it does not apply.

    The alternatives refused:

    • Random dictionary words: the word list differs from host to host.
    • Base64 or hexadecimal text: every literal takes a code of one length, and Z2 pairs no two.
    • A fetched file: none of the corpora holds text that level 3 leaves mostly as literals.

26. The checks and the fuzzer run on macOS arm64 too. Ruled by the owner on 2026-09-27, who asked that stdx be tested on Linux on x86-64 and aarch64 and on macOS on arm64. It amends entries 15, 19 and 20.

    • .github/workflows/main.yml runs tools/ci.sh on ubuntu-24.04, ubuntu-24.04-arm and macos-26 on each push to main, and the fuzz workflow fuzzes each module on the same three. The labels are pinned, as entry 20 pins Linux's: macos-latest moves to each new macOS. macOS 26 is the owner's.
    • Some code runs on macOS alone. There, codec.Features.detect() returns a fixed set, since every aarch64 Mac is an Apple M-series part, and the set's madd_addend_slow makes the Zstandard decoder check Content_Checksum with XXH64's aarch64 path. On Linux the decoder takes the scalar path, and only differential-checksum calls the aarch64 path, directly. Before this entry, only a person's run on the Mac ran the one or the decoder's use of the other.
    • Zig 0.16's fuzzer runs on macOS as well as on Linux, so fuzzing need not stay on Linux, as entry 15 had it.
    • Numbers stay on Linux (decision 10): macOS runs the checks and the fuzzer, never the costs or the benchmarks.

    Cost: a macOS runner has 3 virtual CPUs, so its CI job takes the longest; like the Linux runners, it is free for a public repository.

    The alternatives refused:

    • macOS by hand alone: a check nobody runs between steps drifts (entry 19).
    • macos-latest: the macOS under the checks would change without a commit.

27. A JSON module, json: RFC 8259's encoder and decoder, with RFC 7464's text sequences. Ruled by the owner on 2026-09-28, who asked for it. colibri writes qlog (draft-ietf-quic-qlog-main-schema-14) as JSON text sequences, and today carries its own small JSON writer; the owner wants that code to come from stdx, so that colibri's qlog module imports json. It amends entry 1, which held stdx to compression codecs, entry 6, which listed the modules, and entry 13, which left every format but the four HTTP codings out of version one.

    • json imports codec alone, and no module imports it (design §3, invariant 14). It is exported by name as every module is (entry 6).
    • Every non-negotiable holds as for the codecs: no heap, no I/O, calls that stream and resume, output that is a pure function of the input, and each refusal citing its RFC section.
    • What colibri needs shaped three parts of the API, each one any writer of structured records needs: a whole-buffer writer that fails whole, hex strings, and fixed-point decimals. It shaped nothing else, and no source names it (invariant 15).

    The encoder. A struct the caller places, started by init(framing), and one call: encode(token, input, output), which returns decision 11's codec.Progress.

    • A token is the start or the end of an object or an array, a name, a string, a hex string, a number's text, an unsigned or a signed integer, a fixed-point decimal, a boolean or null. A name's, a string's, a hex string's or a number's octets are the call's input, all at once (Piece.last) or over several calls (Piece.more).
    • needs_input says the token is written, or all of a token's input so far is taken, and the text goes on; needs_room says the output filled first, and the next call passes the same token with the input not consumed; done says the token ended the text.
    • The encoder writes the value separators and the name separators itself, and no insignificant whitespace, so its output depends on the tokens and the framing alone, never on how the caller split the input and the output (invariant 5).
    • A string escapes what RFC 8259 §7 requires and nothing else: the quotation mark, the reverse solidus, and U+0000 to U+001F, in the two-character form where one exists and as \u00 and two lowercase digits where none does. Every other character is written as its UTF-8.
    • It refuses input three ways: a name or a string that is not UTF-8, as error.InvalidUtf8 (RFC 8259 §8.1, RFC 3629 §4); a number's text that is not one number, as error.InvalidNumber (§6); and a container past depth_max, as error.DepthTooLarge (§9). Unlike entry 11's encoders, this one can fail: its input is text in a grammar, not octets that are all valid. A token the grammar does not allow where it comes is a programmer error, which an assertion catches.
    • Numbers come from integers and from fixed-point decimals, never from floating point. A decimal is an integer part and a fraction of a fixed number of digits, so "1234.567" is 1234, 567 and 3 digits: qlog's milliseconds with three digits of microseconds, with no rounding and the same text on every host. A number token writes a caller's own text, checked against §6.
    • A hex string holds two lowercase hexadecimal digits for each octet, the more significant first: qlog's hexstring.
    • The whole-buffer helper, TextWriter, hands each token all of its octets and all the room left, so each token fits whole or the call fails with error.NoSpaceLeft, writing nothing past the buffer. written gives the text. A caller that writes one record into the free part of a buffer keeps it only when written returns, as colibri does.

    The decoder. A pull reader: a struct the caller places, started by init(framing), and one call: decode(input, output, piece), which returns a Progress whose status adds token to decision 11's three. A call returns at most one token, and kind names it.

    • A name's or a string's octets, unescaped into UTF-8, and a number's text are written into the caller's output, over several calls when it fills (needs_room). Numbers are not converted: their range and precision are the caller's to choose (§9), and no floating point enters.
    • piece says whether the input holds the text's last octets. Entry 11 refused a flag for the end of the input, because every compression format marks its own end. A JSON text does not: a number, or the whitespace after the text's value, can go on in the next call, so only the caller knows where the text ends.
    • It follows RFC 8259's grammar strictly and refuses, with a distinct error each, what §2 to §7 refuse. Where the RFC lets a parser choose, it fails closed (entry 15):
      • a byte order mark, which §8.1 lets a parser ignore, is error.ByteOrderMark;
      • an escape of a surrogate that no other completes, which §8.2 leaves unpredictable and UTF-8 cannot hold (RFC 3629 §3), is error.LoneSurrogate;
      • a text deeper than depth_max is error.DepthTooLarge (§9).
    • It reports every member of an object, duplicate names included, in order. Refusing a duplicate would take storage for every name of every open object; §4 says names should be unique, not must, and the caller holds the names.
    • error.DepthTooLarge and error.LoneSurrogate are Unsupported, texts RFC 8259 allows that stdx refuses; every other refusal is Corrupt. refusal gives the class (entry 11).
    • The whole-buffer helper, TextReader, gives each token with its octets in the caller's storage, and turns the operational statuses into error.Truncated and error.NoSpaceLeft.

    Text sequences. Framing.sequence delimits a text as RFC 7464 does. The encoder writes a record separator, the text and a line feed (§2.2). The decoder reads one text per init: one record separator or more, then the octets up to the next one or the input's end (§2.1). Its done leaves the next record separator in the input, as a gzip member's done leaves the next member. A text a record separator cuts is error.IncompleteText, and a text whose number or literal name has no whitespace after it is error.UndelimitedValue, which §2.4 requires a parser to drop. §2.1 asks a parser to go on after a text it cannot read; the decoder refuses the text, and a caller that wants the rest calls init at the next record separator.

    The limit. json.constants.depth_max, 1,024 levels, one bit of state each in the encoder and in the decoder: 128 octets.

    SIMD, as entry 21 asks. Each loop below has a vector path of 16 octets, SSE2's and NEON's, the baseline of both architectures, so no feature detection chooses it. The vector paths take a slice and return a count; the encoder and the decoder take the counted octets through codec.Reader and write them through codec.Writer, so no bound goes unchecked (entry 16), and the input-index rule reads them. The scalar paths stay as the reference, and claims.zig switches each claim at comptime, as entry 14's claims are.

    Claim	Cost it removes	Test
    J1. The encoder finds the run of a string's octets that need no escape a vector at a time	A compare and a branch per octet of a string	A/B against the octet-at-a-time path
    J2. The encoder writes a hex string's digits a vector of octets at a time	Two table loads and two stores per octet	A/B
    J3. The decoder finds the run of a string's octets up to the next quotation mark, reverse solidus, control character or non-ASCII octet a vector at a time	As J1, in the decoder	A/B
    J4. The decoder skips whitespace a vector at a time	A compare and a branch per octet of whitespace	A/B on pretty-printed texts
    J5. Inside J1's and J3's runs, UTF-8 is validated a vector at a time, so a run goes on past non-ASCII characters	A state machine step per non-ASCII octet	A/B on text of two- and three-octet characters
    J12. Inside claim J10's loop, two \u escapes that follow each other are read at once, their reverse solidi and us checked in one word and their eight digits turned into two code units in another, and the plain ASCII and the letters' escapes between escaped words stay in the loop	Four table loads and a store by length an escape, and a return to the walk at each word's end	A/B on text of \u escapes
    J13. Inside claim J10's loop, a long string's stretches whose letters' escapes come close together are taken a block of 16 at a time: each escaped letter becomes its character where it stands, and one shuffle a half leaves the reverse solidi out	A stop at each escape and a scan from the octet after it, about 25 cycles an escape on the N2	A/B on a text with an escape every 8 octets
    J14. Inside claim J11's loop, past an escape that ends an ASCII run, a string is written a block of 16 at a time: a plain block as it is, a block with one or two characters a letter escapes with each escape written where it stands (aarch64), and a block with more by one lookup a half, each character written as its letter with a reverse solidus before it	A stop at each octet to escape and a scan from the octet after it, 57 to 62 instructions and a branch the predictor misses an escape	A/B on a text with an escape every 7 octets, and on the corpus's texts

    A claim stays only where bench-json shows its vector path faster than the scalar path by more than the noise, on each runner of entry 20; design §8 step 16 records the runs. J12 joined on 2026-09-30, when the owner chose the \u escapes, and design §8 step 18 records its runs. On 2026-10-04 its loop took in a letter's escape between two \u escapes and went out of line on every target. J13 joined on 2026-10-04 by the owner's ruling, which reverses the rejection of 2026-09-30 that design §8 step 18 records. J14 joined on 2026-10-05 by the owner's ruling; design §8 step 18 records its forms, the two that lost on the N2, and its runs. On 2026-10-09 the owner ruled that on x86-64 its blocks go two a pass, a pair of plain ASCII as it is and any other pair through one lookup, where a pair with no branch on its octets lost. Wider vectors behind entry 21's per-level objects wait until a measurement asks for them.

    The checks, with no oracle. A conformance corpus such as JSONTestSuite and an oracle are each a dependency, which CLAUDE.md asks the owner about; neither is added. In their place:

    • an independent recursive-descent parser in the tests, written from RFC 8259 §2 to §7 and RFC 7464 §2.1 and §2.4, whose verdict and tokens the decoder must match on every seeded and fuzzed input;
    • round trips: seeded and fuzzed lists of tokens encode to a text the parser accepts and decode back to the same tokens;
    • every text under seeded splits of the input and the output with the state moved between calls (invariants 5 and 12), and every vector path against its scalar one;
    • the fuzzer over every property above, on each runner of entry 26.

    The baselines. Ruled by the owner on 2026-09-28, after the first A/Bs: simdjson, yyjson and Zig's std.json time beside stdx in bench-json, in the same interleaved run as the claims' candidates. Each reaches the benchmark through its documented API alone, and nobody working on stdx reads its source (entry 9).

    • simdjson 5.0.2 and yyjson 0.13.0 are lazy packages pinned by hash, compiled in bench/ for the host in ReleaseFast, as the codecs' baselines are. simdjson picks its kernel at run time. simdjson was 4.6.11 until 2026-10-05, when the owner asked for 5.0.2; every number recorded before that day is against 4.6.11.
    • std.json comes with Zig 0.16.0, so it adds no package; the benchmark builds it for the host in ReleaseFast.
    • Decoding visits every value of each text: names and strings unescaped, numbers checked, and the text's end checked. Every baseline must count what stdx counts before any is timed.
    • Encoding writes each text from its tokens. A hex string's digits and a decimal's text are formatted as a caller of that library would, since none has either, and yyjson writes numbers from values, as its API documents no raw number for a document it builds. Each baseline's text must decode, through stdx, to what stdx's own text decodes to.
    • Zig 0.16.0 builds no libc++ for macOS 26, so a macOS host builds the benchmark without simdjson; the benchmark's numbers come from the Linux runners (entry 10).

    The alternatives refused:

    • A JSON writer in each project that needs one, colibri's today. Each copies the escapes and the UTF-8 rules, and each copy drifts.
    • Zig's std.json in the library. Its scanner takes an allocator, and its writer a std.Io writer, which entries 2 and 3 refuse.
    • A decoder that writes every token of a text into one buffer in one call. It crosses the call boundary in bulk, but a caller then parses a second format to read the tokens.
    • A decoder that builds a tree of the text. It needs the heap.
    • Floating point in the encoder or the decoder. The shortest text of a float needs an algorithm no RFC gives, and a decimal of fixed digits writes what a log means.
    • Accepting a byte order mark or a lone surrogate, as RFC 8259 §8.1 and §8.2 allow. stdx fails closed (entry 15).
    • Framing left to the caller, with RS and LF written and split by hand. RFC 7464 §2.4's check needs the decoder to know where a text ends.

    What design §8 step 16 found. Recorded on 2026-09-28; the rulings above stand, and these are what measuring the claims changed. Step 16's entry holds the runs and the numbers.

    • The benchmark first measured a fault of its own. LLVM inlines a function by how many callers it has, and building the workloads called two candidates' codecs a second time: every claim on read CLDR's texts, and every claim off built the other texts. Those two codecs alone then paid a call for each token, which slowed them by 14 to 26% on texts of short tokens, CLDR's and qlog's, on the N2, and every claim's column on those texts carried it. The workloads now take claims no candidate takes, and each candidate's loop inlines its codec. run, each call's loop, is inline in encode_with and decode_with, so a caller's build cannot pay the same call.
    • J4 left with its code on runs that carried that fault. It ran CLDR's texts and qlog's records at 1.14 to 1.18 of all on with J4 off, on the N2, about what the fault alone gave every column there, so those runs cannot show whether J4 wins or loses. It stays out: entry 21 keeps a path only where a measurement shows it faster, and none does.
    • J5 starts only where the scalar path would check a non-ASCII octet: past the encoder's escapes and past the decoder's delimiters. Inside J1's run, as the table has it, it tested the end of each run, and cost the encoder 3 to 4% on ASCII strings on the N2. In the decoder, J5 left, and came back once the fault above was gone: J5 off ran CLDR's texts at 1.18 of all on, but that was the fault's call, and with it gone J5 changes no workload but the text of non-ASCII characters, which it decodes 5 to 6 times as fast.
    • J3's run starts only after an ASCII octet, and only at a plain one. Tested before every octet, it found no run at each character of a non-ASCII text, and the decoder ran such text at 0.71 of the scalar path's speed on the N2.
    • The vectors hold 16 octets on every target. constants.vector_len first took the width the build target suggests, 32 or 64 octets for AVX2 or AVX-512: widths no benchmark measured, chosen from the build target, which entry 21 refuses. At 64 octets, a string shorter than a vector, as most of qlog's are, never reaches one.
    • Every function that takes or returns a vector of bool is inline. LLVM keeps such a vector in AVX-512's mask registers, and its Debug build cannot pass one across a call: it stopped with "Cannot emit physreg copy instruction" in CI run 36377079320. zig build test-avx512 builds every module's tests for an AVX-512 CPU in Debug, since the hosted x86-64 runner draws one only now and then.
    • Zig's own x86-64 backend, a caller's Debug default on x86-64 Linux, indexes a vector only at a lane known at comptime, so first_lane's fallback runs inline for over the lanes.

    The alternatives refused:

    • J5 out of the decoder, as it was from cbea12b to 4721a9f. Text of non-ASCII characters then decoded at 81 MB/s on the N2, where J5 decodes about 600, and no text measured faster without it once the benchmark was fixed.
    • J4 kept, as the faulty runs cannot show it slower either. Entry 21 asks the reverse: a path shows itself faster before it stays.
    • Each target's own width. It needs a measurement of its own, and entry 21's objects to choose it at run time.

28. The json module's state machines, proved in Lean. Ruled by the owner on 2026-09-28, who asked for the proofs design §10 had left open after entry 27. The UTF-8 machine of src/json/utf8.zig and the number machine of src/json/number.zig each have a twin in spec/lean/, a Lake package built with Lean's core alone, and each twin is proved against its RFC's grammar, transcribed one ABNF rule to one definition:

    • Utf8.accepts_iff: from the start, the UTF-8 machine ends between characters exactly on the sequences RFC 3629 §4's UTF8-octets derives. Utf8.refuses_iff: it refuses an octet exactly when no such sequence starts with the octets up to it, so it refuses at the first octet that rules a text out.
    • Number.accepts_iff: from the start, the number machine takes every octet and ends whole exactly on the texts RFC 8259 §6's number derives. Number.refuses_iff is its refuses_iff.
    • Number.accept_taken, ended_spec and invalid_spec: what each verdict of Number.accept tells the decoder, which ends a number at the first octet the machine does not take. A number it ends is whole, and no number goes on with the octet it ends before; after an octet it refuses, no number goes on at all.

    What ties a proof to the Zig code. spec/lean/Vectors.lean writes every step of each proved machine, from every state and for every octet, into src/json/utf8_vectors.txt and src/json/number_vectors.txt. A unit test of each Zig machine replays every line: the same verdict, and the same state after it, and for UTF-8 an unchanged state after a refused octet. The number machine has nine states, and the UTF-8 machine reaches eight from its start, so the replay covers every step either Zig machine can take, and what is proved of the Lean machines holds of the Zig ones.

    • zig build lean builds the proofs through pepegrillo's lean engine, then checks the vector files are what the proved machines give; zig build lean -- write rewrites them. zig build test replays them, with no Lean needed.
    • spec/lean/Stdx/Axioms.lean pins each theorem's axioms with #guard_msgs, so a proof left as sorry fails lake build. Each rests on Lean's standard axioms alone.
    • spec/lean/lean-toolchain pins Lean 4.34.0, the release colibri's proofs pin. On each push, tools/install_lean.sh installs it on the x86-64 Linux runner, checked against the SHA-256 GitHub records for the release, and tools/ci.sh builds the proofs where lake is on the path.

    The alternatives refused:

    • A Zig tool that writes the Zig machines' tables into Lean, with the proofs about those tables. It ties the proofs to the code as tightly, but puts the code under test in the grammar's place: the proofs would start from the code, not from the RFC.
    • Mathlib, whose tactics would shorten the proofs, at the cost of a package of gigabytes for a proof of two small machines; Lean's core, omega included, is enough.
    • The vector UTF-8 check of claim J5 proved equal to the scalar one on every window of four octets, which design §10 first listed with these proofs. The owner asked for the machines; design §10 keeps it open.

29. An assembly symbol loop for the DEFLATE decoder. Ruled by the owner on 2026-09-28, after a review of the proposal, during issue 13, which asks the DEFLATE decoder to beat libdeflate's on both hosted runners. The owner had asked for assembly once the Zig compiler gives no more gains, as decision 23's extension asked for the Zstandard decoder. The Zig loop had stopped gaining:

    • At fbcaf44, stdx decoded gzip at a median of 0.662 of libdeflate's speed on the EPYC 7763 and 0.696 on the N2 (run 36372046166).
    • Five commits to the Zig loop took that to 0.856 and 0.862 at 402c953 (run 36378152449). The next two measured 0.840 and 0.873 at b735d6a (run 36380031615), inside the noise.
    • In that run, the program built ReleaseFast runs the fast path at 1.084 of its ReleaseSafe speed on the EPYC and 1.119 on the N2. Without Zig's checks, the loop would reach about 0.91 and 0.98 of libdeflate.
    • On the N2, stdx executes 1.27 to 1.50 times libdeflate's instructions per octet on the files where it loses most (run 36378148401).

    The rule.

    • It amends decision 16 for one function: the common loop of the DEFLATE decoder's fast path (decode_common in fast.zig: S1, S2, S4 and S11) may run as hand-written assembly. The assembly reads and writes memory by address, without the bounds checks Zig adds, inside the margins the checked Zig code sets up. aarch64 comes first, then x86-64.
    • The assembly takes the symbols the Zig common loop takes, and leaves every other symbol to decode_rare, in Zig: a code longer than the table, a block's end, a match that reaches the window, and a value the checked path refuses.
    • The Zig loop stays for every other target and every other claim setting. It is the reference the assembly must match: the tests and the fuzzer run the assembly wherever the CPU does, and differential-deflate compares it with zlib and Wuffs.
    • Its loads read only the input, the tables and octets this call already wrote (invariant 10), a match whose distance is below a chunk included; its stores write only the room the output margin holds.
    • Decision 24's measures, once ruled, cover this loop as they cover the Zstandard loops.

    The alternatives refused:

    • @setRuntimeSafety(false) in the Zig loop, the exception decision 16 allows. It leaves the code to the compiler, which reaches ReleaseFast's speed at best: about 0.91 and 0.98 of libdeflate.
    • Staying in Zig: the last two commits measured inside the noise.
    • C compiled into the library. Its compiler would be LLVM, as Zig's is, so it would reach about ReleaseFast's speed, and the library would hold a second language.

30. A structural index for the json decoder, and the module's vector paths picked at run time. Ruled by the owner on 2026-09-28, after entry 27's baselines measured stdx at a sixth or a seventh of simdjson's and yyjson's speed on texts of short tokens: a structural index for the decoder, and every vector path chosen at run time, by the caller's codec.Features passed to init as the codecs pass theirs. The design below is proposed with those rulings, and design §8 step 17 measures it before any of it stays.

    The structural index. Each decoder call classifies the next block of its input, 64 octets or what the call holds, a vector at a time, into masks of its octets:

    • the octets inside strings, found from the quotation marks and the parity of the reverse solidi before each, carried from the decoder's state;
    • the structural characters outside strings, and whitespace;
    • the octets that start and end a number or a literal name.

    The call finds its token's start and end with bit scans of those masks, not an octet at a time.

    • The index holds no octet the call has not consumed. A call builds it from its own input, since the streaming contract lets a caller pass other octets past the ones consumed (entry 11).
    • It locates, and checks nothing. A string's, a number's and a literal name's octets go through the checks they go through today, so every refusal and every theorem of entry 28 stands.
    • It is claim J6, switched at comptime as J1 to J5 are, and the decoder without it stays the reference (entry 16).
    • One token per call stays (entry 27). The index removes the scans between tokens, not the call each token costs. On compact texts such as qlog's records those scans are short, and step 17's profile says how much of a token's cost each part is before the index is built.

    Picked at run time. The vector paths of J1, J2, J3, J5 and J6 are compiled once per feature level of entry 21, by build/variants.zig: x86-64 with AVX2, 32 octets a block, and with AVX-512, 64. The module's own target keeps SSE2's and NEON's 16.

    • Encoder.init, Decoder.init, TextWriter.init and TextReader.init take a codec.Features, and each keeps the level those features allow, which each call reads once. A caller passes codec.Features.detect(), read once per process, or codec.Features.target().
    • Each level is claim J7's candidate, timed against the 16-octet paths on the same machine.
    • It changes the API entry 27 gave: each caller adds the argument.

    The alternatives refused:

    • A tape of the whole text's structure, as simdjson builds. It needs the whole text at once and storage that grows with it, which entries 2 and 3 refuse.
    • An index kept from one call to the next over octets not yet consumed. A caller may pass other octets there.
    • The width the build target suggests. Entry 21 refuses choosing instructions from the build target alone, and entry 27 found that width measured by nothing.
    • init keeping its arguments, with an init_with that takes the features. The owner ruled for one way to start each type.

    What design §8 step 17 measured, and the owner's rulings on it, 2026-09-28.

    • J6 is dropped, ruled by the owner. The step's profile put the scans at a quarter of a token's time on the N2, and at 10% to 15% once entry 31's fast paths had cut the calls. An index built within one call does more vector work for a token than the 16-octet scan it replaces, and one kept across calls is refused above. It returns only if a call ever takes many tokens. Entry 33's batches take many; the owner ruled on 2026-09-29 that J6 returns as an experiment over a batch's input, design §8 step 19, landed on its A/B alone. Built that day, it decoded CLDR's texts and qlog's records 36% to 41% slower on both runners, and left with its code.
    • J7 stays, for a name's or a string's run and a hex string's digits. On an AMD EPYC 9V74 it ran the runs of plain ASCII and the hex strings 1.3 to 1.45 times as fast.
    • J5's UTF-8 scan stays at 16 octets. At 64 it ran text of Cyrillic and CJK characters 37% slower, as each escape that stops a run sends the whole block to the scalar path. Entry 47 reopened this for the walks in the AVX2 variant object: the decoder's takes 32 since 2026-10-10 and the encoder's since 2026-10-11.
    • A run takes the 16-octet path for its first 64 octets (constants.wide_run_len_min) before a kernel takes the rest. English text encoded 5% to 8% slower with the kernels from the 17th octet, its lines ending before a call paid for itself.
    • J7 takes AVX2's 32 octets alone, on a CPU with AVX-512 too. On an Intel Xeon Platinum 8573C its 64-octet kernels ran hex strings 14% slower than 16 octets, and English text 5% to 12% slower, where the AMD EPYC 9V74's won and AVX2's won on the EPYC 7763 (design §8 step 18).

31. A fast path for each token of the json decoder and encoder (claims J8 and J9). Ruled by the owner on 2026-09-28, who accepted both after design §8 step 17 measured them; proposed the same day from that step's profile. On the N2, stdx's decoder took 600 to 625 instructions a token, five to six times simdjson's and yyjson's. perf put about a quarter of them in the scans entry 30's index would replace, and the rest in the calls and switches each token passes through.

    The fast path. At the start of a call between tokens, the decoder takes the next token in one straight line, when the input holds all of it and the output has room for its octets:

    • whitespace, and at most one separator;
    • then a structural character, a name or a string of plain ASCII, a whole number with the octet that ends it, or a literal name.

    Every other case returns having consumed nothing and changed nothing, and the checked path takes the call from its start: an escape, a non-ASCII octet, a record separator, a token the input cuts, an output without the room, and every refusal.

    • It reads through the checked reader and writes through the checked writer. It reads each octet the grammar turns on with read_octet, counts each run with a scan of scan.zig or number.zig, and takes the run with take. So entry 16 needs no new exception, and the input-index rule reads it.
    • It leaves the state the checked path leaves, field for field. The tests require the same progress and the same state after every call with J8 on and off, whole and under seeded splits.
    • It is claim J8, switched at comptime as J1 to J5 are. Off, every token takes the checked path, the reference (entry 16).

    J9, the encoder. The encoder's profile had the same shape, with up to five calls of memcpy a token, each copying a few octets held in pending. At the start of a token, when the call's input holds all of its octets and the output has room for every octet it writes, the encoder writes them in one straight line, holding none:

    • the record separator that starts a sequence's text, the value separator before a value, the token, and the line feed that ends a sequence's text;
    • a structural character, a name or a string of plain ASCII, a hex string, a number's text that is one whole number, a number it formats, or a literal name.

    Every other case writes nothing and changes nothing, and the checked path takes the call: an octet to escape, a non-ASCII octet, a token whose octets go on in a later call, an output without the room, and every refusal. It leaves the state the checked path leaves, and the tests require that as they require it of J8.

    The alternatives refused:

    • Inlining the checked path's functions alone. The calls go, but each token still passes the same switches, and what LLVM inlines changes with the callers each build gives it, the fault entry 27's benchmark found.
    • Many tokens a call. The owner kept one token a call (entry 30).
    • A fast path that indexes the input itself, as DEFLATE's does. Entry 16 would have to name it, and the profile did not find the checked reader's cost.

32. brotli's fast path checks the room of each write, and runs to a call's last octets. Proposed on 2026-09-28, during design §8 step 12. Ruled by the owner the same day: the owner chose to amend decision 16 for it, reviewed this text before any code, and approved it as written. The code then revised the rule in two places, the second mode and the copy one octet at a time, and the owner had the revision committed with it. It amends decision 16 for one function: the brotli command loop.

    The cost it removes. Decision 16's output margin for the brotli command loop, chunk_len_max plus 16, is 272 octets, so the checked path decodes the last 272 octets of every call: a quarter of a 1 KiB body, at about 90 instructions an octet. On the owner's M1 Pro, decoding the corpus's 1 KiB HTTP files into an output 512 octets longer than they need takes 15 to 25% fewer instructions and 12 to 19% fewer cycles than into one that holds them exactly, and html-16k 3% fewer of each. Numbers for publication come from the runners.

    The rule. For the brotli command loop alone:

    • The input margin stands as decision 16 sets it: 8 octets, checked where the loop refills.
    • While at least chunk_len_max plus 16 octets of room remain, one check at the top of a chain and one after its literals cover every write the chain makes, as decision 16's margin does.
    • Below that, the loop runs in a second mode, where each write checks the room it stores into, against the most it stores:
      • a literal run stores its batch, which the run bounds by the room left;
      • a copy in chunks (S4) stores its length rounded up to a whole chunk of 16 octets, and at least the two chunks it writes first; with less room it copies its octets one at a time, as with S4 off;
      • a dictionary word's wide transform stores transform.wide_output_len, 64 octets, and its exact transform, near DICT's end, transformed_word_len_max;
      • the rest of a word the checked path started stores what is left of it.
    • A write whose room is short returns to the checked path, as a failed margin does now, so the checked path decodes at most the last write's octets, not 272.
    • Each mode is its own instance of the loop, so the margin's mode compiles as though the other did not exist. The checks at each write cost the common case nothing: on the owner's M1 Pro they cost command-heavy files 3% when every chain made them.
    • Zig's safety checks stay on (decision 16). Each check is one compare against the room the loop already holds in a register, and the slice checks behind it still turn a wrong bound into a panic.
    • Every write's check has a test that decodes a stream into every room up to its length, as the fast path's margin tests do now.

    The measurement that admits it: bench-brotli on both runners, the 1 KiB and 16 KiB HTTP files ahead of the fixed margin by more than the noise, and no corpus file behind it.

    Measured on 2026-09-28, at d238737 against 5bf6b54, two jobs a side (runs 36497674142, 36497679595, 36497685015 and 36497690375). On the N2: the 1 KiB HTTP bodies +4 to +13%, json-1k -0.6%; the 16 KiB bodies tie, json-16k and json-1m -0.7%; grammar.lsp +3%, dickens-1m +2%, the rest within the noise. The x86-64 jobs drew two CPUs, so one pair compared like with like: html-1k +9%, css-1k +7%, css-1m -8%. The json files never run the second mode, decoded whole in one call, so their loss is the margin instance's placement; the owner kept the decision the same day under decision 20's 1% floor, and the x86-64 pair waits for the workflow's base input. Inlining the wide transform into both instances, which two instances had left out of line, gained nothing on the N2 and cost html-16k 2% (runs 36501364058 and 36501370200); dropped.

    The assembly loops, 2026-10-04 (842f015). The loops of decision 23's brotli extension ran in the margin's mode alone and left with 272 octets of room, to the second mode in Zig. They now go on below the margin in the one instance they have: a block the common path never enters takes the room, less the two chunks a copy may store past its length, as the meta-block's octets left, so the loop's checks of RFC 7932 §9.3 are the checks of each write; a dictionary word checks the room its wide transform stores into; and with less room than the two chunks the loop leaves to Zig's second mode, as a write whose room is short does. An exit below the margin hands Zig a phase and no link, since Zig's margin instance checks no room. The margin's mode keeps every instruction in its place, which is what an instance of its own gave the Zig loop. Design §8 step 12 holds the runs and the mutations.

    The alternatives refused:

    • The fixed margin, as decision 16 ruled it: every call's last 272 octets at the checked path's speed.
    • A faster checked path. It is the reference, one octet a step; making it fast writes the fast path a second time.
    • Slack the caller provides, which decision 16 refused.

33. Many tokens a call for the json codecs. Ruled by the owner on 2026-09-28, who lifted the one token a call that entries 27 and 30 kept: "everyone knows batching is good for performance". Design §8 step 17 had put about a third of a decoded token's time on the N2 at the call's boundary once entry 31's fast paths ran: loading and storing the decoder's state, invariant 7's checks at the call's entry and exit, building its Progress, and the caller's dispatch on it. simdjson and yyjson pay that once a text.

    The decoder. Decoder.decode_batch decodes tokens one after another into slots the caller owns, until the slots are full, the input or the output runs out, or the text ends. The output holds the octets of the call's names, strings and numbers one after another, and each slot holds its token's kind, where its octets start in the output and how many there are.

    • A call that ends inside a name, a string or a number fills its last slot with what it wrote, marked as not ended. The next call's first slot holds the rest of that token.
    • Its status adds needs_slots to decode's: every slot is filled, and the text goes on.

    The encoder. Encoder.encode_batch writes a list of items, each a token with its octets as encode takes them, until the list ends, the output is full, or the text ends. It returns how many items it wrote whole and how many octets of the next it took, so the caller passes the rest of the list, and the rest of that item's octets, to the next call.

    Both keep the streaming contract:

    • A call takes what the caller has, says whether it needs input, room or slots or is done, and the next call resumes where it stopped (non-negotiable 3).
    • A token takes at least one octet of input decoding, and writes at least one octet encoding, so a call's work stays bounded by the octets it consumes and writes (invariant 17). Invariant 7's checks run at each batch's exit.
    • A refused text fails the call, as with one token a call. The tokens the call took before the refusal are not reported, since the text is refused whole.
    • decode and encode stay, one token a call, and are the reference: a batch gives their tokens, octets and verdicts, whatever the slots, the items and the splits (invariant 5).

    Entry 30's structural index was dropped for one token a call. Its question reopens with batches, and design §8 step 18 asks it again from its profile.

    The alternatives refused:

    • A callback for each token, through a pointer: a call a token, the cost this entry removes. A comptime callback inlines, but compiles every caller's handling into the decoder's loop.
    • An iterator over a whole text: it needs the whole text in one call, which the streaming contract refuses (non-negotiable 3).

34. The DEFLATE decoder's losing files: smaller gains, a wider table measured, and an x86-64-v3 benchmark. Ruled by the owner on 2026-09-28, during issue 13. At 11b5b8a, stdx decoded gzip faster than libdeflate at the median on every runner CPU drawn: 1.101 of libdeflate's speed on the N2, 1.028 on an AMD EPYC 9V74 and 1.005 to 1.021 on Intel Xeons. It was slower on 13 to 20 of the 39 corpus files, depending on the CPU. The owner asked for every file ahead of libdeflate, and was asked which rules to relax for it. The rulings:

    • A change for the losing files may stay when it beats those files' own run-to-run spread in two jobs and costs no other file more than that file's spread. It amends decision 20's noise floor, the larger of 5% and the job's spread, for this work: the losing files need several gains of 1 to 3%.
    • The fast path's literal/length table may be measured at 12 bits (literal_length_table_bits, decision 14's S2), with the decoder's state budget of decision 12 raised to hold it. The limits change only if the runners' numbers earn it and the owner rules on it.
    • On x86-64, bench-deflate also times the gzip decoders with stdx built for x86-64-v3, beside the build for the architecture's baseline, which decision 21's caller ships: what a caller that builds for its servers' CPUs gets. The program runs where the host has x86-64-v3's instructions.

    Kept as they were: assertions in production (decision 17), the RFCs as the only source (decision 9), and decision 16's margins.

35. Runtime safety off in the json encoder's token loop (claim J11), at the caller's choice. Proposed on 2026-09-28, as decision 17's json measurement asks once construction has taken what it can (design §8 step 18). Ruled by the owner the same day, who asked whether it could be the caller's choice: a caller chooses it at compile time, and every check stays on unless it does. It is decision 16's first exception.

    The measurement. One A/B of the four files of claims J10's and J11's token loops, with runtime safety on, at d2998c4, and off, at 43698a9, where every function of the four starts with @setRuntimeSafety(false): bench-json's workloads, five runs a side, both builds in one job on each runner (run 36515654643). Throughput with safety off over throughput with it on:

    Workload	N2, decoding	N2, encoding	EPYC 9V45, decoding	EPYC 9V45, encoding
    CLDR's texts	1.025	1.071	1.044	1.057
    qlog's records	1.024	1.053	0.987	1.089
    The 17 text files as strings, median	0.995	1.025	1.100	1.030
    The same, lowest to highest	0.931 to 1.022	1.000 to 1.054	0.981 to 1.184	0.985 to 1.209
    The non-ASCII text	0.982	0.998	1.020	1.082

    The larger spread of each token row was under 1% on the N2, but for qlog's decoding at 10.7%, and 2.8% to 8.3% on the EPYC.

    What it shows.

    • The encoder's loop gains 5.3% to 8.9% on the token workloads on both runners, past decision 16's 5% floor. On the EPYC, CLDR's 5.7% is inside that row's 6.0% spread, decision 20's floor for the job.
    • The decoder's loop gains 2.4% and 2.5% on the N2, and −1.3% to +4.4% on the EPYC, under the floor. Its text files gain a median of 10% on the EPYC and nothing on the N2: escapes decoded on x86-64, which design §8 step 18 takes up by construction.

    The ruling.

    • Claims gains encoder_token_loop_runtime_safety, true by default. A caller that sets it false in the claims it passes Encoder.encode_batch_with runs the loop at that call site with Zig's runtime safety checks off in every function of encoder_loop.zig, encoder_loop_string.zig and, since claim J14, encoder_loop_escapes.zig. encode_batch and every other call keep them.
    • A test build, the fuzzer's included, and a Debug build keep the checks whatever the field says. A test of encoder_loop_test.zig requires every @setRuntimeSafety of those files to take the field, and the decoder's loop and the string walk to hold none.
    • bench-json times the encoder with the field false beside every claim on, and puts both beside the baselines.
    • Decision 16 gains a row with these numbers.
    • The decoder's loop keeps every check.
    • It amends decision 17's refusal of a switch that turns checks off for callers, for this loop alone: the switch is a call site's, and stdx's assertions stay on.

    What stays checked in the encoder's loop: Zig turns safety off in the function that says so, and a function it calls, inline or not, keeps its own, so scan.zig, wide.zig and string_walk.zig keep their checks. std.debug.assert keeps its check too, as its unreachable lies in its own function. An unreachable written in the loop's own functions becomes an assumption, as decision 17 warns.

    What it risks: a wrong room check in the loop writes past output instead of panicking. The loop reads the caller's items and not a peer's octets, and it checks the room an item takes before it writes the item (decision 16's J11 row).

    The alternatives refused:

    • The checks off in every ReleaseSafe build, stdx's choice and not a caller's, as first proposed.
    • The caller's choice with the checks off by default.
    • A switch at run time. It compiles the loop twice, and every binary carries both.
    • Both loops. The decoder's gain on the token workloads is under the floor, and the decoder reads a peer's octets, where a wrong margin reads or writes outside the buffers.
    • Neither. Every check stays, and the encoder gives up 5% to 9% on the token workloads.

36. The finder's two rules that spend speed on size: the greedy level's covered inserts, and the lazy step's distance penalty. Ruled by the owner on 2026-09-29, from a comparison of libdeflate's output with stdx's that keeps decision 9: the symbols of each stream, parsed and counted by a scratch tool, and never its source.

    What the comparison found, on the corpus of decision 15:

    • Level 1 emitted twice libdeflate's literals on dickens. A position a match covers never entered the hash heads, so a later match could not start there, and a head named a farther position than the nearest.
    • Levels 6 and 9 paid 13.3 bits a distance on json-1m against libdeflate's 10.6. The lazy step took the longer of its two matches whatever their distances, and a distance 8 KiB back costs 13 more extra bits (RFC 1951 §3.2.5) than one under 32.

    The rules:

    • The greedy level makes every position a match covers, after its first, its hash's head, for a match of covered_insert_len_max octets or fewer, 8 at level 1: the 4 octets slide through a word, one octet loaded a position. Output over libdeflate's, the median of dickens, json-1m, css-1m, html-16k, x-ray, mozilla, E.coli, webster, kennedy.xls and nci: 1.121 to 1.054, E.coli 1.215 to 1.016.
    • The lazy step compares the match waiting from the position before with the next position's by length less lazy_distance_penalty_octets, octets of literals a distance costs: 0 under 32, 1 under 512, 2 under 4096, 3 beyond. Level 6's median goes from 1.017 to 1.007 and json-1m from 1.077 to 0.994; level 9's from 1.029 to 1.026 and json-1m from 1.108 to 1.013. No file grows more than 0.5%.
    • What they cost, run 36559451595 against run 36556604513 on the same CPU models, a Neoverse N2 and an EPYC 7763: level 1 at 0.91 and 0.93 of its speed, the text files 0.80 to 0.88 and E.coli 0.68 and 0.73; level 6 at 0.98 and 0.97, json-1m 0.75 on both; level 9 at 0.98 and 0.99, json-1m 0.63 and 0.66, samba 0.70 and 0.74. The 39 files' median output over libdeflate's: level 1 1.101 to 1.055, level 6 1.000 to 0.999, level 9 1.011 to 1.010.

    The alternatives refused, each measured on the same ten files (speed on the M1, which publishes no number, decision 10):

    • Inserting every covered position at level 1: 1.048, and json-1m a quarter slower, as its matches run long.
    • 15 and 16 hash bits at level 1, 64 and 128 KiB of heads against the 32 of decision 12: 1.048 and 1.047 with the inserts above. Not worth the state.
    • The same penalty inside the chain walk, a farther candidate having to beat the found match by it: nothing on json-1m, E.coli 0.5% worse. The walk visits candidates nearest first and keeps the nearer of two equals already.
    • A lazy_len of 16 or 8 at level 6, the waiting match's length that skips the next search: json-1m 3.9% and 6.5% larger.
    • The four most recent positions of each hash in one row of the heads, with a 13-bit hash so the heads keep their 64 KiB: four times the false candidates, level 6 a fifth slower, level 1's output 5% larger.
    • A second table keyed by 8 octets, another 64 KiB, so one probe finds a long match: skipping the walk after a match of 16 octets or more cost json-1m 3.2%, and after 32 or more the walk ran anyway for the table's insert.
    • A budget of 64 candidates at level 6 instead of 128, its cut 16: the M1 put it at 0.3 to 0.6% larger for 12 to 27% of the time, and the owner asked for the runners' numbers. Run 36568474837, main (bb0d5c8) and the budget in one job: level 6 at 1.12 times its speed on the N2 and 1.15 on an EPYC 7763, no file slower, E.coli 1.7, reymont and json-1m 1.3; output over libdeflate's at the median from 0.999 to 1.000, a median 0.15% larger, 16 of 39 files by more than 0.2%, E.coli 1.6%, reymont 1.2%, xml 1.1%. Ruled adopted on 2026-09-29.

37. A table lookup in the json module's UTF-8 check (claim J5). Proposed on 2026-09-29, from design §8 step 18's non-ASCII rows, once construction in plain vectors had taken what it could. Ruled by the owner the same day: as an experiment first, on an unmerged branch with the A/B below, and landed only on its numbers. At 1247c3e stdx decodes the non-ASCII text at 0.866 of simdjson's speed on the N2 and 0.643 on an Intel Xeon Platinum 8573C, and encodes it at 0.757 and 0.662 (run 36522098273). The encoder writes its input's UTF-8 as it is and the decoder writes a string's as it is, so on that text the check is the work.

    The check today. scan.utf8_error_lanes judges a block of 16 octets with each rule of RFC 3629 §4 as a compare against a splat: about 14 compares, 3 shuffles that bring in the previous block's last octets, and 10 ANDs and ORs, before the stop test every block pays. With the check skipped, an experiment never merged (run 36522631693, branch exp-json-utf8-ceiling against 1247c3e), the non-ASCII text decoded 2.21 times as fast on the N2 and 1.83 times on the Xeon, and encoded 1.94 and 1.87 times as fast, with every other row inside its spread: the check is more than half of the time on that text, and that is the most a faster check can give.

    The proposal. Three lookups of 16 entries each, indexed by the high nibble of the octet before, its low nibble, and the high nibble of the octet itself, ANDed together: a lane left nonzero is an octet RFC 3629 §4 rules out within a window of two octets, which covers C0, C1 and F5 up, a continuation octet after an octet that asks for none, an octet that is no continuation after one that asks for it, and the narrower second octet after E0, ED, F0 and F4. A character's third and fourth octets are judged as today, from whether the octet two or three back asks for them. About 9 vector operations a block against 27. The technique is Keiser and Lemire's, from their paper "Validating UTF-8 In Less Than One Instruction Per Byte" (2021); no implementation's source is read (entry 9).

    • The lookup is tbl on aarch64, in NEON's baseline, and pshufb on x86-64, which needs SSSE3, above the baseline: there it goes in the variant object claim J7's kernels already build for AVX2 (build/variants.zig), at 16 octets, and the baseline keeps the compares. Both are inline assembly with register operands only, as entry 16 admits.
    • The three tables are comptime data built from RFC 3629 §4's rules in constants.zig, and a test requires the lookup to judge every pair of octets, 65536 cases, as utf8.zig's machine does, entry 28's reference; the existing tests over every sequence of three octets and the fuzzer cover the rest.
    • It is measured as entry 16 asks: the A/B of the lookup against the compares on both runners, five runs a side, and the gain must pass the noise on the non-ASCII text with no loss past the noise elsewhere. Below that, it leaves with its code.
    • The compares stay as the reference for every target without a kernel, and the tests run both.

    What it costs: a second kernel per architecture for one function, assembly that no lint reads, and a build variant on x86-64 the lookup alone needs; the module's first inline assembly.

    The measurement, 2026-09-29. The lookup against the compares alone, on the N2 (run 36558384471, before the x86-64 kernel existed): the non-ASCII text decoded 1.333 times as fast and encoded 1.332, every other row inside its spread, and the J5 loss on samba gone. The batch that carried it, against main at 1247c3e (run 36568234592): the non-ASCII text decoded 1.668 times as fast on the N2 and 1.443 on an AMD EPYC 9V45, and encoded 1.332 and 1.433; the text with its characters as \u escapes, which the module's own UTF-8 writer in the same batch serves, decoded 1.228 and 1.218 times as fast; tokens, ASCII text and hex strings stayed inside the noise on both. stdx decodes the non-ASCII text at 1.441 of simdjson's speed on the N2, from 0.864, and encodes it at 1.008, from 0.756; on the EPYC 9V45, at 0.744 from 0.516 and 0.929 from 0.648. Landed at c134f2c.

    What was built, 2026-09-29. scan_utf8.zig holds the check apart from scan.zig, in both forms, and takes the lookup where the target has one: every aarch64 target, and x86-64 with AVX2, which the module's baseline target lacks and its AVX2 variant object has. There the loops' string functions compile into the object whole (variants/loop_string.zig), and each loop calls its kernel once a string when the caller's level is AVX2. A first cut called the walk's block loop from inside the string function, and on an AMD EPYC 7763 the text files decoded at 0.62 to 0.83 of main's speed and encoded at 0.52 to 0.72, ASCII files included, the kernel called or not (run 36558900057): x86-64 saves no vector register across a call, so a call inside the block loop kept the walk's state in memory through the loop. The call now sits outside every loop. The tables are built from RFC 3629 §4's constants at compile time. Two tests pin them: every pair of octets judged as utf8.zig's machine judges it, an octet UTF-8 never holds counted by its place alone since the tables flag it on the lane after; and every sequence of an edge octet and two more, at six places across two blocks of letters, run to the length the machine gives (scan_test.zig). Nine mutations were CAUGHT (commits b7baada and 7957e71).

    The alternatives:

    • The compares, as today. The reference, and the non-ASCII rows stay where the table above puts them.
    • The check at 32 or 64 octets a block. Refused by entry 30's measurement: at 64, each escape that stopped a run sent the whole block to the scalar path, 37% slower. Entry 47 takes 32 in the decoder's walk and the encoder's in the AVX2 variant object.
    • No check. RFC 8259 §8.1 requires UTF-8, and entry 27's decoder refuses a text that is not.
    • A lookup written in plain Zig from selects. Sixteen selects a lookup, more than the compares it replaces.

38. simdutf as the baseline of the json module's UTF-8 check. Proposed and taken on 2026-09-29, from the question entry 37 left. Entry 37 measured the check inside the loops, where it is one cost among the stops, the copies and the escapes, against simdjson's and yyjson's whole parsers. The owner asked how the check itself compares with simdutf, the library that publishes the fastest UTF-8 validation and the algorithm entry 37 built from its paper. Only a check run alone answers that, so the module exports one, and bench-json times it beside simdutf's.

    • json.is_utf8(octets), scan_utf8.valid: whether a buffer is UTF-8 whole (RFC 3629 §4), through the same error_lanes the loops run a block of 16 octets at a time, and utf8.zig's machine, entry 28's reference, over the character the last block cuts and the octets after it. It is a function of its input alone, keeps no state, and follows the module's rules: no heap, no I/O, a loop bounded by the input.
    • simdutf 9.2.1 joins bench-json's baselines as a lazy package pinned by hash (entry 8), built from its sources with Zig's clang as simdjson is, its AVX-512 kernel left out on an x86-64 host without evex512 for entry 27's reason, and absent on macOS with simdjson (entry 10). Its binding calls validate_utf8 and nothing else (bench/json/baselines/).
    • bench-json's section "UTF-8 validation against simdutf" (bench/json/json_utf8.zig) runs both over each string workload's octets, the ASCII texts and the non-ASCII one, interleaved in one run, the median of five with the spread, and lists the losses under entry 20's floor. Before it times anything, both must judge every buffer alike.
    • Nothing in src/ reads simdutf's source or its output; the bench holds the only call.

    What it costs: a third C++ baseline in bench/, about 4 MB of source the first -Doracles build fetches, and one public function the module must keep.

    What was built, 2026-09-29. is_utf8(octets, features) is scan_utf8.valid at the level the caller's features pick. valid takes the buffer a group of utf8_group_blocks blocks at a time, four blocks and 64 octets: a group with no octet from 0x80 up costs its four loads, three ORs and one test, and its one possible fault, a character the block before it cut, is found on that block's last three lanes (incomplete_octets, cut_character_len as lanes); any other group runs the lookup a block at a time and ORs the verdicts as octets, which are read once, after the last group. The blocks after the last group run one at a time, and the last octets through utf8.zig's machine. Two costs were found on the way. LLVM split each block's load into a load of 13 lanes and three loads of one lane, with a copy between them, for the shuffles that read a block's low lanes: 24 loads and 12 copies a group in place of 4 loads. An empty assembly statement that takes the block in a vector register and gives it back (loaded) stops it. And the lookup's verdict took five vector instructions to become bools; XORing the lanes asked into the bit that marks a continuation octet after one leaves a nonzero octet on exactly the lanes that break a rule, in two (error_octets_lookup), and error_lanes now reads it with one compare, so the loops' block walk shares the saving. On x86-64 the check runs in the AVX2 variant object when the caller's features name the level (variants/scan_utf8.zig, wide.is_utf8), where it takes the lookup; the module's own x86-64 target keeps the compares. Tests: valid against utf8.zig's machine with a sequence of edge octets at every offset of a buffer of two groups and a block, the ASCII group's cases by hand, and one verdict at the target's level and the features' (wide.zig), and the fuzzer draws buffers of up to four groups against the machine. Six mutations of valid: the cut before an ASCII group ignored, the asked lanes ORed, 0x80 let through, the last group's verdict alone kept and the thresholds one too high were CAUGHT; previous left behind an ASCII group was NOT CAUGHT and is equivalent, since a block that ends between characters asks nothing of the next, and a group of ASCII after one that ends inside a character is refused for the cut (commit 5369a3c).

    Measured, 2026-09-29. Paired against 0551a38, the check as the walk ran it, in run 36603861241. On the N2, is_utf8 runs the ASCII text files at 1.005 to 1.083 of simdutf's speed (median 1.037; 30.0 to 43.6 GB/s against simdutf's 34.0) and the non-ASCII text at 1.039 (5.77 GB/s against 5.56), from 0.075 to 0.100 and 0.542 before: 10 to 14 times its old speed on ASCII and 1.9 on the non-ASCII text. On an Intel Xeon Platinum 8370C, the AVX2 object's copy runs the ASCII files at 0.340 to 0.460 of simdutf (36.2 to 47.4 GB/s against 106.2) and the non-ASCII text at 0.407 (7.24 GB/s against 17.8), from 0.019 to 0.038 and 0.116: 17 to 23 times its old speed, and simdutf's AVX-512 kernel, 64 lanes to the object's 16, holds the rest. The loops share the lookup's shorter verdict: the non-ASCII text decodes at 1.098 and encodes at 1.101 of 0551a38 on the N2, 1.056 and 1.045 on the Xeon; the ASCII files move within their noise, the hex strings among them, whose path the change does not touch. The second paired run, 36605683401, drew the N2 again and an AMD EPYC 7763: on the N2 the ASCII files at 1.012 to 1.083 (median 1.047) and the non-ASCII text at 1.039; on the EPYC 7763, where simdutf runs its AVX2 kernel of 32 lanes against the object's 16, at 0.575 to 0.844 (median 0.824; 43.2 to 50.4 GB/s against 59.4) and 0.520 (7.99 GB/s against 15.4), from 0.027 to 0.034 and 0.151. In the loops there, the non-ASCII text decoded at 1.026 and encoded at 0.988; on the N2 at 1.100 and 1.101. bible.txt and html-1m decoded at 0.951 to 0.957 and 0.948 to 0.951 on the N2 in both runs, past their spread, and within 1.5% on both x86-64 CPUs, while the hex strings, whose path the change does not touch, moved 2 to 5% up on the N2 in both runs: the binary's layout moved with the change, not the loops' work. By decision 20's rule no file loses in every job, and the change stays. What the x86-64 gap wants, a copy of the check at 32 lanes for AVX2 and 64 for AVX-512, is a width the json module's variant object does not build (wide.zig), and is left to the owner.

    The alternatives:

    • No measurement of the check alone: entry 37's rows against simdjson and yyjson stand for it. Refused because they mix the check with the rest of the loop, and a check slower than the state of the art would hide behind a faster copy.
    • simdjson's validate_utf8, already built. It is the same family of algorithm in a copy simdjson keeps, not the library that leads on UTF-8, and the owner named simdutf.
    • Timing the check through the walk with the copies and stops taken out. The walk's shape is the walk's; a validator that a caller may run over a buffer is what a caller compares.

39. A wider copy of the json module's UTF-8 check on x86-64. Ruled by the owner on 2026-09-29, from the gap entry 38 measured. There is_utf8 ran 16 lanes a block in the AVX2 object, while simdutf ran 32 on an AMD EPYC 7763 and 64 on an Intel Xeon Platinum 8370C: stdx ran the ASCII text files at 0.575 to 0.844 and 0.340 to 0.460 of simdutf's speed, and the non-ASCII text at 0.520 and 0.407 (runs 36603861241 and 36605683401). The owner ruled to build the wider copies.

    • scan_utf8.valid_by(width, form, octets) holds the check at any width: groups of utf8_group_blocks blocks, then blocks of the width, then blocks of 16, and the last octets through utf8.zig's machine. The variant object runs it by the lookup at AVX2's 32 lanes and at AVX-512's 64, a level it now builds for the check alone (variants/scan_utf8.zig, build/modules.zig). VPSHUFB looks each 16 lanes up in their own 16 entries, so each table repeats once for each 16 lanes.
    • wide.CheckLevel picks the width from the caller's features: AVX-512's, then AVX2's, then the module's own 16. The loops keep Level, whose AVX-512 kernels ran hex strings slower (entry 30), and the walk keeps 16 lanes, which step 17 measured against 64 (37% slower on non-ASCII text, a rescan for each escape); the check has no octet to stop at, and takes the widest.
    • Zig's own x86-64 backend cannot place a 512-bit operand of inline assembly, so the lookup at 32 and 64 lanes compiles in the variant object alone, which LLVM builds. The module's own code runs those widths by the compares, the reference, and its tests hold them to utf8.zig's machine on every target (scan_utf8.Form).
    • Tests: each of 25 characters and faults on every lane of a buffer that crosses every boundary a width has, at 32 and 64 lanes by the compares on every target, by the lookup where LLVM builds it for the CPU, and through each kernel at every level the CPU runs; the fuzzer does the same with its own buffers. The AVX-512 kernel runs only on a CPU with AVX-512, which the M-series host and Rosetta lack. So bench-json requires every copy of the check to judge 20000 seeded buffers and each workload, whole and with an octet changed, as the module's own 16 lanes and simdutf do before it times anything, and every run on such a runner checks it.
    • The wider copies start their loads on a line of their width (aligned_start): they judge one block from the buffer's first octet, unaligned, then load every block from the next line on, the three octets before it placed in one shuffle. A first cut loaded from the buffer's first octet, and on an Intel Xeon Platinum 8370C the 64-lane copy ran 16 files at 55 GB/s and the one whose buffer sat on a line at 163, while simdutf ran the same files at 107 and 55 (run 36628661670). bench-json now times the check at six offsets from a line. On an AMD EPYC 9V45 the aligned start took an ASCII text at those offsets from 142 to 147 GB/s up to 199 to 207, with simdutf at 132 to 140 (run 36634334010). A buffer under utf8_aligned_len_min, 1 KiB, keeps its unaligned loads, where the setup costs more than the loads it saves.
    • The same run decoded the hex strings at 0.62 to 0.79 of their speed on the EPYC 9V45 with the decoder's code unchanged: the variant object's text was aligned to 16 octets, so every change to the module's own code moved the object and its kernels within their lines. Each kernel and each of the loops' string functions now starts on a 64-octet line (kernel_alignment), and its loops sit where its own code puts them.

    Measured, 2026-09-29. Two paired runs of ee0f015 against main at 6fa420f, 36643926336 and 36643928058, drew the N2 twice, an AMD EPYC 9V45 and an AMD EPYC 7763.

    • On the EPYC 9V45, with AVX-512, is_utf8 ran the ASCII text files at 1.206 to 1.603 of simdutf's speed (median 1.422) and the non-ASCII text at 1.087: 1.86 to 2.46 times main's speed on the ASCII files and 3.84 on the non-ASCII text. Its 64 lanes ran the ASCII files at 1.08 to 1.57 of its own 32 in the same job (median 1.42) and the non-ASCII text at 2.12, so a CPU with AVX-512 keeps the 64-lane copy.
    • On the EPYC 7763, with AVX2, the 32-lane copy ran the ASCII files at 0.950 to 1.190 of simdutf's speed (median 1.008) and the non-ASCII text at 0.952, from 0.596 to 0.979 and 0.521 at main.
    • The N2's copy, which the batch leaves as it was, stayed at 1.01 to 1.08 and 1.04.
    • No Intel Xeon was drawn after the aligned start. Before it, the Xeon 8370C ran the 64-lane copy at 0.511 to 0.551 of simdutf's speed on the 16 files off a line (run 36628661670).
    • The placement proof (entry 20) built bench-json for both runners' targets at both commits. Outside the check and bench-json's own UTF-8 section, every function of stdx and of the benchmark's timing loop is identical apart from its address. So the decoder's and encoder's rows moved by placement alone, the kernels now each on a line: on the EPYC 9V45 the hex strings decoded 1.27 to 1.65 times as fast and E.coli 1.45, and on the N2 bible.txt and html-1m decoded at 0.95 in both runs.

    What it costs: two kernels more in the variant object, the AVX-512 level built for the json module, a width through the check, and up to 63 octets of padding before each aligned kernel.

    The alternatives:

    • The check at 16 lanes on x86-64, as entry 38 built it. simdutf stays 1.2 to 2.9 times as fast there.
    • A 32-lane copy alone, for every x86-64 CPU with AVX2. Measured against the 64-lane copy in the same job on a CPU with AVX-512; the numbers decide.
    • The walk at the wider widths too. Refused by step 17's measurement above, and reopened by entry 47 for the decoder's walk and the encoder's at 32 in the AVX2 variant object.

40. platform: what the CPU offers, probed once when a program starts, and the one module that may make a syscall. Ruled by the owner on 2026-09-29, in colibri's session, as issue 15 records. The owner confirmed each point below on 2026-09-30, through the question prompt.

    What it is. A program calls platform.probe() once, at start, and passes the fields of the Cpu it returns down its stack: to a TLS stack's configuration, for example, which hands them to each session. It is the shape of codec.Features.detect() (entry 21). No library calls it, and stdx keeps no copy of the answers (invariant 4).

    pub const Answer = enum { yes, no, not_known };
    pub const Cpu = struct { aes_clmul: Answer, dit: Answer, sha256: Answer, sha512: Answer, sha3: Answer };
    pub fn probe() Cpu;

    • aes_clmul says whether the CPU has the AES instructions and a carry-less multiply of two 64-bit values. It is yes only when the CPU has both.
    • dit says whether the CPU has Arm's FEAT_DIT: the PSTATE.DIT bit, under which Arm's timing statement for instructions such as MADD and UMULH holds.

    Where each answer comes from.

    • x86-64, on every system: CPUID leaf 1, ECX bit 25 (AES-NI) and bit 1 (PCLMULQDQ). dit is not_known: x86-64's matching mode, DOITM, is one the operating system sets on Ice Lake and later parts, and user code cannot read it.
    • aarch64 Linux: the AT_HWCAP word through getauxval, bit 3 (HWCAP_AES), bit 4 (HWCAP_PMULL) and bit 24 (HWCAP_DIT).
    • aarch64 macOS: sysctlbyname of hw.optional.arm.FEAT_AES, hw.optional.arm.FEAT_PMULL and hw.optional.arm.FEAT_DIT. All three read 1 on the owner's M1 Pro, under macOS 26.6.2.
    • Any other aarch64 system: yes for a feature the build target guarantees, and not_known otherwise.
    • Any other architecture: not_known.

    Where Linux's or macOS's source gives no answer, the target's guarantee stands in, as on any other aarch64 system. getauxval returns 0 when it finds no AT_HWCAP entry, which Zig's reader does in a program that starts in libc but whose Zig code was built without it. A macOS release older than a sysctl name has no value for it.

    The one module that may make a syscall. macOS's sysctlbyname is a syscall; Linux's getauxval reads memory the kernel wrote at start, and CPUID is an instruction. platform may make the syscall because a program makes the call once, and no codec path runs it.

    • tools/lint/io.zig reads src/platform/ under a configuration of its own. It allows std.c.sysctlbyname beside getauxval and refuses every other chain the rule's list names, so the probe opens no file, starts no thread and prints nothing. Every other directory under src/ refuses sysctlbyname as before (invariant 2).
    • No module imports platform, and platform imports nothing (invariant 14). The module-graph rule's table gives no module an edge to it, and zig build graph-check requires an import of it from src/deflate/ to fail.

    It only reads. platform changes no CPU state. It offers no call that sets PSTATE.DIT for the calling thread (msr dit, #1). The owner revisits this only if chapulin issue 186 rules that the caller sets PSTATE.DIT.

    Its own CPUID. platform runs CPUID in a function of its own and does not import codec, whose Features runs the same instruction. The copy is one function of a dozen lines, and the module graph gains no edge from platform.

    The check. zig build platform-check, which zig build test runs, so it runs on every runner of entry 26. tools/platform_check.zig requires probe() to give the answers the host reports through a source the probe does not read: on Linux, /proc/cpuinfo's flags line on x86-64 and its Features line on aarch64; on macOS, sysctl hw.optional.arm.

    • The build compiles platform for the generic CPU of the host's architecture, so no answer comes from the target. The check refuses to run when built for a CPU that guarantees a feature it reads.
    • On Linux it runs twice, without libc and with it: the probe reads the auxiliary vector through Zig's getauxval in one and libc's in the other.
    • It is a tool and not a unit test because it reads a file and runs a process, which no source under src/ may do. The unit tests hold each bit, each sysctl value and each target's guarantee to its answer.

    What it costs: a syscall inside stdx, a second configuration of the io rule, and a ninth module.

    The alternatives refused:

    • Fields in codec.Features, the issue's first form. Features chooses the codecs' SIMD paths and makes no syscall (entry 21). macOS's sysctl would put a syscall in the module every codec imports, and neither answer chooses a codec path.
    • A module of the caller's own. Each program that needs the answers would write the CPUID, AT_HWCAP and sysctl reads again.
    • One call per answer. A program passes one value down its stack, as it passes codec.Features, and one call reads each source once.
    • A call that sets PSTATE.DIT, above.
    • A compile error on any other system. A program built for one would fail to build to learn what it can do without, and not_known says what the program cannot read.
    • Importing codec's CPUID. It adds an edge to the module graph, which CLAUDE.md asks the owner about first, to save a dozen lines.
    • src/platform/ left out of the io rule. The probe could then open a file, start a thread or print, and it needs none of them.
    • sysctlbyname added to the io rule's shared exceptions. Every module could then call it.
    • The comparison as a unit test of the module, above.

    Amended on 2026-10-24: three SHA answers. colibri asked for them in issue 19, and the owner approved the request. colibri states chapulin's SHA bits from them; chapulin measured one TLS 1.3 handshake on the N2 at 1.36 times OpenSSL's instructions without them and 0.97 with them.

    • Cpu gains sha256, sha512 and sha3, each an Answer, read with the others.
    • aarch64 Linux: AT_HWCAP bits 6 (HWCAP_SHA2), 21 (HWCAP_SHA512) and 17 (HWCAP_SHA3), as the kernel's arch/arm64/include/uapi/asm/hwcap.h numbers them.
    • aarch64 macOS: hw.optional.arm.FEAT_SHA256, FEAT_SHA512 and FEAT_SHA3.
    • x86-64: sha256 from CPUID leaf 7, EBX bit 29 (SHA, the SHA-1 and SHA-256 instructions). sha512 from leaf 7's subleaf 1, EAX bit 0 (SHA512), and yes only when leaf 1 reports AVX and OSXSAVE and XCR0 holds the SSE and AVX state: the instructions take YMM registers. A leaf or subleaf past the CPU's last reads as 0. sha3 is no: x86-64 has no SHA-3 instructions.
    • From the target, on any other aarch64 system and where a source gives nothing: Zig's sha2 guarantees sha256, and its sha3 both sha512 and sha3.
    • The check compares them with /proc/cpuinfo (sha2, sha512 and sha3 on aarch64, sha_ni and sha512 on x86-64) and with macOS's sysctl values (leaf7_features' SHA on x86-64, where no Intel Mac has the SHA-512 instructions).

    Amended on 2026-10-25: SSSE3 and SSE4.1. The owner filed issue 20 for colibri. chapulin's SHA-256 code on x86-64 runs SSSE3 and SSE4.1 instructions beside the SHA ones. The x86-64 baseline holds neither, and a hypervisor can hide them while it shows the SHA extensions.

    • Cpu gains ssse3 and sse4_1, Zig's names for the features, each an Answer.
    • x86-64: CPUID leaf 1, ECX bit 9 (SSSE3) and bit 19 (SSE4.1).
    • aarch64, on every system: no. No aarch64 source reports them, and a target never answers no, so probe sets them after it reads the source.
    • The check compares them with /proc/cpuinfo's ssse3 and sse4_1 on x86-64, and with machdep.cpu.features' SSSE3 and SSE4.1 on an Intel Mac.

41. Fills go through @memset, and the program that links stdx owns memset. Ruled by the owner on 2026-09-30, after 97e42ca gave every benchmark program a vector memset of its own. Zig 0.16's compiler runtime defines memset one octet at a time, and on Linux that copy serves every memset call in a program, glibc linked or not (design §8 step 9); Zig's next release stores vectors. codec.fill (b47400b) kept stdx's clears and fills out of that loop: the DEFLATE encoder's clear of its heads at each stream's start, which took 33,500 cycles on a Neoverse N2 at level 1 and 66,000 at level 6 through it, the encoder's counts at each block, the Zstandard decoder's repeated blocks, which decoded 3.40 times as fast through codec.fill on the N2 (design §8 step 11), and brotli's runs of zeros in a context map. With their own memset, the benchmarks no longer show those costs. So stdx calls @memset, and a program on Linux under Zig 0.16 that wants these fills fast exports a memset of its own, as bench/timing/memset.zig does for the benchmarks (pepegrillo's performance_zig.md, "Copies and fills"). A library exports none, since its caller's program owns the symbol.

    Measured with the benchmarks' own memset (design §8 step 11), the removal encoded the 1 KiB bodies 2% to 15% faster on both architectures, and fields.c and js-16k at level 9 4% to 6% slower on x86-64 in both jobs; the owner ruled it in with those losses.

    Cost: a program on Linux under Zig 0.16 without a memset of its own pays the octet loop in these fills, and no benchmark shows it. Gain: the library fills one way, and no code in it answers one Zig release's runtime.

    The alternatives refused: keeping codec.fill until stdx moves to a Zig release whose memset stores vectors; and keeping it with its vector path limited to Zig 0.16, so that the upgrade turns it into @memset.

42. A lazy level prices its matches after a block of cheap literals. Ruled by the owner on 2026-10-01, from a comparison of libdeflate's output with stdx's that keeps decision 9: the symbols of each stream, parsed and priced by a scratch tool, and never its source.

    What the comparison found, on the corpus of decision 15:

    • E.coli came out 7.4% larger than libdeflate's at level 6 and 5.9% at level 9. libdeflate's parse was 71% literals at 2.24 bits each, with matches of 9 octets or more; stdx's was 4% literals at 4.0 bits, with matches of 5 to 8 octets about 14 bits back.
    • Both parses keep themselves. A literal's code carries the bits of the literals' share of the block's literal/length symbols, log2 of the symbols over the literals: a parse of many short matches makes literals rare, and so dear, and a parse priced by them takes as many.

    The rule:

    • At a block's end, a lazy level works out what the block's literals cost among themselves: their average code length less the share. When that is at most cheap_literal_cost_max, 2.25 bits, the next block prices its matches by this block's codes, each literal's less the share. Its lazy step takes a match only when the match costs fewer bits than the literals it covers, the first priced_len_max octets priced one by one and the rest at the literals' average, and its searches try cheap_candidates_max candidates, 16 at level 6 and 32 at level 9: with the full budget, each literal walked 53 of E.coli's candidates.
    • On the corpus, E.coli's blocks alone qualify, at 15 to 17 eighths of a bit; the next cheapest block, one of xml's at level 9, costs 19, and css-1m's 22.
    • Output over libdeflate's on E.coli: level 6 1.074 to 1.001, level 9 1.059 to 1.007. No other file of the 39 changes.
    • The priced steps are a copy of their own, which the encoder's step chooses, and the prices live in the encoder, outside the match finder: every other block runs the instructions it ran before. The prices take 548 octets at levels 6 and 9, whose budgets go from 259 to 260 KiB.
    • What it costs, runs 36863971833 and 36863984298, main and the change in each job, on a Neoverse N2 and an EPYC 7763: E.coli at level 6 at 0.89 of its speed on the N2 and 0.96 on the EPYC, still 1.52 and 1.36 times libdeflate's; at level 9 1.78 and 1.93 times its speed. No other file ran slower in both jobs but js-1k at level 9 on the EPYC, 0.965 and 0.887, which runs the instructions it ran before.

    The alternatives refused, each measured on the M1, which publishes no number (decision 10):

    • Every lazy decision priced in bits, octet by octet: the 39 files' geometric mean over libdeflate 1.0059 to 1.0034, shuffled dickens 6.1% smaller, ptt5 1.1% and nci 0.9%, but level 6 2.5% to 17% slower and real text 0.1% to 0.2% larger.
    • The same with the literals' average for every octet: 0.99 to 1.03 of the time, but css-1m 1.3% larger, its rare characters mispriced, and most of shuffled dickens' gain lost.
    • 3-octet matches from a 32 KiB table of the latest position for each 3-octet hash, 8191 octets back at most: x-ray and sum 3.6% smaller, ooffice 2.6%, mozilla 2.0%, but text 0.3% to 1.3% larger, and keeping the table current alone cost 4% to 9% of level 6's speed. Entry 50 takes them up for the blocks that follow dear literals.
    • Literal prices from the block's octets rather than its codes: E.coli right, mr 0.8% larger, its zeros cheap as octets and mostly inside matches.
    • The share taken off whenever it is large beside the literals' own cost: it fired on css-1m and bible, whose literals are rare because they compress well, and css-1m grew 1.2%.
    • The full walk budget in a block of cheap literals: E.coli's time at level 6 1.88 times libdeflate's, and at level 9 1.22 times.
    • Both copies of the lazy loop in one function, and the prices in the match finder: runs 36856227214 and 36856238411 put level 6 at 0.98 of its speed on x86-64 where the output was the same, 16 files slower in both jobs, as the larger function's registers fell out differently.

43. A batch's exit checks its ends, not each slot. Ruled by the owner on 2026-10-01, from callgrind's count of a decoded token's instructions (design §8 step 18).

    What the exit did:

    • check_batch read every slot of a batch back before Decoder.decode_batch returned: each slot started where the one before it ended, each but the last had ended, and the last ended where the batch's octets do.
    • The read cost 11 instructions a token, about a tenth of a decoded token. qlog's records took 103 instructions a token with it and 92 without on aarch64, and 114 and 103 on x86-64; CLDR's texts 87 and 76, and 101 and 90.

    The rule:

    • A batch's exit checks invariant 7's counts, the slots filled against the slots given, the last slot's ended against the batch's status, and the last slot's octets ending where the batch's do.
    • The slots before the last are as the token loop and Decoder.run wrote them, each from the one position in the output that the call only moves forward. decoder_loop_test.zig's lockstep requires every slot of every batch to equal the checked path's, whole, under seeded splits and slot counts, and over seeded corruptions.

    The alternatives refused:

    • The read kept as it was: a tenth of decoding qlog's records and CLDR's texts.
    • The read with no branch a slot, its mismatches gathered into one test: estimated at 8 instructions a token where the read took 11, and not built.

44. A lazy level's block ends where its codes stop paying, and may cross a slide of the window. Ruled by the owner on 2026-10-01, from the comparison of decision 42. It amends what decision 14's E3 rested on until now: a block ended before each slide of the window, so that its input stayed in the window and its stored form stayed a choice.

    What the comparison found, on the corpus of decision 15:

    • libdeflate's blocks ran two to three times as long as stdx's: mr's first MiB in 9 blocks against 31, asyoulik in 2 against 4. Each block pays a header, 0.7% of mr's and x-ray's output against libdeflate's 0.2%.
    • Longer blocks alone are not the gain. stdx's own parse, cut into blocks at other places and priced with the codes each block would get, came out 0.19% to 0.60% smaller with the best cuts at 1024-symbol marks within the 16,384 symbols a block already holds (mr, lcet10, dickens, x-ray, mozilla), and 0.04% to 0.15% smaller again with 65,536. Cut every 16,384 symbols whatever the slides, mr came out 0.62% larger, as its statistics change.
    • sum, one block of 38 KiB, came out 2.3% smaller as three, where libdeflate cuts it too.

    The rule, at levels 6 and 9 (Level.block_checks):

    • Every block_chunk_symbols symbols, 4096, a check prices the block's newest symbols apart from the rest and together with them: each side's entropy from its counts, in eighths of a bit, and a dynamic header estimated at header_estimate_symbol_bits a symbol with a code and header_estimate_bits more. When apart costs less, the block ends before its newest symbols, and they start the next block.
    • The newest symbols are checked too where a block would otherwise go on unchecked: at a slide of the window, at a flush and at the stream's end. Fewer than a check's worth must save 1 part in partial_chunk_margin_divisor, 256, of the price together, as few symbols' entropy undercounts what their codes cost.
    • At a slide, a block whose fixed code prices below its stored form, or failing that whose plan does, goes on across it. Its symbols before the slide can then only be coded, as their octets leave the window. If the block's end finds the stored form cheapest, those symbols go out alone, coded, as the slide found them cheaper so, and the rest starts the next block. So every block still costs at most its octets stored and a stored block's header.
    • The match finder's loops and the block writer stay the code they were. A wrapper calls the loops in rounds, and shows each round the window only as far as the lookahead of as many positions as the block takes symbols before its next check: a position decided adds a symbol at most. To the loops a round is a call whose input ended there, which changes no decision (invariant 5). The last positions before a flush or the end, fewer than lookahead_min, are decided a step at a time as before, and may pass a check by as many symbols. The checks' state lives in the encoder, outside the block, so the block's fields stay at the offsets the loops and the writer read.
    • encoded_len_max at these levels counts a block for every 4096 octets of input where it counted one for every 16,384, and three for each slide where it counted one: 40 octets more for every 32 KiB of input, 0.17% of the input in all against 0.05%.
    • The checks' state, the counts of the symbols before the newest and three prices, takes 688 octets: level 6's budget goes from 260 to 261 KiB, and level 9's 260 KiB hold it.

    What it gains, the 39 files' geometric mean over libdeflate's output: level 6 1.0041 to 1.0016, level 9 1.0122 to 1.0098. sum comes out 2.3% smaller, kennedy.xls 1.5% at level 6, nci, asyoulik and xml 0.5% to 0.6%, and 27 files are smaller at level 6. The largest growth is cp.html's 0.09% at level 6 and E.coli's 0.09% at level 9.

    What it costs. A check takes about 2000 instructions and a block about 58,000 whatever it holds, so a file pays where the checks add blocks and gains where they take blocks away. Callgrind's count of one encode of each file against main, on aarch64: level 6's median is 0.07% fewer instructions, from nci's 2.0% fewer to sum's 4.2% more, kennedy.xls 2.6% more and cp.html 2.0%; level 9's is 0.06% fewer, from osdb's 1.3% fewer to E.coli's 4.2% more, its matches priced (decision 42) from blocks that end elsewhere. Runs 37173839885 and 37173841351 paired the change with its base, whose hot functions start on 64-octet lines (design §8 step 9), on a Neoverse N2 and on an EPYC 7763 and an EPYC 9V74. The medians held: level 6 at 1.002 and 1.001 of its speed on the N2 and 1.000 and 1.004 on the EPYCs, level 9 at 1.000 and 0.997 and at 1.000. The files that ran slower in both runs are files that run more instructions: at level 6 sum at 0.95 on the N2 and 0.95 and 0.87 on the EPYCs, for 2.3% less output, cp.html at 0.96 to 0.97, for 0.09% more, and on the N2 kennedy.xls at 0.96, for 1.5% less, and dickens-1m at 0.99; at level 9 E.coli at 0.97 on the N2 and 0.98 on the EPYCs, and on the N2 cp.html at 0.98 and sum at 0.99. The owner accepted these costs on 2026-10-04.

    Level 1 keeps its blocks, its output, its bound and its 163 KiB. With the checks it came out 0.2% smaller at the mean, 1.0570 to 1.0549 of libdeflate's, sum and kennedy.xls 2.1%, and runs 37164372472 and 37164373926 put nine files slower in both jobs on the Neoverse N2: text at 0.98 of its speed, kennedy.xls at 0.94 and sum at 0.96. A check costs about 2000 instructions and a block about 58,000 whatever it holds, which level 1 spends on fewer octets of work than the lazy levels.

    The alternatives refused:

    • The checks inside the loops, a bound of positions or a count of symbols tested each turn, and the state inside the block, where it moved the block's symbols 688 octets: the same runs put level 6 at 0.981 of its speed on an EPYC 9V74, 21 files slower in both jobs, while it ran 2% to 7% fewer instructions there. The loops' code had changed, and instruction counts did not predict it.
    • No check at a slide, on the M1 at level 6 (decision 10): json-1m 2.3% larger and css-1m 1.3%, as a block that crosses a slide mixes statistics that change, and kennedy.xls 0.5%.
    • A check that ends a block only once it holds 4096 symbols: json-1m and css-1m 1% larger.
    • A margin on every check, 1 part in 32 to 128 of the price: sum's gain and json-1m's lost.
    • 6 or 7 header bits a symbol: sum 1.7% smaller where 5 bits give 2.3%.
    • No margin on a check of few symbols: dickens 0.10% smaller where the margin gives 0.22%, and asyoulik 0.37% where it gives 0.57%.
    • 32,768 or 65,536 symbols a block, 64 or 192 KiB more state: 0.02% to 0.15% in the best cuts' count.

45. The benchmarks time a small HTTP body as slices of its 1 MiB payload, in rotation. Ruled by the owner on 2026-10-04, from a probe of how bench-deflate times a small file (design §8 step 9). It amends what entries 10 and 15 give the benchmarks as the 1 KiB and 16 KiB payloads, the rows entry 14's predictions for those sizes are read on, and the rows entry 20's rule reads.

    What a row did until now: it timed one file by coding it again and again, for 50 ms a run. A CPU's branch predictor learns a small file coded that way, so the row measured a state that a caller who codes payloads that differ never reaches.

    What the probe found. It cut each HTTP payload of 1 MiB into 1024 slices of 1 KiB and into 64 of 16 KiB, and timed zlib, zlib-ng, libdeflate, Wuffs and stdx over them four ways: the first slice alone, again and again, as a row was timed; copies of the first slice, one after another in memory; each slice 64 times before the next; and every slice once a round, in rotation. The last two code the same octets. Runs 37219738358 and 37219744137 ran it on main (5da4397, the probe at 046c4c7 on its branch), on a Neoverse N2 in both, an EPYC 9V45 and an EPYC 7763.

    • Every candidate ran slower in rotation on every CPU. A candidate's speed in rotation over its speed with each slice repeated, at the median of the four payloads: 0.61 to 0.95 on the N2, 0.55 to 0.91 on the EPYC 7763 and 0.39 to 0.84 on the EPYC 9V45. stdx's encoder ran at 0.78 to 0.82 over 1 KiB slices on the N2 and 0.62 to 0.64 on the 9V45; over 16 KiB slices at 0.72, 0.89 and 0.94 at levels 1, 6 and 9 on the N2, and 0.48, 0.57 and 0.66 on the 9V45. Its decoder ran at 0.77 on the N2 over both lengths, and at 0.63 and 0.57 on the 9V45.
    • The N2's counters put the whole gap in branch misses. Each slice repeated and the rotation counted the same instructions an octet, within 0.04% in each of 136 cells, and the rotation took 1.1 to 15.8 times the misses, 6.7 at the median: stdx's encoder at level 1 over 16 KiB slices 37 a KiB against 188, zlib's over 1 KiB slices 103 against 1370. Each added miss cost 11 to 26 cycles, 16 at the median.
    • Copies of one slice at other addresses ran at 0.99 to 1.00 of the slice's own speed, at the median of each candidate's rows on every CPU: where an input lies plays no part.
    • In rotation the predictor learns nothing: over 16 KiB slices stdx's decoder took 37 to 80 misses a KiB, where bench-profile's report of 2026-09-28 had counted 22 to 56 for the HTTP payloads of 1 MiB whole and 1.7 to 12 for the files of 16 KiB.
    • The rows misstated where stdx stands. With each slice repeated, stdx's level 1 encoder over 16 KiB slices ran at 1.06 to 1.11 of zlib-ng's speed on the N2 and at 0.93 to 1.07 on the EPYCs; in rotation, at 0.90 to 1.01 and at 0.71 to 0.93, for output of 23.5% of html's octets against zlib-ng's 28.9%. Its decoder over 16 KiB slices ran at 0.87 to 0.89 of libdeflate's on the N2 and 0.86 to 0.92 on the EPYCs; in rotation, at 0.84 to 0.87 and at 0.63 to 0.78.
    • The corpus's 1 KiB and 16 KiB files are each payload's first octets, and not its mean: zlib at level 6 codes html's first KiB to 49.4% of its length, and its 1024 slices to 38.0%.
    • Rotation steadies a pair and does not make it immune. Runs 37219749131 and 37219754894 paired ccb4b1b, which only moved the encoder's code, with its parent, on a Neoverse N2 in both, an EPYC 9V74 and a Xeon 6973P-C. The baselines are the same C in both programs. On the N2, 15 of their 104 cells moved past the larger of their spreads and 1% in both jobs with the first slice repeated, the largest by 5.5%, and 8 in rotation, the largest by 1.9%; on x86-64, 13 and 6, the largest by 9.4% and 8.6%.
    • The learning shrinks with an input's length and has no edge. Runs 37224864097 and 37224868777 cut the same payloads into slices of 64, 128 and 256 KiB, and runs 37227179878 and 37227185061 cut dickens and webster into slices of 64 KiB to 4 MiB for the decoders, on a Neoverse N2, an EPYC 7763 and a Xeon 8573C. In rotation the decoders ran slices of 64 KiB at 0.57 to 0.97 of their repeated speed, slices of 128 KiB at 0.64 to 0.98, slices of 256 KiB at 0.75 to 0.99, and slices of 1 MiB and 4 MiB at 0.98 or more. The encoders came back sooner: at level 1 0.68 to 0.97 at 64 KiB and 0.90 or more at 256 KiB, at level 6 0.90 or more from 64 KiB on.
    • stdx's decoder gains the most from a repeated input. Over dickens in slices of 64 KiB it ran at 1.24 of libdeflate's speed with each slice repeated and at 1.11 to 1.12 in rotation on the N2, and at 1.38 to 1.40 and at 1.03 on the Xeon 8573C. In rotation it took 95 branch misses a KiB there on the N2 against libdeflate's 83, and 82 against 70 at 256 KiB.

    The rule:

    • Every benchmark program times a corpus file named <kind>-1k or <kind>-16k as the slices of that file's length of <kind>-1m, whose first octets the file holds: 1024 slices of 1 KiB, 64 of 16 KiB. The row takes the file's name, x and the count of slices: http/html-1kx1024. bench-json, whose rows of those files are hex strings, takes each slice as a hex string in a text of its own.
    • One repetition codes every slice once, in the payload's order, each from a state started anew. An encoder writes one stream a slice. A decoder takes one stream a slice, each encoded once before any candidate is timed, by the encoder the table names. Every candidate takes the same slices in the same order, into one buffer of a slice's length.
    • Throughput counts the slices' octets, and a row's compression is its streams' octets over its slices'.
    • The 1 KiB and 16 KiB files stay in the corpus: the differential checks and the tests read them as before.
    • A file that a row still takes whole and repeats is learned the more the shorter it is, and none has a larger file to cut. Each table names, beneath its heading, the files of learned_len_max octets, 256 KiB, or less: Canterbury's grammar.lsp, xargs.1, fields.c, cp.html and sum, 3.7 to 38 KiB, and asyoulik.txt and alice29.txt, 125 and 152 KiB. Entry 20's rule reads their rows as before, with its amendment of 2026-10-04 for a move the counters put in branch misses. lcet10.txt, plrabn12.txt and ptt5, 427 to 513 KiB, carry no mark: text measured 0.94 to 0.99 at 256 KiB and 0.98 or more at 1 MiB.

    The owner ruled three times: for the slices in place of the eight rows of one file repeated; for the five Canterbury files of 38 KiB or less kept and marked; and, once the longer slices were measured, for the mark at 256 KiB.

    Cost: a small-body row published before this entry compares with none after it; a slice starts and ends where the cut falls, inside an element or a token; and a pair still moves with placement. Gain: the small-body rows measure a payload the CPU has not seen, as a caller codes one; they rank the candidates as a caller finds them; and fewer rows move in a pair on code a change leaves alone.

    The alternatives refused:

    • The rotation rows beside the repeated ones: eight more rows a table and about a fifth more run time, to keep publishing rows that overstate every candidate.
    • The benchmarks as they were, with the pitfall written in docs/performance.md.
    • The five Canterbury files taken out of the timing tables, which removes five of the corpus's 39 files from the rows the per-file standard reads.
    • The same five timed unlearned: a clock read around each coding, and other data through the candidate in between. Estimated at 20 times the wall time a row, and not built.
    • The mark at 64 KiB, which leaves asyoulik.txt and alice29.txt unmarked where slices of 128 KiB measured 0.64 to 0.98; and the mark under 1 MiB, which names ten files, three of them where text measured 0.94 or more.

46. Level 9's search moves to the chain of its match's tail. Proposed on 2026-10-05, from a profile of the files level 9 encoded slower than libdeflate (design §8 step 9). Ruled by the owner on 2026-10-11: what improves main lands. It adds nothing to decision 12's state, changes none of level 9's limits, and changes 33 of the 468 recorded outputs.

    What the profile found, on main (c33aab1):

    • Level 9 led libdeflate's at the median and trailed on 15 of the 39 files on a Neoverse N2: ptt5 at 0.37 of its speed, sum at 0.51, mozilla at 0.54, json-1m at 0.59 and kennedy.xls at 0.80; on an EPYC 9V74, at 0.31, 0.39, 0.45, 0.54 and 0.59.
    • callgrind counted 1.18 to 4.67 times libdeflate's instructions over the first MiB of 38 files, E.coli alone under it: 564 an octet against 121 on ptt5, 589 against 182 on sum, 533 against 205 on json-1m, and 1,434 against 664 on kennedy.xls.
    • The instructions were candidates. A walk read 315 on kennedy.xls, 223 on json-1m, 210 on ptt5, 114 on nci, 92 on mozilla and 77 on sum, and the 4 octets that decide a candidate turned all but about one a walk away, 4 on nci and 14 on ptt5. A quarter of kennedy.xls's walks and 12% of ptt5's read their whole budget, 4,096 candidates, or 1,024 after a match of 8 octets.
    • The candidates a walk reads hold its position's first 4 octets. In a JSON text those are a key's, which thousands of positions hold, and the octets that would make a match longer are a value's, which few hold.

    The rule (best_by_tail, encoder_match_walk_tail.zig):

    • A candidate longer than the match found holds the match's tail, the 4 octets that end one past it, at the same place in its own match. Every inserted position that holds those octets is on their hash's chain. So each candidate left is named twice: by the chain the walk follows, and by the tail's chain, each position of it as many octets after its candidate's start as the tail lies into the match.
    • After each longer match the walk reads the tail's chain from its head, past at most tail_skips_max positions of candidates it has passed, 4, and moves there when that chain's next candidate lies farther back than the next of the chain it follows. A chain that ends, or leaves the reach, ends the walk: no candidate left is longer.
    • The walk has passed every candidate as near as the match is long, as none of them holds a longer one. Were one to, the octets from the match's candidate to its end would repeat at both distances, so at their greatest common divisor's (Fine and Wilf's theorem), and the octet after the match at its candidate would equal the octet after it at the position, where the match ended because the two differ. So the tail of each candidate left lies before the position, at a position the lazy loop has inserted.
    • A move is tried only where the walk's own chain names its next candidate near enough that tail_move_candidates_min more, 16, could follow at that gap before the reach's edge. A chain of fewer costs less to read than a move costs to try: the tail's head, the links it passes, and a branch that goes either way.
    • candidates_max and cut_candidates_max now bound the links a walk reads, the ones a move passes included. A block of cheap literals (decision 42) keeps the plain walk and its 32 candidates: there every 4 octets recur about as often as any other.
    • Nothing is added to the state: the tail's chain is the hash heads and the chain the level keeps. Levels 1 and 6 run the instructions they ran: the bench, built for both targets at both commits, differs in level 9's search alone.

    What it gives:

    • Links read an octet, over each whole file: 75.4 to 1.6 on kennedy.xls, 34.3 to 1.9 on mozilla, 27.2 to 1.4 on sum, 26.9 to 1.4 on json-1m, 22.6 to 2.2 on ptt5 and 9.3 to 1.3 on nci; over the 38 files that run it, from as many, on html's first KiB, to 46 times fewer, 4.1 at the median. Walks that read their whole budget went from 25.5% to 0.02% on kennedy.xls and from 11.9% to 0.6% on ptt5.
    • Output: 12 files change. ptt5 comes out 0.39% smaller, nci 0.30%, sum 0.016%, and world192.txt, json-1m, mozilla, samba, ooffice, lcet10.txt and xml 0.009% or less; mr 0.005% larger, and js-1m by one octet. The 39 files' geometric mean over libdeflate's goes from 1.0098 to 1.0097.
    • Speed. Runs 37270186280 and 37270188195 paired the walk with the commit before it, level 9's read of short chains, on a Neoverse N2 in both, an EPYC 7763 and an EPYC 9V74. Level 9 ran at 1.41 and 1.40 of its speed at the median on the N2, and at 1.61 and 1.72 on x86-64: kennedy.xls at 15 times its speed on the N2 and at 18 and 22 on x86-64, json-1m at 7.6 and at 7.4 and 8.3, mozilla at 4.9 and at 5.4 and 6.0, sum at 4.7 and at 5.1 and 6.5, ptt5 at 3.9 and at 4.1 and 4.8, and nci at 3.0 and at 2.9 and 3.2. 31 rows ran faster in both N2 jobs and 32 in both x86-64 jobs.
    • stdx over libdeflate at level 9 went from 1.14 to 1.73 at the median on the N2, and from 1.09 to 1.81 and from 1.05 to 1.93 on x86-64. Of the 39 files, 35 stand at or above libdeflate on the N2 where 25 did, and 38 and 37 on x86-64 where 27 and 26 did.
    • Three rows ran slower than the read alone in both N2 jobs, files of few matches whose chains a move cannot shorten: osdb, x-ray and shuffled dickens, each at 0.986 to 0.988. With the read they stand at 1.04, 1.21 and 1.12 of main's speed, the two pairs' ratios multiplied, as no job ran the walk against main. No row ran slower in both x86-64 jobs. The owner's ruling admits the three: with the read, the walk leaves each faster than main.

    The alternatives refused, each measured on the M1, which publishes no number (decision 10), as speed over libdeflate's at the median of the 39 files and output over its at their geometric mean, where main reads 1.06 and 1.0098:

    • A budget of 1,024 candidates and a cut of 256: 1.19 and 1.0107, ptt5 1.5% larger and nci 1.0%. One job each on a Neoverse N2 and an EPYC 9V74 (37255608067) ran kennedy.xls at 3.0 times main's speed, ptt5 at 2.7, mozilla at 2.0 and json-1m at 1.5 to 1.7: less than the rule gives on each, for output the rule makes smaller.
    • A budget of 2,048 and a cut of 512: 1.06 and 1.0101; run 37255613952 ran ptt5 and kennedy.xls at 1.7 times main's speed.
    • A walk ended 256 candidates after its last longer match: 1.23 and 1.0114, the trade a budget of 512 makes.
    • The chain of 4 octets inside the match, entered at the candidate that gave it, which needs no head: 1.16 and 1.0097, json-1m at 0.92 of libdeflate and kennedy.xls at 0.86. Those octets are as common as the position's first.
    • A move at every longer match, past however many positions of candidates passed: ptt5 at 0.34 of libdeflate's speed, from main's 0.44, and nci at 0.67 from 0.82. A run of zeros has a tail as common as its start.

    And on the M1 against the rule itself:

    • The move chosen by a select in place of a branch: 0.96 of the rule's speed at the median of 21 files' first MiB, kennedy.xls 0.88. The branch, mostly predicted, lets the chain's reads run ahead of the loads that decide the move.
    • tail_skips_max of 1, 2 or 8: within 2% of 4 at the median, text 3% to 5% faster at 1, and json-1m and nci 2% to 3% slower.

    And one on the runners:

    • tail_move_candidates_min of 4, runs 37269201602 and 37269203304 against the same base: level 9 ran 1% faster at the median, and 24 files 0.6% to 2.8% faster on the N2, but the three rows that run slower than the read stood at 0.966 to 0.985 there, and x-ray at 0.985 on an EPYC 7763, where 16 leaves no such row. At 16 the files of few matches, the ones nearest libdeflate's speed, lose the least.
    • The move in a block of cheap literals too: E.coli at 0.56 of its speed.

47. The decoder's walk and the encoder's take their UTF-8 blocks 32 octets at a time in the AVX2 variant object. Ruled by the owner on 2026-10-09 for the decoder's walk, as an experiment landed on its pairs, from an analysis of the rows still below a baseline at c5e619e (design §8 step 18), and amended on 2026-10-11 for the encoder's. It reopens three refusals for the walks in that object: entry 30's "J5's UTF-8 scan stays at 16 octets", entry 37's check at 32 or 64 octets a block, and entry 39's walk at the wider widths. Each rested on one measurement at 64 octets, where each escape that stopped a run sent the whole block to the scalar path, 37% slower on text of Cyrillic and CJK characters; no walk at 32 had run on a runner. At 16 the decoder's walk decoded that text at 0.84 to 0.86 of simdjson's speed on an AMD EPYC 9V45, the check alone ran 1.8 times as fast at 32 as at 16 on an EPYC 7763 and, by entry 39's ratios, on the 9V45, and the analysis found no cheaper change that closed the row.

    What changed:

    • Past a run's ASCII, the decoder's walk in the AVX2 variant object's copy_rest takes its blocks 32 octets at a time, judged by entry 37's lookup at 32 lanes (string_walk_wide.zig). The variant object names the width as a comptime value. Every other caller and every other target keep 16: entry 30 refuses a width the build target suggests. The encoder's walk kept 16 until the amendment below.
    • The blocks run out of line, in string_walk_wide.take_blocks. Its loop keeps its eleven constants in registers, and the walk's function holds no register of 32 lanes.

    What it gave, and the rows the owner accepted below main, are in design §8 step 18:

    • The Cyrillic and CJK text decoded at 1.37 to 1.50 of main's speed on an EPYC 7763, an EPYC 9V74 and a Xeon 8370C, above simdjson's speed on each, where it stood at 1.02 to 1.19.
    • The EPYC 7763 decoded nine ASCII texts with escapes at 0.92 to 0.99 of main's speed. A null build moved them as far, so they count as placement (entry 20, amended 2026-10-04); the null build keeps the change's walk, and so does not tell that function's new form from its new place.
    • The Xeon 8370C decoded js-1m at 0.97 and world192.txt at 0.99 in both jobs, with no null build there. A single job on an EPYC 9V74 decoded reymont at 0.85 of main's speed, 0.74 of simdjson's. The first form's single job on a 9V45 decoded reymont at 0.83 and bible.txt at 0.51, which are 0.56 and 0.72 of simdjson's.

    Amended by the owner on 2026-10-11: the encoder's walk takes the same blocks, in the AVX2 variant object's copy_escaped. The owner allowed it on 2026-10-10 as an experiment, and it landed on its pairs. The same text encoded at 0.905 to 0.920 of simdjson's speed on the EPYC 9V45, the one CPU drawn that the row still lost on.

    • On x86-64 the encoder's walk was one loop, the ASCII question asked inside it. Two loops inline spilled its state to the stack, and the loops out of line, once a run, read and wrote it through memory (design §8 step 18, "The transfers"). The wide blocks run out of line and take their slices by value, so the walk that calls them holds one loop, the ASCII one. The variant object's walk takes that loop (encoder_loop_string.ascii_loop); the baseline target's keeps one loop and 16.
    • The row encoded at 1.377 to 1.544 of main's speed in five jobs on the EPYC 9V45, 1.385 to 1.408 of simdjson's speed after, and at 1.333 to 1.554 in one job each on an EPYC 9V74, an EPYC 7763, a Xeon 6973P-C and a Xeon 8370C. No row moved down in all five; design §8 step 18 holds every row, and what the four single jobs read.
    • The EPYC 9V74's one job decoded qlog's records at 0.916 of main's speed, 0.903 of yyjson's, on functions the change leaves the same. The owner landed the change with that row recorded as one job reads it, while a draw of that CPU was measuring. Its two jobs then read the row at 1.003 and 1.000. Over the three jobs bible.txt decoded at 0.955 to 0.986 of main's speed and a hex string of json-1m encoded at 0.978 to 0.983, each on functions the change leaves the same, which counts as placement (entry 20).
    • What it costs: copy_escaped grows from 602 instructions to 710, by a second copy of the run's scan.

    The alternatives refused:

    • Keeping 16. The row stayed below simdjson on the EPYC 9V45 and near it on the 9V74, at 1.02, and the analysis found no other change that closed it.
    • 64 octets on a CPU with AVX-512. Entry 30's loss at 64, and the Xeon 8573C's 64-lane kernels ran hex strings 14% slower; codec.Features cannot tell the CPUs where 64 would gain.
    • The blocks inline in the walk, the first form measured. It decoded the row at 1.41 to 1.43 of main's speed on the EPYC 7763, where the form that landed reads 1.37; it held eight constants of 32 lanes across the walk's other loops and read three more from memory on each pass, and no null build priced its rows.
    • The encoder's walk kept at 16, as this entry first ruled it, since that walk had spilled the last time it took a second loop. Its row stayed below simdjson on the EPYC 9V45.
    • The encoder's two loops inline, or its loops and its run out of line once a run: the two forms design §8 step 18's "The transfers" measured on an EPYC 9V74. The first encoded bible.txt at 0.705 of main's speed, and the second the text files at a median of 0.752 of the first's.

48. Level 6 searches by its match's tail, and reads 24 links a search. Proposed on 2026-10-11, the day the owner set the encoder's bar: no level writes more than libdeflate or runs slower (design §8 step 9). Ruled by the owner the same day, twice. First: the search lands with entry 46's limit of 16 on its moves, the three rows it ran slower than main on a Neoverse N2 ruled in. Then, when the check on the owner's M1 found that form slower than main on nine rows: it lands with that limit at 48 for level 6, if its pairs show no row slower than main in both jobs beyond those three. They showed two of the three. The entry extends entry 46's search to level 6, takes level 6's pair walk out, and changes three of level 6's limits: candidates_max from entry 36's 64 candidates to 24 links, cut_candidates_max from its 16 to 8, and tail_move_candidates_min, now a level's own, to 48 where level 9 keeps 16.

    What level 6 was, on main (0b53229):

    • It ran at 0.95 of libdeflate's speed at the median on a Neoverse N2, at 0.84 on an EPYC 7763 and an EPYC 9V74, at 0.81 on an EPYC 9V45 and at 0.79 on a Xeon 8573C, under it on 24 to 27 of the 39 rows on the N2 and on 32 to 34 on x86-64. It wrote 1.0015 of libdeflate's octets at the rows' geometric mean, and more on 15 rows.
    • callgrind, on aarch64 over dickens's first MiB, counted 151.6 instructions an octet where libdeflate retires 87.3. 26 of them were the pair walk's: the two searches after a taken match walked their chains in one loop (entry 14, E2's history), and one search at a time retires 125.6.
    • Text's time was not those instructions. One search at a time ran the nine text files at 0.84 of libdeflate's speed on the M1 at the median, where the pair walk ran them at 0.87. callgrind's branch model counted 1.38 mispredicted branches an octet on dickens against libdeflate's 0.82.

    The rule:

    • Outside a block of cheap literals, level 6's lazy step runs best_by_tail at each position that searches, inline in its loop, one search at a time. best_pair is gone, and with it the cut that followed the first walk's match into the second.
    • A search reads at most 24 links, the ones a move passes included, and 8 once a match of cut_len octets waits. The tail's chain names the candidates that can be longer, so 24 links find more than 64 candidates of the position's own chain found.
    • A search tries a move where 48 more candidates could follow on its own chain before the reach's edge (Level.tail_move_candidates_min). Level 9 keeps entry 46's 16.
    • A block of cheap literals keeps the plain walk and its 16 candidates (entry 42), each search alone. cheap_candidates_max no longer stays under cut_candidates_max: one counts candidates and the other links.
    • Level.walks_inline names what pair_walks also stood for, the padded window and the search inline in the lazy loop. tail_skips_max stays what entry 46 made it, for both levels.

    What it gives:

    • Output. 132 of the 156 recorded outputs of level 6 change, and none of levels 1 and 9. The 39 rows' geometric mean over libdeflate's goes from 1.0015 to 0.9998, and the rows that write more than libdeflate from 15 to 11. 24 rows come out smaller than main's: ptt5 by 2.1%, json-1m by 1.0%, nci and xml by 0.8%, and reymont and json's 16 KiB slices by 0.5%.
    • Instructions. callgrind counts 105.2 an octet over dickens's first MiB, from 151.6. On the M1, over each whole row, level 6 retires 0.74 of main's instructions at the median, and 1.01 of libdeflate's, from 1.40.
    • Speed. Runs 38186691384 and 38186693796 paired the search with main on a Neoverse N2 in both, an EPYC 7763 and an EPYC 9V74. Level 6 ran at 1.086 and 1.088 of main's speed at the median on the N2, and at 1.139 and 1.132 on x86-64: json-1m at 1.48 and at 1.51 and 1.55, kennedy.xls at 1.22 and at 1.38 and 1.37, reymont at 1.23 and at 1.31, and ptt5 at 1.20 and at 1.20 and 1.23. 29 rows ran faster in both N2 jobs and 31 in both x86-64 jobs.
    • stdx over libdeflate at level 6 went from 0.95 to 1.07 at the median on the N2, and from 0.84 to 0.98 and to 1.02 on x86-64. Of the 39 rows, 26 and 27 stand at or above libdeflate on the N2 where 15 and 13 did, and 19 and 20 on x86-64 where 7 did. The nine text files stand at 0.96 to 1.02 of libdeflate's speed on the N2, from 0.80 to 0.94, and at 0.87 to 0.94 on x86-64, from 0.70 to 0.84.

    What it costs:

    • Two rows ran slower than main in both N2 jobs: osdb at 0.959 and x-ray at 0.990, each the same in both. The EPYC 9V74's job ran them at 0.976 and 0.978, and the EPYC 7763's at 1.007 and 1.003. The owner's first ruling admits both. osdb retires 6% fewer instructions than on main, and callgrind's model counts no more mispredicted branches for it. Neither count explains the row; what is left is the pair walk, which read two chains side by side.
    • 13 rows come out larger than main's: css-1m by 0.84%, where it stands 0.4% under libdeflate's, mr by 0.33%, sum by 0.13%, json's 1 KiB slices by 0.12%, E.coli by 0.11%, and eight rows by 0.04% or less. E.coli's blocks that are not priced searched 64 candidates of four letters; it stands at 1.0018 of libdeflate's, from 1.0008.
    • callgrind's branch model counts more mispredicted branches an octet on text, 1.57 on dickens from 1.38: after each longer match the search tests whether a move could pay, and on text that test goes either way.
    • On the M1, which publishes no number (entry 10), two rows ran slower than main past 1% in both of two passes: mr at 0.976 and 0.981, and html's 1 KiB slices at 0.989 and 0.953. The 39 rows' median reads 1.044, and the nine text files 1.003 to 1.122.
    • Three recorded test outputs change at level 6: the seeded input from 46,111 octets to 45,802, two seeded letters from 4,009 to 4,132, and seeded DNA from 26,296 to 26,876. Two letters and four make every tail as common as its start, and 24 links reach less far there than 64 candidates did.
    • Levels 1 and 9 run main's functions, and on the N2 seven of their rows moved in both jobs all the same: level 1's nci, osdb, fields.c, grammar.lsp and xargs.1 at 0.956 to 0.988, and level 9's grammar.lsp and xargs.1 at 0.980 to 0.985. zlib-ng's level 1, whose code no commit touched, ran seven rows 1% to 6% slower and four 1% to 5% faster in the same jobs. Entry 20 counts such moves as placement; design §8 step 9 holds the proof.

    The alternatives refused. On the runners, against main:

    • The limit on a move at 16, level 9's, the form the owner first ruled in (runs 38174634741 and 38183329992): 1.079 and 1.077 of main's speed at the median on the N2, with mr slower than main in both jobs too, at 0.987 and 0.986; 1.148 on an EPYC 7763 and 1.084 on a Xeon 8573C; output 0.9993 of libdeflate's, ten rows above its. On the M1 it ran nine rows slower than main past 1% in both passes: mr at 0.93, osdb at 0.94 to 0.95, dickens, asyoulik.txt, alice29.txt and lcet10.txt at 0.95 to 0.97, x-ray at 0.97 to 0.98 and two rows of 1 KiB slices at 0.97 to 0.99, and the 39 rows' median read 1.015. callgrind's model counts 1.64 mispredicted branches an octet on dickens there. The limit at 48 gives back 0.05% of output for those rows.
    • The limit at 32, on the M1 alone: the median at 1.032, eight rows slower than main past 1% in its one pass, and output 0.9996 of libdeflate's.
    • The limit at 64 (run 38174636586, one job a CPU): 1.087 on the N2 and 1.154 on an EPYC 7763, and 1.055 at the M1's median, but the output at 1.0001 of libdeflate's. At 256 (run 38174638329): 1.086 on the N2, 1.116 on a Xeon 8573C, and the output at 1.0024. osdb ran at 0.95 to 0.96 of main's speed on the N2 at every limit: its loss is not the moves'.
    • The same search with level 6's old limits, 64 links and 16, and moves at 16 (run 38166811149, one job a CPU): 1.046 of main's speed at the median on the N2 and 1.074 on an EPYC 7763, where 24 and 8, with 8 candidates in a block of cheap literals too, gave 1.095 and 1.151 in the same hour (run 38166813121); output 0.9969 of libdeflate's, and mr at 0.81 and 0.85 of main's speed.
    • The pair walk kept for blocks of cheap literals (runs 38171566022 and 38171568728): every recorded output the same, and E.coli at 0.986 of main's speed in both N2 jobs.
    • 8 candidates in a block of cheap literals, where 16 stay: E.coli at 1.42 of main's speed on the N2 for 0.02% more output, and seeded DNA 3.6% larger than main's against 2.2%.
    • stdx built for x86-64-v3 (run 38174701845, an EPYC 7763): 0.985, 0.991 and 1.022 of the baseline build's speed at levels 1, 6 and 9. What level 6 still gives libdeflate on x86-64 is not the baseline CPU's instructions.

    And on the M1, each with moves at 16, as speed over libdeflate's at the median of the 39 rows and of the nine text files, and output over its at the geometric mean, where main reads 0.92, 0.87 and 1.0015 and the search with 24 links and 8 reads 0.99, 0.87 and 0.9993:

    • One search at a time on the position's own chain, 64 candidates and 16: 0.93, 0.84 and 1.0015. With 24 and 8: 1.11, 1.02 and 1.0077, the output a search by the tail takes back.
    • The search with no move: 1.10, 0.99 and 1.0077. No form measured that writes no more than libdeflate ran text faster than 0.87, and the four that ran text at 0.99 or above wrote 0.8% to 1.0% more.
    • 16 links and 8: 1.01 and 1.0014. 32 and 8: 0.99 and 0.9984. lazy_len 16: 1.04, 0.88 and 1.0032, and 8: 1.12, 0.99 and 1.0076. nice_len 64: 0.97 and 1.0001. cut_len 4: 1.00 and 1.0005.
    • One branch for a candidate's deciding octets and its first octets together: 0.98, with 0.23 more mispredicted branches an octet on dickens. A move chosen by selects: 0.83, text at 0.69. The search by the tail in the single searches and the pair walk at a fresh start: 1.02, 0.97 and 1.0062.
    • A lazy check that starts on the chain of the waiting match's tail and looks only for a longer match: 1.07, 0.95 and 1.0043; bounded by what the waiting match is worth in the lazy step's comparison, so that the choice is an unbounded search's: 0.97, 0.85 and 1.0006.
    • The next position's head read while a search runs, and its candidate's octets and link prefetched: 11 rows at 0.97 to 1.08 of the search's speed, on 0.3% to 5.4% more instructions.

49. Runtime safety off in the DEFLATE encoder's loops, at the caller's choice. Proposed on 2026-10-11, as entry 17's measurement of the encoder asks, and ruled by the owner the same day: the caller's choice, as entry 35 made it for json. It is a compile-time option, the checks stay on unless the caller turns them off, a test build, the fuzzer's included, and a Debug build keep them, and the benchmark prints both. It joins entry 16's exceptions, after the brotli command loop and entry 35's.

    The measurement.

    • Entry 17's, the benchmark built ReleaseFast over ReleaseSafe in one job, at 0b53229 (run 38170242086): the gzip encoder at levels 1, 6 and 9 ran at 1.095, 1.052 and 1.114 of its ReleaseSafe speed at the median of the 39 rows on a Neoverse N2, and at 1.077, 1.039 and 1.055 on an EPYC 9V45.
    • Entry 16's A/B, every function of the encoder's five inner files with @setRuntimeSafety(false) after the same code with it on, in one job (run 38172161400, on the form of level 6's search entry 48 first ruled): 1.049, 1.044 and 1.067 at the median on the N2, where 38, 39 and 39 of the 39 rows ran faster past their spread and none slower, and 1.044, 1.037 and 1.050 on a Xeon 8573C, where 11, 6 and 12 did and none ran slower.
    • Level 9 passes entry 16's 5% floor on the N2 and stands at it on the Xeon. Levels 1 and 6 gain 3.7% to 4.9%, under it. The owner ruled the choice in for the encoder as a whole: a caller sets it for an encoder's type, and measures its own level and input.
    • On the M1, which publishes no number (entry 10), the encoder retires fewer instructions with the checks off over seven rows: 0.87 to 0.93 of its count at level 1, 0.88 to 0.96 at level 6 and 0.75 to 0.84 at level 9.

    The ruling.

    • EncoderOptions gains runtime_safety, true by default. gzip's and zlib's encoders take the same options, so the choice reaches them.
    • With it false, in a build that is neither a test nor Debug, these run with Zig's runtime safety checks off: every function of the match finder (encoder_match.zig and its pieces for the greedy level and the two walks), the block's appender and its code lookups (Appender, Block.add_literal, Block.add_pair, length_code, distance_code), and every function of the block writer under write (encoder_emit.zig).
    • Everything else keeps its checks: the encoder's entry and its steps, the block's plans and prices, the codes, the codec writer, and the decoder.
    • The block writer's functions all take the flag, the header's phases too, so that each choice has its own copy of them. With the flag on the symbols' loop alone, a program that held both choices called the phases from two copies of write, and the compiler stopped holding them inline in either: 1,236 instructions to 358 on aarch64.
    • Encoder joins the caller's choice with the build's in one place, its level's runtime_safety, and each function the choice reaches starts with a @setRuntimeSafety of that field, or of the flag a caller copied from it. A test of encoder_safety.zig requires every @setRuntimeSafety of those files to take one of the two, and every other file of the encoder and the decoder to hold none.
    • differential-encode encodes each of its inputs with the choice off too and requires the checked encoder's octets. bench-deflate times stdx unchecked beside stdx, once it has written the same members.
    • The default build is main's. bench-deflate, which holds both choices, holds every function of the library it held before, unchanged, on both targets, but one on aarch64, the decoder's execute, whose padding follows its address. The functions that take the flag became instances and changed their names alone.
    • Entry 16 gains a row. It amends entry 17's refusal of a switch that turns checks off for callers, for these functions, as entry 35 amended it for json's loop: the switch is a type's, and stdx's assertions stay on.
    • Ruled by the owner on 2026-10-12, with the pairs of design §8 step 9 in hand: the change lands, and the encoder's bar, no slower than libdeflate, reads stdx, the checked column, which a caller gets without asking. stdx unchecked is reported beside it.

    What stays checked in those functions: std.debug.assert, as its unreachable lies in its own function, and every function they call that the choice does not reach, the codec writer among them. An unreachable written in them becomes an assumption, as entry 17 warns.

    What it risks: a wrong bound in the match finder reads or writes outside the encoder's state instead of panicking. The octets an encoder is given may be a peer's, a server compressing what a client sent, and their values steer the search: which chains it follows, how far a match runs. The fuzzer runs that code with the checks on, and differential-encode and the benchmark run it with them off over the corpora.

    The alternatives refused:

    • The checks off in every ReleaseSafe build, stdx's choice and not a caller's, and the caller's choice with the checks off by default: entry 35 refused both.
    • A switch at run time. It compiles the loops twice, and every binary carries both.
    • The choice at level 9 alone, where the A/B passes entry 16's floor: a caller would meet an option that works at one level of three, for a floor its own input may pass at the others.
    • Every function of the five files, the block's plans and prices among them, as the A/B built it: the flag would thread through every plan and price for the instructions of a stream's one block.
    • Neither. Every check stays, and the encoder gives up 4% to 7% on the N2.

50. A lazy level takes matches of 3 octets after a block of dear literals. Proposed on 2026-10-11, the day the owner set the encoder's bar (entry 48), from the rows that wrote the most over libdeflate. Ruled by the owner three times. On 2026-10-11: such matches may be tried only in blocks whose literals cost many bits, so that text never takes them and their table is kept only there, measured first, and brought back before anything lands, as they add state. On 2026-10-12, with the measurements below: levels 6 and 9 take them, from a table of 32 KiB. And the same day, with the pairs at the commit that lands: the rows whose tries the verdict turned off go in too. The entry takes up an alternative entry 42 refused, for the blocks where it pays, and changes two of entry 12's budgets: level 6's state goes from 261 KiB to 294, and level 9's from 260 to 293.

    What the levels wrote, on main (3cba95d), in octets of raw DEFLATE counted on the M1:

    • x-ray came out 4.3% larger than libdeflate's at level 6 and 4.5% at level 9, sum 3.6% and 2.2%, ooffice 2.8% and 2.9%, mozilla 1.9% and 2.2%, and sao 1.7% and 1.8%.
    • The match finder hashes 4 octets (hash_len), so it finds no match of 3. A scratch count of libdeflate's symbols (entry 9) found 22% of its matches on sum 3 octets long at level 9.
    • Entry 42 tried such matches in every block and refused them: text came out 0.3% to 1.3% larger, and keeping the table cost 4% to 9% of level 6's speed.

    The rule:

    • Three, beside the match finder in the encoder as entry 42's prices are, holds the latest position with each hash of 3 octets: 14 bits of hash (three_hash_bits), a table of 32 KiB. A level keeps it only in blocks that take these matches. It is cleared when a block first keeps it, so a stream that never does pays for no clear, and it slides with the window.
    • The trigger, at the end of a block that took none (dear): the next block takes them when this block's literals were dear. Their codes took 6.5 bits or more a literal at the mean (dear_literal_cost_min), and 5 bits or more for each of the block's literal/length symbols (dear_symbol_cost_min): each literal costs many bits, and literals are most of the block's symbols.
    • The verdict, at the end of a block that took them (Three.paid): the next block takes them too when they saved 1% of the block (three_saving_divisor). The block's symbols are priced by their own frequencies twice: as they are, with the extra bits of the matches' distances, and with each 3-octet match turned back into its three literals. The encoder counts the octets inside the matches it takes, and their distance codes, for that.
    • After a block they did not pay in, the level waits one block end before literals may trigger them again, and twice as many after each such block in a row, 64 at most (three_wait_growth, three_wait_max).
    • A block of cheap literals takes none (entry 42). Level 1 takes none.

    A block that takes them runs a third copy of the lazy steps (encoder_match_dear.zig), as a block of cheap literals runs its own (entry 42), so every other block runs the instructions it ran before. In that copy:

    • Every position joins the table beside its chain.
    • A position after a literal, or after a 3-octet match that waits, reads no short chain. It reads its chain's nearest candidate and the table's, each on no branch, and searches where either holds its 4 octets. Where neither does, a position after a literal takes the table's candidate as a match of 3 octets, when it holds the 3 within 8,191 octets (three_distance_max).
    • The lazy step takes a 3-octet match that waits unless the next position's search finds a match that scores more.
    • A position after a match of 4 octets or more searches as in any other block.

    The entries it touches:

    • Entry 12. The table and the counts take 33,344 octets. Level 6's state is 299,760 octets and level 9's 299,504, from 266,416 and 266,160.
    • Entry 49. The caller's choice reaches the new steps: every function of encoder_match_dear.zig, and the table's insert, tally and probe, start with their level's flag. The trigger, the verdict and the wait keep their checks.
    • Entry 17. Its list of wrapping operators gains two, each with its bound beside it: Three.tally's counts, and the two differences of positions in probe.
    • Entry 42. Its refusal of these matches stands for every block but these.

    Where the trigger fires, counted by a scratch build over the 39 rows as the share of a row's octets the copy decides, at levels 6 and 9:

    • x-ray 99.8% at both, ooffice 98.7% at both, sao 81.0% and 85.7%, and mozilla 54.3% and 60.7%: the four rows the matches pay on.
    • sum 38.4% and 38.5%, osdb 4.8% at both, mr 2.0% and 2.2%, samba 1.6% and 0.8%, and reymont 0.7% and 0.3%. There every try ended at its first verdict: 1, 8, 7, 9 and 2 tries at level 6.
    • The other 30 rows never trigger them: the eight other text files, the twelve rows of HTTP bodies, nci, xml, kennedy.xls, ptt5, E.coli, cp.html, fields.c, grammar.lsp, xargs.1 and shuffled dickens.

    What it gives, in the same octets:

    • Level 6: x-ray comes out 2.86% smaller than main's, ooffice 2.11%, sao 1.40%, mozilla 1.24% and sum 0.06%. They stand at 1.0133, 1.0062, 1.0027, 1.0067 and 1.0356 of libdeflate's, from 1.0431, 1.0279, 1.0170, 1.0194 and 1.0362.
    • Level 9: x-ray 2.86% smaller, ooffice 2.08%, sao 1.43%, mozilla 1.37% and sum 0.07%. They stand at 1.0152, 1.0072, 1.0038, 1.0084 and 1.0212 of libdeflate's, from 1.0452, 1.0285, 1.0183, 1.0224 and 1.0220.
    • The 39 rows' geometric mean over libdeflate's goes from 0.9998 to 0.9979 at level 6 and from 1.0100 to 1.0080 at level 9, and the rows' octets together come out 0.74% and 0.77% fewer. No row crosses libdeflate's: 11 rows write more than it at level 6 and 33 at level 9, as before.
    • 60 of differential-encode's 468 recorded hashes change, all at levels 6 and 9: every hash of sum, mozilla, ooffice, osdb, samba, sao and x-ray, and the whole file's of mr and reymont.

    What it costs:

    • Speed, on the four rows. Runs 38276705433 and 38276710244 paired the change at 264ff57 with main, on a Neoverse N2 and an EPYC 7763 in both. At level 6 x-ray ran at 0.900 of main's speed in both N2 jobs, ooffice at 0.926 and 0.925, mozilla at 0.936 and sao at 0.937 and 0.936; on the EPYC they ran at 0.896 and 0.888, at 0.961, at 0.958 and 0.957, and at 0.956 and 0.960. At level 9 x-ray ran at 1.050 and sao at 1.038 in both N2 jobs, and mozilla at 0.980; on the EPYC x-ray ran at 0.884 and 0.879, sao at 0.956, mozilla at 0.970 and 0.975 and ooffice at 0.975 and 0.977.
    • Speed, on rows whose tries the verdict turned off. sum ran at 0.955 and 0.954 at level 6 and at 0.978 and 0.976 at level 9 on the N2, and at 0.984 at level 9 on the EPYC: its one try covers 38% of its 38 KiB. osdb ran at 0.988 at level 6 in both N2 jobs. The owner ruled both in on 2026-10-12, beside the four.
    • Against libdeflate. Four rows that ran above it run under it: at level 6 mozilla, at 0.96 from 1.03, and sum, at 0.97 from 1.02, on the N2, and sao, at 0.996 and 0.992 from 1.030 and 1.042, on the EPYC; and at level 9 x-ray on the EPYC, at 0.92 to 0.93 from 1.05. At level 6 x-ray stands at 0.85 of libdeflate's speed on the N2 and at 0.84 on the EPYC, from 0.94 to 0.95. At level 9 on the N2 it gains, to 0.89 from 0.84. At level 6 the 39 rows' median reads 1.073 in both N2 jobs, from 1.066 and 1.075, and 0.987 in both EPYC jobs, from 0.991 and 0.999; 24 and 25 rows stand at or above libdeflate on the N2 where 27 and 26 did, and 18 on the EPYC where 19 did.
    • Level 1, and every row that never triggers the matches, run main's loops. No row of level 1 moved in both jobs on either CPU, and no other row of levels 6 and 9 ran slower in both.
    • Output. Four rows come out larger than main's, by the blocks that tried the matches and did not keep them: osdb by 0.06% at level 6 and 0.07% at level 9, mr by 0.03% and 0.09%, reymont by 0.008% and 0.003%, and samba by 0.001% and 0.006%. reymont is one of the nine text files: two of its blocks triggered the matches at level 6, and one at level 9.
    • sum keeps its 3.6% over libdeflate's at level 6. Its one try ended at its verdict, and the matches in every block took 0.19% off it.
    • State: 33,344 octets more at levels 6 and 9, in every encoder of those levels, whether its input has dear literals or not.
    • The unit tests' recorded output for the seeded input grows by 8 octets at level 6 and by 4 at level 9, to 45,810 and 45,743.
    • On the M1, which publishes no number (entry 10), over each whole row the four rows retire 1.16, 1.10, 1.09 and 1.06 of main's instructions at level 6, x-ray first, and 1.07, 1.04, 1.03 and 1.03 at level 9. The check against main the owner asks for read 0.997 of main's speed at the median of the 39 rows at level 6, with sao at 0.956, ooffice and mozilla at 0.958, sum at 0.962, x-ray at 0.975 and osdb at 0.980; 1.000 at the median at level 9, with mozilla at 0.984; and 0.999 at level 1. A copy of main's binary against main's own read fields.c at 0.907 and cp.html at 0.935 at level 6 in the same hour, on a laptop under other work.

    The alternatives refused. On the runners at level 6, as x-ray's, ooffice's, sao's and mozilla's speed over main's in both jobs of a pair, in the order they were built:

    • Every position of such a block searching as in any other block, short chains read whole, and the table's match taken where the search found none (b47c88b, runs 38260231175 and 38260235287): 0.69, 0.78, 0.77 and 0.87 on a Neoverse N2, and 0.72 to 0.75, 0.83, 0.83 to 0.84 and 0.90 on an EPYC 7763. Its gzip members stood at 1.0071, 1.0030, 1.0000 and 1.0059 of libdeflate's, where the rule's stand at 1.0136, 1.0060, 1.0022 and 1.0067.
    • One candidate of the chain read after a literal, behind a branch on whether the chain names one in reach (07fb2e3, runs 38260506298 and 38260510185): 0.79 to 0.80, 0.84 to 0.85, 0.83 to 0.84 and 0.90 on the N2, and x-ray at 0.76 on an EPYC 9V45 and at 0.81 on an EPYC 7763. Data that does not repeat answers that branch either way. Read on no branch (3ff278b, runs 38262565419 and 38262569382), the same steps ran at 0.89, 0.92, 0.93 to 0.94 and 0.93 on the N2: the rule's steps.
    • The search of those steps by a call at level 6, as level 9's is (9238170, run 38268833686, one job a CPU, against the rule's steps): 0.945, 0.908, 0.945 and 0.939 of their speed on the N2, and 0.997, 0.974, 0.989 and 1.001 on a Xeon 8573C.
    • A table of 13 bits, 16 KiB (75d7c48, run 38268837381, the same way): 1.023, 1.020, 1.021 and 1.009 on the N2, and 0.996, 1.007, 1.002 and 1.003 on an EPYC 7763, for an x-ray 0.49% larger than the rule's, an ooffice 0.16%, a sao 0.10% and a mozilla 0.07%. The owner ruled the 32 KiB in.
    • The table's entry of the next position prefetched (3217a71, run 38268841381, the same way): 1.005, 0.999, 1.008 and 0.997 on the N2, and 1.004, 0.987, 0.998 and 0.988 on an EPYC 9V74.

    And on the M1, at level 6, as octets against main's and instructions an octet over main's, both exact:

    • The matches in every block, entry 42's trial built again: x-ray 3.47% smaller, ooffice 2.60%, sao 1.91% and mozilla 1.90%, but mr 1.34% larger and dickens 0.39%, and 1.16 of main's instructions on osdb, 1.09 on mr and 1.07 on dickens. With the table kept and no match taken: 1.13, 1.05 and 1.04.
    • A price for each match in place of the verdict: a match taken only where the last block's codes price it 1, 2 or 3 bits under its literals left mr 1.29%, 1.05% and 0.48% larger, and took x-ray's gain down to 3.26%, 2.48% and 1.59%. On mr each of the trial's 3-octet matches cost 15.4 bits where its literals cost 20.5 under its block's code, and the stream came out 1.34% larger: a parse that takes many such matches makes the literals left rare, and so dear, which makes the next block's matches look cheaper still. Entry 42 found the same of E.coli's short matches. libdeflate's own mr holds 26% of its matches at 3 octets, and is 1.2% larger than main's.
    • The table's candidate asked first, and its 3-octet match taken with no read of the chain: ooffice 0.04% larger than main's and sao 1.15%, where the rule takes 2.11% and 1.40% off.
    • Two, three and four links of the chain read in place of one: x-ray 3.29%, 3.41% and 3.44% smaller at 1.21, 1.29 and 1.36 of main's instructions, where one link gave 2.86% at 1.13 on the same build. The plain short-chain read behind the verdict: x-ray 3.45% smaller, ooffice 2.40%, sao 1.63% and mozilla 1.35%, at 1.26, 1.16, 1.16 and 1.08, where the rule's steps retire 1.16, 1.10, 1.09 and 1.06. The walks of hash collisions that the rule's steps skip are what pays for the table.
    • A reach of 255 or 1,023 octets: x-ray 1.18% and 1.16% smaller. Of 4,095: 2.51%. Of 16,383 and of 32,506, a search's own: 2.97% and 2.93%, ooffice 2.17% and 2.23%, where 8,191 gives 2.86% and 2.09% on that build.
    • A table of 12 bits: x-ray 1.65% smaller, and of 13 bits 2.39%.
    • A wait that grows four or eight times: sao 0.78% and 0.76% smaller in place of 1.40%. With a longest wait of 255 block ends beside it: sao as on main.

51. A pinned count of the safety check sites in each SIMD kernel. Proposed on 2026-10-24, after the owner asked what stdx can learn from the Goose language, and ruled the same day: "Build the zero-check guard". Goose finds that a scalar bounds check costs almost nothing, but a check inside a vector loop can stop the loop from staying vector. Goose has a "checkfree" contract: the build fails when a check survives in a function that names it.

    What an audit of the ReleaseSafe objects found, on main (f4c20dd):

    • No SIMD kernel of decision 21 is free of checks. The counts go from 1 site, XXH64 on AVX-512, to 55, the decoder's copy_rest on AVX2.
    • Decision 35's unchecked encoder loop still reaches 29 sites, against 36 for the checked kernel: the choice turns off the checks of claim J11's loop alone.
    • With Zig's checks off, DEFLATE's level 1 ran at 1.029 of its checked speed on the N2 (decision 49). The checks are not where the time goes, so the audit removes none.

    The rule:

    • zig build checkfree-check builds each variant object ReleaseSafe for Linux on x86-64 and aarch64, as assembly. pepegrillo's checkfree engine (decision 7), which tools/checkfree.zig configures, counts for each exported kernel the calls and branches to a panic function in the functions the kernel reaches. A call to reachedUnreachable is an assertion of decision 17, and the count leaves it out.
    • Each count must equal the one tools/checkfree_sites.zig pins, and each pin must name a kernel the objects export. So a change that adds a check to a kernel fails the build, and so does a change that removes one, until the table follows it in the same commit.
    • The count is a property of Zig's and LLVM's output for a fixed target, so every host gives the same counts. A change of Zig's version may move them; the table follows it.
    • tools/ci.sh runs the step. zig build test checks the configuration, and not the step, which compiles nine objects. pepegrillo's tests cover the engine.

    The alternatives refused:

    • Zero sites in each kernel, as Goose asks: no kernel has zero today, and removing them is a speed change that needs its own measurement.
    • Checks inside a loop alone: the tool would have to find loops in the assembly, where a loop is only a backward branch and inlined loops overlap. A count of sites needs no such guess.
    • The DEFLATE encoder's and decoder's loops: they are generic functions inside the library module, with no object of their own, and decision 49 measures their checks' cost already.

52. Level 6 compares its lazy matches by bits, as libdeflate's level 6 does. Proposed on 2026-10-25 and ruled the same day: the owner chose "Plain walk + lazy rule" for a prototype, and "Prepare it to land" for this part of it. It amends decision 36 at level 6 alone.

    What a black-box study of libdeflate's level 6 streams found (2026-10-25, the owner's "Study libdeflate's parse"). It read the 31 corpus files' streams alone, and never the source.

    • One model gives libdeflate's level 6 symbols exactly on all 31 files, 10,076,646 symbols. Its lazy step drops a waiting match (L1, d1) for the next position's (L2, d2) when L2 is at least L1 and 4*L2 - bsr(d2) passes 4*L1 - bsr(d1) by more than 2.
    • stdx's distance penalty of decision 36 drops a waiting match for an equal one a penalty step nearer, and for one an octet shorter three steps nearer. Priced by substitution, those choices cost stdx 0.19% of dickens's output and 0.14% of mr's.
    • libdeflate reads 1.12 to 1.20 times stdx's links an octet on text. It does not search less.

    The rule: at level 6 (lazy_by_bits), keeps_waiting keeps the waiting match unless the next one is at least as long and its bits_score, lazy_length_weight (4) an octet less its distance's bit length, passes the waiting one's by more than lazy_bits_margin (2). Level 9, and the dear blocks of decision 50 at level 6, keep decision 36's penalty.

    What it gives, at 14fb929 against main (43256da) in runs 40204845133 and 40204847410, each job on one VM:

    • Output, counted on the M1, which writes the runners' octets: 23 rows smaller than main's, the text files by 0.08% to 0.34%. 9 rows larger: json-1m by 0.35%, ptt5 by 0.28%, json-16k by 0.26%, nci by 0.15%, and 5 by 0.11% or less. Over libdeflate's at the geometric mean 0.9979 becomes 0.9976, and the same 10 rows stay larger.
    • Neoverse N2, both jobs: the median 1.006 and 1.011 of main's speed. 10 rows faster, dickens 1.014/1.016, reymont 1.017/1.020, plrabn12 1.018/1.015, mr 1.013/1.014; E.coli 0.972/0.975.
    • x86-64, a Xeon 8573C and an EPYC 7763: the median 1.021 and 1.040. 21 rows faster in both, dickens 1.036/1.042, mr 1.039/1.047, plrabn12 1.048/1.042, json-1m 1.025/1.042; E.coli 0.932/0.986. On the EPYC 7763, level 6 over libdeflate goes from 1.022 to 1.072 at the median, and 14 rows trail it, where 16 and 17 did on 2026-10-17.
    • The M1, 5 runs a row and 9 for the rows that read slower: the median 1.002 of main's speed, E.coli 0.966 and 0.969. E.coli's instructions an octet rise 3.0%, and the median row's 0.3%.
    • Levels 1 and 9 do not move.

    The alternatives refused:

    • The rest of the prototype: a plain chain walk of 64 links and 32 on the lazy step in place of the tail chains (decision 48), with and without a 16-bit hash. The study's model of the branches ranked it first. Built, it wrote 0.06% to 0.3% more than this rule alone on text, retired 13% to 18% more instructions and ran 5% to 8% slower on the M1.
    • The rule in the dear blocks too (57a470c, runs 40193002779 and 40193005912): x-ray retired 4.3% more instructions and ran at 0.989 of main's speed on the M1, and ooffice wrote 0.08% more. Both trail libdeflate.
    • Decision 36's penalty in the blocks of cheap literals, to spare E.coli: its instructions rose 2.2% in place of 2.8%, at the same output.

53. An assembly loop for the DEFLATE encoder's level 6. Proposed on 2026-10-25, after the owner chose "Draft the assembly entry" the same day, and ruled the same day: "Approve, aarch64 first". The owner's bar: no level may write more or run slower than libdeflate, on any row (decision 48's entry). The owner asks for assembly once the Zig compiler gives no more gains, as decision 29 records for the decoder. This entry gives the evidence, the rule and its checks.

    Where level 6 stands, on main (5fd03aa). Runs 40204845133 and 40204847410, decision 52's pairs:

    • Neoverse N2: stdx at 1.103 of libdeflate's speed at the median, 11 rows slower in each job. x-ray 0.870, mr 0.908, ooffice 0.924, and the slower text rows 0.986 to 0.995.
    • Xeon 8573C: 0.928 at the median, 29 rows slower. mr 0.780, reymont 0.801, text 0.83 to 0.86.
    • EPYC 7763: 1.072 at the median, 14 rows slower. mr 0.912, ooffice 0.935, text 0.95 to 0.99.
    • Output: 0.9976 of libdeflate's at the geometric mean, 10 rows larger.

    Why Zig stops here.

    • Callgrind on aarch64, the first 2 MiB, instructions an octet: dickens 101.0 against libdeflate's 87.5, mr 105.5 against 87.3, x-ray 107.1 against 81.2. On dickens the tail walk takes 35.0, the lazy loop 31.9, the writer and the symbol appends 16.2, and Zig's assertions 6.6. On mr the tail walk's link loop alone takes about 21, 6 instructions a link.
    • The tail walk reads a link at 13 instructions, where main's walk took 18 and 19 before decision 46. The cheaper Zig form, a countdown of the links left (2026-09-30), ran text 2% to 3% slower on the N2 in both jobs of a pair, and was not landed.
    • Three cuts on 2026-10-25 moved x-ray by 0.5% at most: the reach's edge once a position (worse, 108.0), three literals in a writer pass (106.5, kept on a branch), and no room test in the writer (no change).
    • A plain walk in place of the tail chains cut the branch misses 23% to 33% on the N2, but added 16% to 42% instructions: 0.89 of main's speed on the N2 and 0.83 on two EPYCs (runs 40213997169 and 40213999245).
    • With Zig's checks off (decision 49), level 6 runs at 1.05 of its checked speed on the N2, 1.02 on the Xeon and 0.99 on the EPYC 7763, in the same pairs. Unchecked, the N2 leaves 3 rows below libdeflate (mr 0.951, x-ray 0.919); the Xeon still leaves 27, text at 0.81 to 0.87.

    What the counters say. On the N2 (bench-profile's encoder section, fb53a7d), stdx takes about libdeflate's cycles on text with 1.05 to 1.17 times its instructions and twice its branch misses (dickens 1,351 a KiB against 697), at a higher count of instructions a cycle. mr, x-ray and ooffice lose on instructions, 1.21 to 1.29 times libdeflate's. The x86-64 runners expose no counters, so the Xeon's gap is measured in time alone.

    The rule.

    • It amends decision 16 for one loop: level 6's lazy loop over the positions with the whole lookahead ahead, outside the blocks of decision 42's cheap literals and decision 50's dear literals (advance_lazy in encoder_match.zig with its inline search, best_by_tail in encoder_match_walk_tail.zig, and the short chain's read). That loop may run as hand-written assembly, aarch64 first, then x86-64. The assembly reads and writes memory by address, inside the bounds its Zig caller computes. The Zig loop stays as the reference and the fallback.
    • The loop does what the Zig loop does, in the same order of effect: at each position it inserts the position, skips the search where the short chain misses, walks the chain and the tails' chains under the same budgets and moves, decides by decision 52's compare, adds the symbol and its counts to the block, and inserts the positions a taken match covers. It calls no function.
    • Its caller computes, once a call, the end of the positions and the block's room, as advance_lazy and the rounds of decision 44 do. Before each position the loop returns when the position has reached the end or the block is full, one compare each. The Zig code takes the rest: the last positions (step_lazy), a full block, a slide, and the cheap and dear blocks.
    • Its loads read only the window below filled and its padding, the heads, the chain, and the length code table. Its stores write only the heads, the chain, the block's symbols below its room, and the two count arrays.
    • advance_lazy calls it where a comptime test holds: an aarch64 target, or x86_64 once that loop stays, a builtin.zig_backend of stage2_llvm, and level 6, as decision 29's loop tests. On x86_64 a runtime test of the CPU's vendor follows (decision 54). Elsewhere, and at level 9, it runs the Zig loop. So zig build test-self-hosted runs the Zig loop.
    • Decision 17 holds: no assertion per position inside the assembly. Its caller asserts the bounds at the call's entry, and at its exit that the position moved forward and lies below the end plus match_len_max, and that the count is within the block's room.
    • The template takes every limit, offset and code number from constants.zig and the types' layouts, and comptime asserts pin each one: Symbol's fields, hash_bits, the level's budgets and tail_move_candidates_min, tail_skips_max, nice_len, lazy_len, lazy_length_weight, lazy_bits_margin, match_len_taken_min, match_len_max and first_length_symbol. Each loop stops at a named bound, and the text cites RFC 1951 §3.2.5 where it finds a code.

    The checks.

    • A test runs the Zig loop and the assembly on two copies of one matcher and one block and requires every octet of the copies equal: the window, filled, position, the waiting match, heads, chain, symbols, symbol_count, the counts and input_len. Its seeded states reach each exit and branch: a full block, a match that ends past the end, a short chain that misses and one that holds, a cut lazy search, a tail move taken and refused, a skip run of tail_skips_max, a match at nice_len and at lazy_len, ties under decision 52's margin, candidates at the reach and one past it, and the covered inserts.
    • A fuzz test runs both loops on states it draws and requires the same octets.
    • differential-encode requires its recorded hashes, with decision 49's choice on and off, in its ReleaseSafe and Debug builds.
    • Mutations break each compare of the assembly. Each commit reports them CAUGHT or NOT CAUGHT.
    • Each commit runs zig build test-self-hosted and zig build test-avx512.

    The measurement that admits it. Under decision 20, each pair runs the change after main in one job.

    • The aarch64 loop stays only if dickens and mr run faster on the N2 in both jobs of a pair, past the larger of 2% and the row's spread, and no row at any level runs slower past the larger of its spread and 1% in both jobs, unless decision 20's placement amendments exempt it.
    • The x86-64 loop stays only under the same rule on the x86-64 runners, with the Xeon 8573C's rows reported apart. Decision 54 amends this: the loop ran AMD's EPYC 7763 slower, so it runs on Intel CPUs alone.
    • A loop that does not stay is deleted with its tests, and this entry records its pairs.
    • Before the owner rules on a loop, the M1 times it against main row by row.

    What the aarch64 loop gave. At be101d8 against main (5fd03aa), runs 40261381739 and 40261383705, each job on one VM:

    • Neoverse N2, both jobs: level 6 at 1.064 and 1.068 of main's speed at the median; 36 rows faster in both jobs, dickens 1.067/1.069, mr 1.067/1.067, osdb 1.069/1.066; none slower. Over libdeflate, 1.091 and 1.076 become 1.161 and 1.158, and the rows below it go from 11 to 3: x-ray 0.871, ooffice 0.926 and mr 0.967.
    • EPYC 7763, which runs the Zig loop: no row moved. Levels 1 and 9 did not move.
    • Output: every row's octets are main's, on the M1 and in differential-encode.
    • The M1, 5 runs a row at a load near 10, beside a copy of main against main that read 4 rows slower past 1%: 1.052 at the median, no row slower past 1%, the slowest 0.995; 0.883 of main's instructions at the median, dickens 0.834 and mr 0.865 in a pass of 9 runs.
    • The first form, d8b15e2, read the short chain on four branches. On the N2 it ran at 1.075 and 1.073 (runs 40232666759 and 40232668288), but the M1 ran shuffled dickens at 0.973 and 0.969, osdb at 0.970 and 0.975 and sao at 0.981 and 0.985, at 0.79 to 0.97 of main's instructions. Zig's aarch64 form, one branch for the nearest candidate and one after the rest, ran those rows at 1.074, 1.049 and 1.011 of main's speed. So the loop stays under the rule above.

    What it was to give. The estimate before the loop, kept for the record.

    • On the N2, Zig without its checks reaches 1.15 of libdeflate at the median and leaves 3 rows below. Assembly that holds the loop's state in registers should reach at least that: mr and x-ray from about 0.9 toward 1.0, text past 1.03.
    • On the Xeon, the checks are not the gap: unchecked, text stays at 0.81 to 0.87. The x86-64 loop may not close it, and the kill rule decides.

    What it does not do.

    • It changes no output.
    • It leaves the cheap and dear blocks in Zig: x-ray and ooffice run most of their positions in the dear loop, which this entry does not cover. Decision 59 takes the dear loop on aarch64.
    • It leaves level 9, the writer and the block's plan in Zig.

    What it risks. A wrong bound in the assembly reads or writes outside the encoder's state, with no panic, in every build. The equality test, the fuzz test, differential-encode and the mutations stand against it.

    The entries it touches.

    • Decision 16: a row in its table of fast paths for this loop.
    • Decision 17: the assertions at the call's entry and exit only.
    • Decision 29: a second assembly loop under the same guards, and level 1's unruled draft of 2026-10-21 a third, which this entry does not depend on.
    • Decision 49: the assembly runs in both choices, as decision 29's loop does.

54. Level 6 takes its x86-64 assembly loop on Intel CPUs alone. Proposed on 2026-10-26, after the owner chose "Draft vendor choice" the same day, and ruled the same day: "Approve". It amends decision 53, whose x86-64 loop stays only where it runs no row slower, and decision 21, whose codec.Features it extends.

    What the x86-64 loop gave. Decision 53's x86-64 loop writes the Zig loop's symbols on every input, and cuts about a fifth of level 6's instructions: dickens's first 2 MiB retires 94.8 an octet against main's 116.8, under callgrind's x86-64 count. Six forms were paired against main, each job running main first (perf-deflate-169 to 175):

    • Xeon 8573C, one job, the second form (run 40245672510): level 6 at 1.038 of main's speed at the median, 35 rows faster past their spread, none slower; over libdeflate, 0.929 became 0.970.
    • Xeon 6973P-C, one job, the sixth form (run 40325450647, a VM whose control spread from 0.86 to 1.13): 1.030 at the median, 7 rows faster past their spread, none slower.
    • EPYC 7763, ten jobs over five forms: 0.981 to 0.992 at the median, 14 to 31 rows slower in each job, shuffled dickens and cp.html at about 0.95 in most.
    • EPYC 9V74 and 9V45, five jobs over four forms: 0.993 to 1.018 at the median, with fewer rows slower than on the EPYC 7763.
    • The forms held the symbol count in a register, then the waiting match, rolled the match's compare, read the short chain without a branch and skipped the compare after an empty search. None changed the EPYC 7763's loss by more than its spread.

    The rule.

    • codec.Features gains lazy_loop_fast: on x86-64, CPUID leaf 0 names the vendor "GenuineIntel". It chooses a path rather than naming an instruction, as vpmullq_fast does. Features.target() never sets it, since no build target names a vendor; Features.none() leaves it false.
    • The encoder keeps the flag from init in its field assembly, which exists only where decision 53's x86-64 loop is built: level 6 on x86-64 by LLVM. advance tests it once a call. Where it holds, Matcher.advance_assembly runs the rounds of decision 44 with the assembly; elsewhere advance runs them as main does.
    • The rounds with the assembly are their own comptime instance, and so is the step_lazy they call. So the Zig loop that other CPUs run compiles to main's code: an object of bench-deflate on x86-64 pairs each advance_positions instance of main's with an identical body.
    • Every other rule of decision 53 holds: the same octets, the Zig loop as the reference, the equality tests and the fuzz test, which call the assembly directly whatever the flag, and the mutations. A test of the encoder requires the same octets with the flag on and off.
    • A test requires detection to set the flag from leaf 0's vendor registers alone: true for "GenuineIntel", false for "AuthenticAMD" and any other.

    The measurement that admits it. Under decision 20, on runners that draw Intel CPUs (the intel-pair workflow of a perf branch, which never reaches main): dickens and mr faster past the larger of 2% and the row's spread in both jobs of two Intel pairs, and no row slower in both. On the AMD runners every row must run as main's does, since they run the Zig loop: no row slower past its spread in both jobs of a pair.

    What it gave. At 7992b26 against main (1c08360), each job running main first. The library code is the code that lands.

    • Intel, run 40359653754 of the intel-pair workflow, which ran the same code: two Xeon 8370C jobs at 1.046 and 1.046 of main's speed at level 6, 33 and 34 rows faster past their spread, none slower. dickens ran at 1.06 and mr at 1.05 in both. Over libdeflate, 0.933 and 0.931 became 0.975 and 0.982. Two Xeon 6973P-C jobs ran on VMs whose control spread from 0.82 to 1.16. There, levels 1 and 9, which run main's code, read 0.92 to 1.00; no row read slower in both jobs.
    • Intel, the earlier forms with the same assembly (runs 40333267397 and 40344701178): 1.039 on a Xeon 8573C, 1.045, 1.045 and 1.046 on three Xeon 8370C jobs, 1.051 on a Xeon 6973P-C.
    • AMD, runs 40356831116 and 40356838255, each run twice: two EPYC 7763 jobs, an EPYC 9V74 and an EPYC 9V45. Level 6 read 0.998 to 1.008 at the median. No row read slower in all four jobs. x-ray read slower in both jobs of the first pair, 0.927 and 0.989, and css-1kx1024 in both jobs of the second, 0.980 and 0.974.
    • Those rows move on identical code. An object of bench-deflate built for x86-64 pairs every advance_positions instance with main's body, and Encoder's fields keep their offsets. The setup in the encoder's init stores the flag once a stream. On the N2, whose code is main's by the same proof, the second runs read the 1 KiB rows 0.89 to 0.97 at all three levels. Decision 20's placement amendment exempts them.
    • Output: every row's octets are main's, and differential-encode passes with main's hashes in its ReleaseSafe and Debug builds for x86-64 under Rosetta, which reports GenuineIntel.
    • The M1 runs code identical to main's by object, so it was not timed.

    What it risks. A vendor is not a microarchitecture: an Intel core that runs the loop slower would take it. The three Intel models drawn, Ice Lake, Emerald Rapids and Granite Rapids, all gained; older Intel cores were not measured.

    The alternatives refused:

    • Dropping the loop: Intel's level 6 stays at 0.93 of libdeflate on the runners.
    • Another form for Zen 3: six forms moved its median between 0.981 and 0.992, and no cause was found.
    • A flag per microarchitecture, from CPUID's family and model: a table to keep, for CPUs the runners never draw.
    • The flag in the matcher, tested in advance_lazy: the first form. It changed the compiled Zig loop, and the EPYC 7763 ran cp.html and osdb 1% to 2% slower.

55. An assembly loop for the DEFLATE encoder's level 1, on libdeflate's parse. Ruled by the owner on 2026-10-26: "Approve, aarch64 first", after the owner chose level 1's output as the next gap the same day. Drafted on 2026-10-21 as decision 51, whose number decision 51 now holds; the draft is kept in the records of the work. The owner paused the work on 2026-10-26: "Pause, record it".

    The form it runs on. Main's level 1 writes 1.059 of libdeflate's output at the geometric mean of the 39 rows. The parity form writes 0.9986, and no row larger. It takes libdeflate's level 1 parse, as a black-box study of its streams found it: 2^15 heads of two positions, the multiplier 0x1e35a7bd, a reach of 32,767, and blocks that end at 8,192 pairs or 65,535 octets of input. On 2026-10-20 the owner ruled that a smaller output lands only once level 1 is no slower than libdeflate on each row. The parity form runs level 1 at 0.87 of libdeflate's speed on the N2 and at 0.86 to 0.90 on x86-64, at the median.

    The rule. It amends decision 16 for one loop, advance_greedy, which may run as hand-written assembly, aarch64 first, then x86-64, under decision 53's rules: the Zig loop as the reference and the fallback, the same symbols on every input, the bounds computed by the Zig caller and asserted at the call's entry and exit, and every number from constants.zig. A loop stays only if dickens and webster gain past the larger of 2% and their spread in both jobs of a pair against the parity form, and no row slows in both at any level.

    What the loops gave. Runs on the branch perf-deflate-177 and on, each job running the parity form first. Every stream equals the parity form's on all 39 corpus files.

    • aarch64 (runs 40417841176 and 40417844204): level 1 at 1.086 and 1.085 of the parity form on the N2, 34 rows faster in both jobs, none slower, dickens 1.11 and webster 1.10. Over libdeflate, 0.874 became 0.965 at the median. Callgrind: dickens 40.75 instructions an octet became 36.15, against libdeflate's 36.61. So the loop stays under the rule.
    • Two further aarch64 forms each moved level 1 by 2% at most and slowed other rows in both jobs: the next position's head loaded before a literal's branch (runs 40424597586 and 40424600187), and the match end's octets and head loaded at the pair (runs 40431279300 and 40431281902).
    • x86-64 (runs 40439097352, 40439100187 and Intel run 40439232116): 1.030 and 1.018 on two EPYCs, 1.007 to 1.041 on four Xeons. dickens gained 1.8% in one EPYC job, webster did not gain in both, and x-ray ran slower in both EPYC jobs, 0.984 and 0.959. So the loop does not stay. Callgrind: dickens 47.24 became 44.42, against libdeflate's 37.49; the block writer takes 14.5 of them on x86-64 and 10.4 on aarch64.

    Why it waits. With the aarch64 loop, level 1 runs at 0.94 to 0.96 of libdeflate on the N2, with 25 to 30 rows below it: grammar.lsp 0.81, nci 0.83, json-1m and kennedy.xls 0.86. x86-64 stays at 0.87 to 0.92, with its worst rows near 0.70. The parity form meets the bar on no runner, so neither it nor the loop lands. Main's level 1 stays.

    What the parity form needs to land, besides the bar. It moved state and limits, each a ruling of its own: level 1's state from 163 KiB to 400 KiB and levels 6 and 9's to 430 KiB, as Block holds 49,152 symbols at every level; 15 hash bits and two positions a head at level 1; every covered position inserted; and decision 44's checks at level 1. Its first form gave levels 6 and 9 the 49,152-symbol cap too, which changed their octets; a cap kept per block restores them.

56. Each codec offers its decoder as a std.Io.Reader and its encoder as a std.Io.Writer. Ruled by the owner on 2026-10-26, for issue 21. It amends decision 2, which refused the two interfaces, and invariant 2.

    A server that reads a body through a std.Io.Reader and writes one through a std.Io.Writer asked for adapters, and the owner ruled that they belong in stdx. The streaming call of decision 11 stays the codecs' API. An adapter is a second door to the same call:

    • codec.IoReader puts a decoder behind std.Io.Reader. deflate, zlib, gzip, zstd and brotli each export an IoReader. zstd and brotli also export HttpIoReader, with the window of decision 12.
    • codec.IoWriter puts an encoder behind std.Io.Writer. deflate, zlib and gzip each export an IoWriter. The zstd and brotli encoders get one when design §8 steps 13 and 14 write them.
    • The caller places each adapter, with the decoder or the encoder inside it. The adapter's own buffer is a slice the caller gives to init, and may be empty.

    How an adapter behaves where its source or its destination fails:

    • A source that fails with error.ReadFailed fails the read the same way, and the decoder keeps its state. The next read goes on after the source holds more octets.
    • A read fails with error.EndOfStream only after the source did and the decoder reported done. A source that ends inside a stream stops the reader with error.Truncated.
    • A refusal fails the read with error.ReadFailed, and the reader's failure field names it. Truncated, TrailingOctets and every decoder error are such refusals.
    • After done, the gzip and zstd readers decode the next member or frame (RFC 1952 §2.2, RFC 8878 §3.1). The deflate, zlib and brotli readers refuse more octets as error.TrailingOctets, because their coded octets hold one stream.
    • A destination that fails with error.WriteFailed fails the write, the flush or finish the same way, and the encoder keeps its state. The caller calls again when the destination has room.
    • flush passes Flush.flush to the encoder, and finish passes Flush.finish. Each then flushes the destination.

    What it costs. The adapter adds one vtable call for each call of the decoder or the encoder. The decoder writes into the caller's writer's buffer, and the encoder into the destination's buffer, so neither adds a copy. A small buffer adds calls: each call of the interface makes at most one call of the codec, so its work stays bounded by the octets it takes and writes.

    What the rules allow. tools/lint/io.zig allows three chains under std.Io, each only whole: std.Io.Reader, std.Io.Writer and std.Io.Limit. They move octets between buffers through vtables the caller supplies, and reach no host of their own. std.Io.File, std.Io.net and every chain past the three names stay refused. tools/lint/rfc_citation.zig reads ReadFailed, EndOfStream and WriteFailed as operational errors, as it reads Truncated and NoSpaceLeft.

    The alternatives refused:

    • A module of its own for the adapters, which imports every codec, with the lint exception limited to its directory. The owner chose the adapters in each codec's module.
    • Adapters in the consumer: each consumer writes the same code again, and gets the failure cases wrong in its own way.
    • Keeping decision 2 as it stood, which leaves every consumer to write the loop of design §4.

57. The Zstandard encoder's frames, blocks and tables. The owner ruled on 2026-10-27 that the encoder of design §8 step 13 lands with its speed below libzstd's, recorded in the step's entry, and that its speed work follows. The choices below are the step's own, and the owner may review them.

    The frame.

    • A frame whose first block is its last is single-segment and states Frame_Content_Size (RFC 8878 §3.1.1.1.1.2). Its Window_Size is then its content's size, so a decoder of a small body needs no larger window than the body.
    • Any other frame states its window in Window_Descriptor: 2^19 octets at level 1 and 2^21 at level 3 (decision 12). It leaves Frame_Content_Size out, as a streaming encoder does not know it.
    • Content_Checksum is the caller's choice, EncoderOptions.checksum, and off by default, as libzstd's library leaves it off.
    • Every offset is smaller than Window_Size, as RFC 8878 §3.1.1.4 asks, although stdx's decoder accepts one equal to it (decision 22).

    The blocks. The encoder ends a block of 128 KiB only once input past it arrives. A flush ends a block at the input's end, and a finish ends the last block. So the place a block ends depends on the input and its flush points alone, never on the way the caller split its calls (invariant 5). A block of one octet repeated goes out as an RLE_Block. A block whose sections do not fit one octet less than its own length goes out as a Raw_Block. A block that does not go out compressed puts back the Repeated_Offsets and the sequence tables its parse moved, as RFC 8878 §3.1.1.5 and §3.1.1.3.2.1 read them only from compressed blocks.

    The tables. init clears the match finder's tables: 256 KiB at level 1 and 768 KiB at level 3. A table position of 0 names no candidate, so a frame's first octet never starts a match. The clear keeps every candidate a function of the input alone (invariant 4). For a small body, it costs more than the body's encoding.

    The entropy stages.

    • A literals section is raw, RLE or Huffman-coded, whichever is smallest. A Huffman-coded one takes one stream below 256 literals and four from 256 on. The code is Huffman's tree, its leaves brought up to 11 bits and its Kraft sum made whole again. Its weights go out FSE-compressed or directly, whichever is shorter.
    • Each sequence code takes RLE_Mode when one code occurs. Otherwise it takes the cheapest of the default table, the last compressed block's table, and a table of its own counts with its description.
    • An FSE table gives every symbol that occurs a probability of 1 at least. It uses no "less than 1" probability (RFC 8878 §4.1.1).

    The alternatives refused:

    • Frame_Content_Size in every frame: a streaming encoder learns the size only at the end.
    • A table that the encoder clears only as far as a frame reaches: a table's old positions would come from an earlier frame or from memory the caller never wrote, and the output would depend on them.
    • Content_Checksum on by default: 4 octets more in every frame, which a body under TLS does not need.

    Left for the speed and ratio work, not refused: Treeless_Literals_Block, "less than 1" probabilities, a table sized to a small body, and a clear that costs less than the whole table.

58. Level 9 chooses its symbols by their price in bits. Proposed on 2026-10-26, after the owner chose level 9's output as the next gap the same day. Ruled the same day: "Approve, model first", then "Two rounds, search twice". Stopped on 2026-10-27 by the owner: "Stop, record it". Main's level 9 stays. The owner's bar: no level may write more or run slower than libdeflate, on any row (decision 48's entry).

    Where level 9 stands, on main (b78cfbc). It writes 1.008 of libdeflate's level 9 output at the geometric mean of the 39 rows, 32 rows larger: kennedy.xls 1.104, css-1m 1.028, sum 1.021, x-ray 1.015, the text rows 1.007 to 1.013. It runs at 1.44 to 1.96 times libdeflate's speed.

    What libdeflate's level 9 does, read from its streams alone (decision 9). stdx's level 9 barely improves on its level 6: dickens writes 2.650 times smaller at both, and x-ray writes the same octets at both. Against stdx's level 9, libdeflate's takes more literals and fewer matches (dickens 1.036 million literals against 0.835 million), nearer distances at an equal length (kennedy.xls 53,024 matches 5 to 16 octets back against 28,555), matches of 3 octets, and far fewer blocks (5 against 53 on kennedy.xls). These are the marks of a parse priced in bits.

    The model (pricedmodel.zig in the records: real codes limited to 15 bits, real headers, the cheapest of dynamic, fixed and stored blocks). A parse of least price over runs of 8,192 positions, priced by the block's codes so far, writes 0.9826 of libdeflate's output, 6 rows larger. Prices from the fixed code at a stream's start trap DNA: E.coli takes 55,394 literals and 585,000 short matches against libdeflate's 1.89 million literals. Prices of the first run's literals from its own octets, and a second round priced by the first, give 0.9738 of libdeflate and 0.9664 of main, with html-1k one octet over.

    The prototype (branch perf-deflate-181, c94c00d and e9bf5eb). Level 9 parses each run twice. Between the rounds it restores the heads, the chain links whose slots the run took, and its 3-octet table, so the second search reads what the first read.

    • Size: 0.9749 of libdeflate's level 9 output and 0.9675 of main's, two rows larger than libdeflate (E.coli 1.0062, json-1k 1.0048). Levels 1 and 6 keep main's octets.
    • Speed: 0.063 of main's level 9 speed on the N2 and the EPYC 7763, nci and xml at 0.02 (runs 40482273758 and 40482277265). After three cuts (16 links, a match of 32 taken whole, one relax a length code), callgrind counts 1,063 instructions an octet on dickens against main's 151, 514 on nci against 73, and 653 on xml against 70. About 260 on dickens would run at libdeflate's speed. The cost spreads over the relaxes, the walk and each position's upkeep, twice.

    Why it stops. One round halves the cost, still about twice the budget, and leaves four rows above libdeflate. A cache of the first round's matches costs about 256 KiB of state. Neither reaches the bar on speed without a search far cheaper than level 9's walk.

    What a later try needs: the model and the prototype hold the size; the open part is a search of all lengths at every position at the cost of main's level 9 lazy search.

59. Level 6's dear blocks run their lazy loop in assembly too. Proposed on 2026-10-27, after the owner chose level 6's rows on the N2 and AMD as the next gap and "Draft the dear loop entry" the same day, and ruled the same day: "Approve, aarch64 first". It amends decision 53, which named the dear blocks of decision 50 as outside its loop. The owner's bar: no level may write more or run slower than libdeflate, on any row (decision 48's entry).

    Where level 6 stands. On the N2 at main 5a7ebcc's code, level 6 runs below libdeflate on 4 rows: x-ray 0.865, ooffice 0.919, mr 0.961 and mozilla 0.998 (runs 40356831116 and 40356838255, second attempts). x-ray and ooffice spend most of their positions in dear blocks.

    Where the instructions go. Callgrind on aarch64, level 6, the first 2 MiB, instructions an octet, at origin/main 5aa7845:

    Row	stdx	libdeflate	The dear loop	The writer
    x-ray	107.0	81.2	about 81	16.8
    ooffice	99.5	80.2	about 78	11.9
    mr	90.6	87.3	under 4	8.0

    mr runs decision 53's loop, at 72.6, and stands at libdeflate's count. Its gap is the mispredicts of decision 58's study, not this loop. The dear loop holds x-ray's and ooffice's.

    What decision 53's loop gave the plain blocks. 11% to 17% fewer instructions an octet, and level 6 at 1.064 to 1.068 of main's speed on the N2.

    The rule.

    • Decision 53's rule, its checks and its measurement hold, for one more loop: advance_lazy in encoder_match_dear.zig, with find, decide, take_waiting, the 3-octet table's insert, probe and tally, and the tail walk inline. aarch64 first. The Zig loop stays the reference.
    • The loop does what the Zig loop does, in the same order of effect: each position joins its chain and the table; it searches where the chain's nearest candidate or the table's holds its 4 octets, after a match of 4 or more shorter than lazy_len, and not after one of lazy_len; after a literal with no match it takes the table's 3-octet match; the lazy step keeps decision 36's score; a taken match's covered positions join both; a 3-octet match is counted for the table's verdict.
    • It reuses decision 53's text for the walk, the compare and the symbol's stores, and adds the table's insert and probe, the score, and the tally. The template takes every limit from constants.zig, comptime asserts pin each, and each loop stops at a named bound.
    • advance_lazy in encoder_match_dear.zig calls it where decision 53's guard holds: level 6, an aarch64 target built by LLVM. x86-64 waits for its own ruling, under decision 54's vendor rule.

    The checks. As decision 53's: an equality test of both loops on copies of one matcher, one block and one table, over seeded windows and stopped streams, each exit and branch reached, the table's every state; a fuzz test; differential-encode's hashes in ReleaseSafe and Debug; the mutations of each compare; test-self-hosted and test-avx512.

    The measurement that admits it. Under decision 20, pairs against main on the N2: x-ray and ooffice faster past the larger of 2% and their spread in both jobs, and no row at any level slower past the larger of its spread and 1% in both jobs, unless decision 20's placement amendments exempt it. The M1 times it row by row before the owner rules on landing.

    What the aarch64 loop gave. At 5bf1eae against main (5aa7845), runs 40502111438 and 40502114668, each job on one VM:

    • Neoverse N2, both jobs: x-ray 1.122 and 1.133 of main's speed at level 6, ooffice 1.074 and 1.077, 15 rows faster in both jobs and none slower at any level. Over libdeflate, level 6 leaves 3 rows below it where 4 were: mr 0.969, x-ray 0.976 and ooffice 0.992.
    • The x86-64 jobs, two on an EPYC 9V74, run the Zig loop: no row moved in both.
    • Callgrind, instructions an octet: x-ray 107.0 became 97.05, ooffice 99.5 became 90.52.
    • Output: every row's octets are main's; differential-encode passes with main's hashes.
    • The M1, 5 runs a row at a load near 6, beside a copy of main against main whose rows read 0.976 to 1.028: 1.005 at the median, x-ray 1.101 and ooffice 1.075 at 0.91 and 0.90 of main's instructions. One row read slower past 1%, cp.html at 0.976 at equal instructions, within the copy's spread. So the loop stays under the rule above.

    What it risks. As decision 53's: a wrong bound reads or writes outside the encoder's state with no panic. The equality test, the fuzz test, differential-encode and the mutations stand against it.

60. The Zstandard encoder begins a frame with start, its tables tagged by frame. Ruled by the owner on 2026-10-27, who chose it as the first of the encoder's speed work (decision 57). Drafted as decisions 58 and 59, whose numbers decisions 58 and 59 now hold.

    init clears the match finder's tables, 256 KiB at level 1 and 768 KiB at level 3, which costs more than a small body's encoding. start begins the next frame of an encoder init started, and clears nothing:

    • A table entry holds a window position in its low window_log + 1 bits, as the window and a block hold fewer octets than twice the window. The bits above it, 9 at the largest window, hold the tag of the frame that stored it.
    • init clears the tables and sets the tag to 1. start adds 1 to the tag, and clears the tables again when the tag passes its bits: once in 512 frames at the largest window.
    • An entry is the frame's base, its tag shifted above every position, plus the position. Tags only grow between clears, so an earlier frame's entries lie below the base. One range check then finds a candidate: an entry from base + max(low, 1) to below base + position. find computes both ends once a position, and the entry it stores is the second end.
    • An entry of an earlier frame names no candidate, as a cleared one does. So a frame begun with start writes the octets of one begun with init (invariants 4 and 5). differential-zstd-encode encodes every file again from the same state with start, and requires the same octets.
    • A slide moves this frame's positions down, and empties the ones that slid out and every earlier frame's, eight entries a step with a vector select. Its subtraction wraps: a checked one kept a branch on every entry, which mispredicts on tables that mix both kinds of entry. That form ran nci 0.59 to 0.78 of main's speed at level 3.

    Paired against main on the runners (run 40535059200, reports bench/results/2026-10-27-zstd-encoder-start-*.md), stdx's encoder over main's:

    CPU	Level	Median	1 KiB bodies	Slowest row
    Neoverse N2	1	1.01	1.65 to 1.88	0.979, ptt5
    Neoverse N2	3	1.03	2.97 to 3.71	0.961, shuffled dickens-1m
    EPYC 9V74	1	0.99	1.54 to 1.77	0.967, sao
    EPYC 9V74	3	1.01	2.62 to 3.28	0.974, html-1m

    About 15 rows per CPU read 0.96 to 0.99, past their spreads. The owner ruled on 2026-10-27 that it lands with them.

    start is a programmer's promise that init ran on this state before; on memory init never cleared, an entry's tag could be anything. The benchmark keeps each encoder across frames, as a server keeps one per connection or thread: stdx's state with start, and libzstd's context.

    The alternatives refused:

    • Clearing only the part of each table a frame used: a frame of a few octets still touches entries across the whole table, by its hash.
    • Tables sized to the first block's length: the hash's width would depend on how the input arrived, and a stream would need one table size for its whole frame.