perf(deflate): relax each match length at its own nearest distance

crates/gamut-deflate/README.md

@@ -28,18 +28,24 @@ zlib streams than `zlib -9` and `miniz_oxide` at their maximum levels** on every
 while staying close to `zopfli` — all behind one dependency-free `Level` knob that also spans the
 fast tiers, so codecs don't juggle two external crates.
 
-Output size in bytes (zlib streams; lower is better), reproduced by `cargo bench -p gamut-deflate`:
+Output size in bytes (zlib streams; lower is better). Every row except the pinned `lz77.rs` row is
+reproduced by `cargo bench -p gamut-deflate`; that row is historical — see the note below the table:
 
 | input                         |    raw | `Level::Best` | `zlib -9` | `miniz_oxide`-10 | `zopfli` |
 | ----------------------------- | -----: | ------------: | --------: | ---------------: | -------: |
-| RFC 1951 spec text (36 KB)    | 36 945 |    **10 767** |    11 112 |           11 130 |   10 544 |
+| RFC 1951 spec text (36 KB)    | 36 945 |    **10 664** |    11 112 |           11 130 |   10 544 |
 | English text ×300             | 13 500 |       **103** |       110 |              107 |      103 |
-| Rust source (`lz77.rs`)       | 13 321 |     **4 362** |     4 461 |            4 470 |    4 320 |
+| Rust source (`lz77.rs` at `4f2c2a4`) | 25 031 | **8 003** |     8 268 |            8 269 |    7 950 |
 | pseudo-random (~incompressible)| 20 000 |     **2 236** |     2 290 |            2 291 |    2 122 |
 
-`Level::Best` lands ~1–7% below `zlib -9`; `zopfli` (15 optimization passes + package-merge
-length-limiting, vs. this crate's default 6 passes + a count-floor heuristic) is a few percent
-smaller again at a much higher cost. The pass budget is configurable via
+The `lz77.rs` row compresses this crate's own match finder, so it is pinned to the file at commit
+`4f2c2a4` (25 031 bytes): a later `cargo bench` run compresses whatever the file is by then, and
+reproduces this row only against that revision of the file.
+
+`Level::Best` lands ~2–7% below `zlib -9` and within ~1% of `zopfli` on real text and source, now
+that the optimal parse prices each match length at its own nearest distance (zopfli's `sublen`)
+rather than at the longest match's; what remains of the gap is `zopfli`'s 15 optimization passes +
+package-merge length-limiting, vs. this crate's default 6 passes + a count-floor heuristic. The pass budget is configurable via
 `DeflateEncoder::with_effort` (0 = the lazy seed parse only; 15 ≈ zopfli's budget), so size-vs-time
 curves can be swept along one axis. `DeflateEncoder::with_optimal_parse_limit` is the second axis:
 the optimal parse works in spans of at most 1 MiB by default, each with its own refined cost model

crates/gamut-deflate/STATUS.md

@@ -34,11 +34,43 @@ that need it use `miniz_oxide`. See the crate docs' *Encoder only* / *Scope* sec
 is a few percent smaller again at a much higher, non-configurable cost. Full table in `README.md`;
 the numbers are regenerated by `cargo test --test oracle` / `cargo bench`.
 
+The optimal parse relaxes **each match length at its own nearest distance** (#479). `Matcher::find`
+walks the hash chain nearest-first and returns only the longest match; `Matcher::find_sublen` runs
+the same walk and also fills a caller-owned `[u16; MAX_MATCH + 1]` in which `sublen[len]` is the
+nearest distance achieving at least `len` (zopfli's `sublen`) — a candidate records exactly the
+lengths it is the first to reach, so the first to reach a length is the nearest that does.
+`parse_dp` prices length `len` at `sublen[len]`, where before it priced every length from
+`MIN_MATCH` to the longest at the longest match's distance, forcing a short match to pay a far
+distance code the cost model had every reason to make expensive. The lazy parser
+(`Level::Fast`/`Default`) still uses `find` and is byte-identical. Measured with
+`cargo bench -p gamut-deflate --bench compression` (its size table, `before` at the #478 head;
+the `lz77.rs` row is the file at that head for both columns, since this change edits it):
+
+| input                | raw    | `Best` before | `Best` after | `zopfli` | gap to `zopfli` before → after |
+| -------------------- | -----: | ------------: | -----------: | -------: | ------------------------------ |
+| `text.x300`          | 13 500 |           103 |          103 |      103 | 0.0% → 0.0% |
+| `ramp20k`            | 20 000 |           399 |          399 |      357 | +11.8% → +11.8% |
+| `pseudo20k`          | 20 000 |         2 236 |        2 236 |    2 122 | +5.4% → +5.4% |
+| `rfc1951.txt`        | 36 945 |        10 767 |   **10 664** |   10 544 | +2.1% → +1.1% |
+| `lz77.rs` (#478 head)| 25 031 |         8 055 |    **8 003** |    7 950 | +1.3% → +0.7% |
+
+In `tests/default_bytes.rs` two `Best` pins moved, both down — `skewed` 1061 → 1048, `mixed`
+21643 → 21601 — and no row grew at any level. The synthetic rows are unchanged because their matches
+are either all at one period (`ramp20k`, a 256-byte sawtooth, so distance 256; `text.x300`, a
+45-byte sentence) or essentially absent (`pseudo20k`), so the nearest distance for a short length
+*is* the longest match's distance. The
+remaining gap on real inputs is the pass budget and the count-floor length limiter (below).
+`Level::Best` pays for the table: on the `rfc1950.txt` throughput input its fastest sample went
+22.88 → 24.68 ms (+7.9%; median 23.34 → 24.98 ms), measured back to back in one session — where the
+matcher-free `Store` row also moved 7% between the two runs, so that figure is an upper bound.
+`Default`/`Fast` do not fill the table and moved within that noise (+1.3% / +0.6%).
+
 ### Throughput (measured)
 
-`Matcher::find` — the hash-chain longest-match loop, run up to `max_chain` (1024 at `Level::Best`)
-times per input position — measures each candidate that survives its one-byte prune **eight bytes
-at a time** (#478): both windows are read as a `u64`, the first differing byte is
+`Matcher::walk` — the hash-chain longest-match loop behind `find` (the lazy parser) and
+`find_sublen` (the optimal parse, #479), run up to `max_chain` (1024 at `Level::Best`) times per
+input position — measures each candidate that survives its one-byte prune **eight bytes at a
+time** (#478): both windows are read as a `u64`, the first differing byte is
 `(x ^ y).trailing_zeros() / 8`, and the sub-word tail is compared byte-wise. Pure safe Rust, no
 dependency, identical on every target, and byte-identical output — the same matches are found,
 only measured faster; `tests/default_bytes.rs` pins the exact bytes at every level and passed

crates/gamut-deflate/src/lz77.rs

@@ -89,6 +89,42 @@ impl Matcher {
     /// prune in front of it is still a single byte: it rejects most candidates outright, and a
     /// wide read there would touch bytes past `best_len` for nothing.
     fn find(&self, data: &[u8], pos: usize, max_chain: usize, limit: usize) -> Option<(u16, u16)> {
+        self.walk(data, pos, max_chain, limit, None)
+    }
+
+    /// [`find`](Self::find), and additionally fills `sublen` so that `sublen[len]` is the
+    /// **nearest** distance at which a match of at least `len` bytes exists, for every `len` from
+    /// `MIN_MATCH` up to the returned longest length; every other entry is `0`.
+    ///
+    /// The table is what lets the optimal parse price a short match at its own cheapest distance
+    /// instead of at the distance of the longest one (zopfli's `sublen`). The chain is walked
+    /// nearest-first and a candidate only records the lengths it is the first to reach, so the
+    /// first candidate reaching a length is the nearest that does.
+    ///
+    /// The table is caller-owned so a parse can allocate it once and reuse it at every position.
+    fn find_sublen(
+        &self,
+        data: &[u8],
+        pos: usize,
+        max_chain: usize,
+        limit: usize,
+        sublen: &mut [u16; MAX_MATCH + 1],
+    ) -> Option<(u16, u16)> {
+        sublen.fill(0);
+        self.walk(data, pos, max_chain, limit, Some(sublen))
+    }
+
+    /// The chain walk shared by [`find`](Self::find) and [`find_sublen`](Self::find_sublen): the
+    /// longest match, plus the per-length nearest distances when a table is supplied. The table is
+    /// optional so the lazy parser, which needs only the longest match, pays nothing for it.
+    fn walk(
+        &self,
+        data: &[u8],
+        pos: usize,
+        max_chain: usize,
+        limit: usize,
+        mut sublen: Option<&mut [u16; MAX_MATCH + 1]>,
+    ) -> Option<(u16, u16)> {
         if pos + MIN_MATCH > limit {
             return None;
         }
@@ -111,8 +147,14 @@ impl Matcher {
                 // `c < pos`, so both windows end at or before `limit <= data.len()`.
                 let len = common_prefix_len(&data[c..c + max_len], &data[pos..pos + max_len]);
                 if len > best_len {
+                    let dist = pos - c;
+                    // Lengths up to `best_len` were reached by a nearer candidate already; this one
+                    // is the first -- hence the nearest -- to reach `best_len + 1..=len`.
+                    if let Some(table) = sublen.as_deref_mut() {
+                        table[best_len + 1..=len].fill(dist as u16);
+                    }
                     best_len = len;
-                    best_dist = pos - c;
+                    best_dist = dist;
                     if len >= max_len {
                         break; // can't do better than the maximum
                     }
@@ -310,6 +352,8 @@ fn parse_dp(
     f[0] = 0;
     let mut matcher = Matcher::new();
     matcher.prime(data, start);
+    // The per-length nearest-distance table, allocated once and refilled at every position.
+    let mut sublen = [0u16; MAX_MATCH + 1];
     for i in 0..n {
         let fi = f[i];
         // A literal always advances one byte.
@@ -319,13 +363,21 @@ fn parse_dp(
             blen[i + 1] = 0;
             bdist[i + 1] = 0;
         }
-        let found = matcher.find(data, start + i, max_chain, end);
+        let found = matcher.find_sublen(data, start + i, max_chain, end, &mut sublen);
         matcher.insert(data, start + i);
-        if let Some((max_len, dist)) = found {
-            let (dsym, dbits, _) = symbols::distance_code(dist);
-            let dcost = u64::from(dist_cost[dsym as usize]) + u64::from(dbits);
-            // Every length from MIN_MATCH up to the longest match is reachable at this distance.
+        if let Some((max_len, _)) = found {
+            // Every length from MIN_MATCH up to the longest match is reachable, each at its own
+            // nearest distance: a short match need not pay the distance code of the longest one.
+            // The distance is non-decreasing in the length and changes only a few times over the
+            // range, so its cost is re-derived only when it does.
+            let mut dist = 0u16;
+            let mut dcost = 0u64;
             for len in MIN_MATCH..=max_len as usize {
+                if sublen[len] != dist {
+                    dist = sublen[len];
+                    let (dsym, dbits, _) = symbols::distance_code(dist);
+                    dcost = u64::from(dist_cost[dsym as usize]) + u64::from(dbits);
+                }
                 let (lsym, lbits, _) = symbols::length_code(len as u16);
                 let cost = fi + u64::from(lit_cost[lsym as usize]) + u64::from(lbits) + dcost;
                 if cost < f[i + len] {
@@ -411,6 +463,71 @@ mod tests {
         assert_eq!(common_prefix_len(&a[..11], &a), 11);
     }
 
+    /// Two candidates for the final `abcde`: the nearer (`abcX`, distance 7) reaches length 3, the
+    /// farther (`abcde`, distance 15) reaches 5. Each length must be recorded at the nearest
+    /// distance that achieves it -- 3 at 7, and 4 and 5 at 15 -- with the longest match unchanged
+    /// and every length past it left at zero.
+    #[test]
+    fn find_sublen_records_each_length_at_its_nearest_distance() {
+        let data = b"abcdeQQQabcXQQQabcde";
+        let pos = 15;
+        let mut matcher = Matcher::new();
+        matcher.prime(data, pos);
+        let mut sublen = [0xFFFFu16; MAX_MATCH + 1];
+        let found = matcher.find_sublen(data, pos, 128, data.len(), &mut sublen);
+        assert_eq!(found, Some((5, 15)));
+        assert_eq!(found, matcher.find(data, pos, 128, data.len()));
+        assert_eq!(
+            sublen[MIN_MATCH], 7,
+            "length 3 is reachable at the nearer candidate"
+        );
+        assert_eq!(sublen[4], 15);
+        assert_eq!(sublen[5], 15);
+        assert!(
+            sublen[..MIN_MATCH].iter().all(|&d| d == 0) && sublen[6..].iter().all(|&d| d == 0),
+            "lengths with no match are zero"
+        );
+    }
+
+    /// With far distance codes priced high and near ones low, the shortest path through the second
+    /// `abc` of `abcabcde` is a length-3 match at distance 3 plus two literals (9 + 16 = 25 bits),
+    /// not the longest match: length 5 back to the opening `abcde`, 44 bytes away, costs 8 + 15 +
+    /// 4 extra bits = 27 for the same five bytes. That holds only when length 3 is offered at its
+    /// own distance -- relaxed at the longest match's distance it costs 27 as well and the far
+    /// match wins -- so this pins that the DP sees the nearer one.
+    #[test]
+    fn optimal_parse_takes_a_short_match_at_its_nearer_distance() {
+        let mut data = b"abcde".to_vec();
+        data.extend(0x80u8..0xA4); // 36 distinct bytes: no repeated 3-gram, no match
+        data.extend_from_slice(b"abcabcde");
+        let far = 44;
+        assert_eq!(&data[far - 3..far], b"abc");
+        assert_eq!(&data[far..], b"abcde");
+        // Literals and length symbols: 8 bits each. Distances 1..=4 (codes 0..=3): 1 bit; every
+        // other distance code: 15 bits (plus its extra bits).
+        let lit_cost = vec![8u16; 286];
+        let mut dist_cost = vec![15u16; 30];
+        dist_cost[..4].fill(1);
+        let tokens = parse_dp(&data, 0, data.len(), 128, &lit_cost, &dist_cost);
+        let leading = far; // "abcde", the filler and the first "abc" are all literals
+        assert!(
+            tokens[..leading]
+                .iter()
+                .all(|t| matches!(t, Token::Literal(_))),
+            "{tokens:?}"
+        );
+        assert_eq!(
+            &tokens[leading..],
+            &[
+                Token::Match { len: 3, dist: 3 },
+                Token::Literal(b'd'),
+                Token::Literal(b'e')
+            ],
+            "the short match must take the near distance"
+        );
+        assert_eq!(reconstruct(&tokens), data);
+    }
+
     #[test]
     fn all_literals_when_no_repeats() {
         let data = [1u8, 2, 3, 4, 5];

crates/gamut-deflate/tests/default_bytes.rs

@@ -184,7 +184,7 @@ const PINS: &[Pin] = &[
         digest: 0xb60f_deee_b11e_7422,
     },
     // The clearest ladder in the corpus, and the one that isolates the code-length optimiser:
-    // 1262 -> 1147 -> 1061 on an input compressible almost entirely through Huffman lengths.
+    // 1262 -> 1147 -> 1048 on an input compressible almost entirely through Huffman lengths.
     Pin {
         fixture: "skewed",
         level: Level::Fast,
@@ -200,8 +200,8 @@ const PINS: &[Pin] = &[
     Pin {
         fixture: "skewed",
         level: Level::Best,
-        len: 1061,
-        digest: 0xf014_21a7_4fd2_b82b,
+        len: 1048,
+        digest: 0xd97b_facb_22a9_10d5,
     },
     // Byte-identical at all three levels: every rung recognises the input as incompressible and
     // falls back to the stored floor, 4101 bytes for 4096 of payload. A cost model that stopped
@@ -241,8 +241,8 @@ const PINS: &[Pin] = &[
     Pin {
         fixture: "mixed",
         level: Level::Best,
-        len: 21643,
-        digest: 0x6a34_6952_e9f4_9523,
+        len: 21601,
+        digest: 0x6b73_d9e2_9361_4f30,
     },
 ];