perf(decoder): N-way lockstep step-1 interleave for the HT cleanup decoder (AVX2 + scalar, N=2)#459
Merged
Merged
Conversation
…/ finish Factor the HT block decoder around its existing two-phase seam so the serial step-1 chain (MEL + CxtVLC + UVLC) can later be interleaved across codeblocks: - ht_block_decoding.hpp gains ht_cleanup_step1_nway<N, skip_sigma>, a lockstep N-lane transcription of the former phase-1 loop shared by all ISA files. Per-lane state lives in N-arrays and every sub-step is an unrolled for-lane loop (OpenJPH's own AVX2-decoder structure); the per-lane operation sequence is unchanged, so output is byte-identical. MEL_dec / rev_buf get default ctors + init() for lane arrays. - The AVX2 and scalar files' ht_cleanup_decode monoliths become ht_cleanup_step2<fuse_dequant, store_i32> (MagSgn only); htj2k_decode is now htj2k_dec_setup (validation + modDcup) -> step1 at N=1 -> htj2k_dec_finish (step-2 dispatch + SigProp/MagRef + dequantize). - New htj2k_decode_batch entry point declared in block_decoding.hpp with trivial per-block-loop definitions in all four ISA files and a link-time htj2k_dec_batch_lanes constant (1 everywhere for now). No driver changes and no behavioral change. Verified byte-identical output on five bench fixtures (8r/16r/8i/8r_512x8/kdu_8mp multipass) for both AVX2 and scalar builds; full ctest passes on the AVX2 build. Same-moment cycle A/B against the unsplit baseline is within noise on every fixture for both builds (instructions flat to 0.02%), i.e. the N=1 instantiation collapses back to the original code. (The two security_threaded_decode_abort*_mt failures on scalar-only builds predate this change: those x86-gated fixtures rely on the AVX2 reader throwing, which the scalar reader does not do.) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M9cKpzqghsLxJUu7kkyRAw
Enable the N-way step-1 kernel at N=2 on AVX2 and wire the serial decode driver to feed it: - htj2k_decode_batch (AVX2) now walks the handed-in blocks and decodes pairs of consecutive codeblocks with ht_cleanup_step1_nway<2> whenever they share dimensions and pass-class and both setups validate; step-2 and the refinement passes stay per block in list order, so output is byte-identical. Setup is not idempotent (modDcup mutates the buffer), so every setup is consumed exactly once; on a mid-lockstep throw from malformed input the group is redone lane-by-lane at N=1 from the saved setups, reproducing per-block 1-way results and exceptions. - decode_strip_core's serial path forms greedy runs of equal-sized decodable HT codeblocks (up to htj2k_dec_batch_lanes) and provisions the grow-on-demand scratch buffers with per-lane offsets. ISAs without a batch kernel report lanes=1 and keep the per-block path. Whole-decode cycles vs main (Zen 5, -num_threads 1, min-of-15, same-moment A/B of separately built trees): 8r -7.7%, 16r -5.0%, 8i -6.6%, 8r_512x8 -2.1% (pairing rate ~50% by design), kdu_8mp multipass -0.6%. The split-only refactor accounts for +0.5% of that (within its ±1% gate); the interleave more than pays it back. Byte-identical output on all five fixtures at 1/2/4/6 threads; full ctest passes (463/463) on the AVX2 build. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M9cKpzqghsLxJUu7kkyRAw
…build Mirror the AVX2 batched entry in the scalar TU: htj2k_decode_batch now forms pairs of equal-dimension, equal-pass-class codeblocks and decodes their step-1 chains with ht_cleanup_step1_nway<2>, with the same setup-consumed-once and lane-redo-on-throw semantics. The scalar step-2 is untouched. This also repays the split cost the shared-kernel refactor put on the scalar build (+0.6..2.7% cycles): vs main, whole-decode is now 8r -3.8%, 16r -3.9%, 8i -3.9%, 8r_512x8 -2.7%, kdu_8mp -1.8% (same-moment A/B, min-of-15). Byte-identical output on all five fixtures; ctest matches main (the two security_threaded_decode_abort*_mt failures are pre-existing on AVX2-less builds — their fixtures rely on the AVX2 reader throwing). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M9cKpzqghsLxJUu7kkyRAw
…ed HT decoder Group adjacent equal-sized HT codeblock tasks (up to htj2k_dec_batch_lanes) in decode_strip_core's parallel path and trigger_prefetch, and push one thread-pool task per group calling htj2k_decode_batch. par_cnt now counts groups; each group task decrements it exactly once, so the spin_wait / drain_prefetch semantics (including the PR #445 prefetch-drain invariant) are unchanged. Per-block scratch offsets into the shared pools were already disjoint, so grouping only changes task granularity. Group descriptors are packed into a uint64 so the pushed closure stays within std::function's small-buffer optimisation. Wall clock vs main (-iter 8, min-of-12+): 8r -3.7/-2.8/-3.6% and 8i -9.6/-3.0/-4.9% at -num_threads 2/4/6. Byte-identical output at 1/2/4/6 threads; 463/463 ctest; ThreadSanitizer clean on both a 4-thread normal decode and the single-tile-reuse loop (region_decode_bench) at 4 and 6 threads. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M9cKpzqghsLxJUu7kkyRAw
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Step-1 of the HT cleanup pass (MEL + CxtVLC + UVLC decoding) is a serial per-quad dependency chain — about 60% of block-decode cycles on lossy single-pass streams — and single-block micro-optimization of it is exhausted. Codeblocks are independent, so this PR decodes the step-1 chains of two neighbouring codeblocks in lockstep, letting the out-of-order engine overlap the two chains. (OpenJPH ships the same optimization in its AVX2/AVX-512 decoders; both projects are BSD.)
What changed
ht_cleanup_step1_nway<N, skip_sigma>inht_block_decoding.hpp): a lockstep N-lane transcription of the existing phase-1 loop. Per-lane state lives in N-arrays, every sub-step is an unrolled per-lane loop, and each lane's operation sequence is unchanged — output is byte-identical by construction.MEL_dec/rev_bufgain default ctors +init()for lane arrays.htj2k_decode=htj2k_dec_setup(validation + modDcup) → step-1 →htj2k_dec_finish(MagSgn step-2 dispatch + SigProp/MagRef + dequantize). The 1-way path uses the same kernel at N=1 and measures within ±1% of the old monolith (instructions actually drop ~5%).htj2k_decode_batch(declared inblock_decoding.hpp, real N=2 kernels in the AVX2 and scalar TUs, per-block loop elsewhere): forms groups of equal-dimension, equal-pass-class blocks; step-2 stays per block in list order. Setup mutates the compressed buffer (modDcup), so every setup is consumed exactly once; if a lane'srev_bufunderflows mid-lockstep on malformed input, the group is redone lane-by-lane at N=1 from the saved setups, reproducing per-block 1-way results and exceptions exactly.subband_row_buf.cpp): the serial path forms greedy equal-size HT runs with per-lane offsets into the grow-on-demand scratch; the MT strip and prefetch paths group adjacent tasks and push one pool task per group (par_cntcounts groups; spin-wait/drain semantics unchanged).OPENHTJ2K_HT_DEC_BATCH_N, default 2). N=4 was measured and rejected: +1.0…+3.7% cycles vs N=2 (register/L1d pressure beats the extra ILP).Performance (Zen 5, min-of-15 cycles, same-moment A/B of separately built trees)
Whole-decode vs
main,-num_threads 1:Multithreaded wall clock (
-iter 8): 8r −3.7/−2.8/−3.6% and 8i −9.6/−3.0/−4.9% at 2/4/6 threads.This closes the standing whole-decode gap to the OpenJPH 1-way reference decoder from +10…+12% to +1.0% (8r) / +2.8% (8i) / +5.3% (16r); the 512×8 fixture keeps a larger gap because only ~50% of its blocks can pair (row widths 512,512,512,384).
Verification
security_threaded_decode_abort*_mtfailures there are pre-existing: those x86-gated fixtures rely on the AVX2 reader throwing, which the scalar reader never did — a follow-up should gate them on AVX2 being enabled, not on x86_64).region_decode_bench) at 4 and 6 threads (the prefetch-drain scenario).Not in this PR
NEON and WASM keep their fused single-pass kernels (lanes=1): enabling the interleave there first requires restructuring each into the two-phase-over-scratch form, which is its own measurable, gated change per ISA. Planned as separate PRs.
🤖 Generated with Claude Code
https://claude.ai/code/session_01M9cKpzqghsLxJUu7kkyRAw