Sub-sampling-aware analytic chroma weighting for the Qfactor path (+ MCT guard fix)

[osamu620]

Summary

Makes the analytic CSF visual weighting (Qcsf=mannos|daly) produce correct per-subband weights for chroma components sub-sampled to 4:2:0 / 4:2:2, and fixes a heap-corrupting Cycc guard bug found along the way. Encoder-only: no bitstream syntax change, no decoder change, no conformance impact — only the emitted QCD/QCC step sizes differ, and only when the new options are used.

Background

The analytic chroma path already derived the 4:2:0/4:2:2 weight tables from a single chroma CSF by shifting each axis's frequency by the sub-sampling factor (sx, sy). But for genuinely sub-sampled input that code was unreachable: sub-sampled components rule out the built-in MCT, the codestream therefore cannot label channels, and the no-MCT branch routed every component to the luminance CSF. Sub-sampled YCbCr input silently got full-resolution luma weights on Cb/Cr.

Changes

• Qctype=Y,Cb,Cr (encoder) / --ctype (estimate_qfactor): per-component role hints for no-MCT input, mirroring the convention of other encoders' component-type hints. Hinted Cb/Cr take the low-pass chroma CSF with the component's SIZ XRsiz/YRsiz folded into the per-axis frequency mapping. Unhinted components stay generic (luminance CSF) — now also with the (sx, sy) shift. New API: set_component_types().
• Per-axis frequency mapping everywhere: (sx, sy) now come from the per-component SIZ fields (threaded through QCC_marker), with the 4:4:4/4:2:0/4:2:2 chroma_format mapping kept as a fallback. luma_visual_weights() gains the same per-axis form (LH from the vertical axis, HL from the horizontal); at (1, 1) it is bit-identical to the previous single-radial form by construction.
• Qchromacsf=chroma|luma / --chroma-csf: A/B switch that keeps the sub-sampling frequency mapping but evaluates the luminance CSF for Cb/Cr — separates the effect of the CSF shape from the mapping. New API: set_chroma_csf_reuse_luma().
• visual_weight_check (new test tool): dumps the derived weight table for any (ctype, sx, sy, ppd, zoom, levels) configuration (one command per ITU-T T.802 Annex B comparison) and asserts the mapping invariants in check mode.
• Docs: docs/qfactor.md and docs/cli_encoder.md cover the new options, why a hint is needed, and the two properties of the mapping (below).

Fix: Cycc guard heap corruption (first commit)

The guard that disables the color transform for non-identically-sized components required a mismatch on both axes. 4:2:2-shaped input (horizontal-only mismatch) therefore kept the ICT enabled across differently-sized component buffers, corrupting the heap — reproducible with a plain 3-component PGX encode at default Cycc=yes, no special options:

open_htj2k_enc -i y64x64.pgx,c32x64.pgx,c32x64.pgx -o out.j2c Qfactor=85
double free or corruption (out)

The guard now ORs the per-axis checks (either-axis mismatch rules out the MCT), at both the buffered and streaming entry points. enc_qf_422_legacy regression-guards this. Split into its own commit so it can be cherry-picked independently.

Correctness properties (asserted by qf_vw_invariants)

1. 4:4:4 / legacy byte-identity. Defaults are unchanged; the existing golden-byte test still pins legacy output, and all analytic 4:4:4 encodes were verified byte-identical against pre-change builds across Qfactor × model × zoom. The (1,1) weight vectors are additionally pinned in-tree to 17 significant digits.
2. Isotropic shift. (2, 2) sub-sampling equals a one-level shift exactly (bit-for-bit): the classic "weight chroma like luma one level down" rule falls out of the frequency mapping, so no manual level offset exists (applying one would double-count).
3. 4:2:2 anisotropy. With (2, 1), HL (horizontal detail) strictly outweighs LH at every level — the property a scalar per-subband table cannot represent, and the signature that the per-axis mapping is live.

Tests

• qf_vw_invariants — the invariant suite above.
• enc/qfest_qf_420_ctype, enc/qfest_qf_422_ctype — encode ↔ estimate_qfactor round-trips at residual 0 through the real SIZ-driven plumbing.
• qfest_420_ctype_required — negative control: estimating without the hints must fail, proving the chroma CSF actually reached the emitted QCC bytes.
• enc/qfest_qf_420_reuseluma — the A/B switch round-trip.
• enc/qfest_qf_422_legacy — legacy 4:2:2 encode (crash regression for the guard fix).
• Tiny synthetic 4:2:2 PGX fixtures added (~8 KB total; QCD/QCC bytes never depend on sample data). The 4:2:0 tests reuse existing conformance references.

Full suite: 473/473 pass.

🤖 Generated with Claude Code

https://claude.ai/code/session_01M9cKpzqghsLxJUu7kkyRAw