Skip to content

test(vetting): verify 2D first-order with MATLAB R2026a - #468

Open
vjaganat90 wants to merge 2 commits into
PolusAI:mainfrom
vjaganat90:2d_firstorder_matlab_script
Open

test(vetting): verify 2D first-order with MATLAB R2026a#468
vjaganat90 wants to merge 2 commits into
PolusAI:mainfrom
vjaganat90:2d_firstorder_matlab_script

Conversation

@vjaganat90

@vjaganat90 vjaganat90 commented Sep 1, 2026

Copy link
Copy Markdown
Member

Why this is needed

The 2D first-order family still carried a MATLAB correctness claim without a checked-in MATLAB generator. The registry claimed 33 oracle=matlab features, while test_2d_firstorder_matlab.h contained 28 literals with no tool version, exact functions, recipe, benchmark, or runnable provenance. Those two sets did not even agree: six registry claims had no MATLAB assertion, while the header called histogram UNIFORMITY a MATLAB assertion even though the registry correctly treated PyRadiomics as its oracle.

The old audit had already found the central problem: several percentile literals were Nyxus outputs, not values produced by MATLAB or Octave. This PR replaces that inherited claim with a licensed MATLAB R2026a run and lets the measured results decide the disposition feature by feature.

What the licensed run establishes

gen_firstorder2d_matlab.m downloads the canonical pixel fixture and golden header from the moving PolusAI/nyxus main tree. It extracts the 154 fixture intensities directly from tests/test_data.h, invokes native MATLAB statistics built-ins, checks the generated and pinned feature sets in both directions, prints full-precision values side by side, and fails if a pin exceeds its declared band.

MATLAB honestly covers 31 features:

  • Direct built-ins cover extrema, mean/median/mode, sample and population variance/stddev, skewness, kurtosis, MAD, percentiles, IQR, RMS, energy, and integrated intensity.
  • Derived values use only the feature-defining arithmetic over those built-ins: COV, excess kurtosis, fifth/sixth standardized moments, QCOD, standard error, PIU, and slide-relative intensity range.
  • No Nyxus histogram or first-order algorithm is reproduced.

The C++ table now pins the exact MATLAB R2026a values. For the eight percentile-derived features, Nyxus uses its fixed 100-bin interpolated CDF while MATLAB evaluates the raw sample. They still agree within the measured rel=3e-2 band; the worst residual is QCOD at 2.75%. The other 23 assertions use the SPEC same-definition tier, rel=1e-3.

This restores honest MATLAB coverage for P01, P25, P75, P99, and QCOD, and replaces the old Nyxus-derived pins for P10, P90, and INTERQUARTILE_RANGE with the actual MATLAB values.

What remains outside MATLAB

This PR does not force incompatible semantics into the MATLAB file:

  • UNIFORMITY remains PyRadiomics-vetted. Producing it here would require reconstructing Nyxus histogram discretization.
  • MEDIAN_ABSOLUTE_DEVIATION is moved to regression: MATLAB mad(x,1) is the median absolute deviation, while Nyxus computes the mean absolute deviation about the median.
  • ROBUST_MEAN remains regression: MATLAB rank trimming and Nyxus histogram-derived P10/P90 selection are different definitions.
  • ENTROPY and ROBUST_MEAN_ABSOLUTE_DEVIATION remain PyRadiomics-vetted. The redundant standalone PyRadiomics assertion for the latter is removed; the table-driven oracle assertion already covers it. (I can revert this redundant code elimination but I think we should remove it here)

The honest family result is therefore 34/36 features vetted by at least one oracle, with the two unresolved semantics explicitly regression-only. The generated global report drops two stale vetting claims instead of preserving an inflated headline.

SPEC alignment

  • Adds the licensed MATLAB R2026a generator and complete golden-site provenance.
  • Consolidates the 31 MATLAB comparisons into one table-driven gtest with per-feature tolerance selection.
  • Adds firstorder2d.md, enumerating the MATLAB, PyRadiomics, IBSI, and two production-only regression config points.
  • Defines the canonical 154-pixel benchmark and every referenced config recipe.
  • Rewrites the family registry as 64 assertion rows: 31 MATLAB, 18 PyRadiomics, 12 IBSI, and 3 regression. Every row names its exact recipe, tolerance, test, gtest case, benchmark, and in-tree source.
  • Replaces the stale Octave regeneration guidance and audit language with the licensed run, exact function mapping, measured values, and honest exclusions.

MATLAB remains an offline golden generator; CI consumes only the checked-in C++ literals.

@vjaganat90
vjaganat90 marked this pull request as ready for review September 1, 2026 14:37
@vjaganat90
vjaganat90 force-pushed the 2d_firstorder_matlab_script branch 2 times, most recently from 1c9ab00 to db7f88e Compare September 2, 2026 14:11
@vjaganat90
vjaganat90 force-pushed the 2d_firstorder_matlab_script branch from db7f88e to c9029ad Compare September 2, 2026 15:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant