fix: finish the band-loss and Table 1b propagation Codex found half-done

Codex re-reviewed the repaired manuscript against the repository and found the
same failure mode again: repairs that reached one surface and not the others.

The band-loss correction is the clearest case. I took the TRASH figures from the
artefact in Limitations this afternoon and left Table 3B-b's caption and Section
3.3 subtracting the rounded cells, so the same two numbers appeared as 0.34/18.57
in one place and 0.35/18.56 in two others. All three now come from
table3bc_replacement.md, and the caption says where they come from.

The root README still gave 79.88% for the shared range after C-2 moved Table 1b
to the filtered arm — the same gap the Abstract had.

One finding is mine and is an overstatement I introduced this afternoon. The new
Limitations item said the flake 'does not reach chromosome-scale output' because
'repeated whole-chromosome runs under different environments are byte-identical'.
The evidence is one chromosome, an 18.8 Mb Arabidopsis Chr4, under three
environment sizes. It now says that, and says it is weaker than a guarantee.

One of Codex's findings does not hold: it reports that the CEN180 paragraph still
calls the raw hit set unretained. The only 'not retained' left there is about the
blastn version and command line, which remains true and is the reason the inputs
and outputs were deposited instead of the invocation.

Transcript deposited with that correction noted in its header.

Author: wyim-pgl

README.md

@@ -174,7 +174,7 @@ Col-CEN, maize Mo17; see the manuscript for every caveat):
 |---|---|
 | One wide-range pass | periods 1–2,000 bp in 12.6 h / 25.3 core-hours on GRCh38 (ULTRA: 29.8 h / 59.6 core-hours capped at 100 bp; not range-matched) |
 | Near-flat range cost | widening the reportable maximum 100→2,000 bp costs 1.30–1.41× in paired runs; we did not obtain completed 2,000 bp competitor runs on human (we stopped the TRF and ULTRA attempts terminated incomplete at 6.6 d and 1 d 22 h) |
-| Shared-range accuracy | non-leading: ULTRA ranks first in region recall (81.62%); BWTandem 79.88% at the whole-genome configuration, 81.60% at a permissive setting with lower precision |
+| Shared-range accuracy | non-leading: ULTRA ranks first in region recall (81.62%); BWTandem 78.87% at the whole-genome configuration, restricted the same way its competitors are, 81.60% at a permissive setting with lower precision |
 | Long-period stratum | 3.43% of catalog regions above period 100 (ULTRA: none by construction) |
 | Plant satellites | Col-CEN CEN180 monomer recall 99.72% in 40 min; maize unfiltered coverage in the leading group |
 | Specificity audit | 4 of 400 blinded BWTandem-only calls supported (single reader) — unmatched calls are predominantly over-calls |

docs/2026-09-03-codex-recheck.md

@@ -0,0 +1,58 @@
+# Codex re-review of the repaired manuscript
+
+Run 2026-09-03 against a detached worktree at bb64ad5 — before the Kimi-driven fixes in
+6707dc5 — with the instruction to check the repairs against the repository rather than
+to re-find the original defects. Verdict: major revision.
+
+Some findings were already closed by 6707dc5 when this landed, and one is wrong: its
+finding 5 says the paragraph still calls the raw BLAST hit set unretained, but the only
+"not retained" left there is about the blastn version and command line, which is true.
+
+---
+
+Recommendation: major revision. Most repaired table cells are now numerically correct, but several repairs were not propagated through the manuscript, provenance manifest, documentation, or figures.
+
+## Findings
+
+1. **Table 1b now has two incompatible identities.** The repaired row itself is correct: `3,911,182 / 78.87 / 50.13 / 30.00 / 53.89` in [manuscript.md:134](/data/gpfs/assoc/pgl/tmp/claude-3092957/-data-gpfs-assoc-pgl-devel-bwt-algorithm/ab324600-278d-496b-944b-b0f7ec02f088/scratchpad/review-wt/manuscript.md:134) exactly matches the post-hoc-filtered full-range result in [score_table1_regen_v2.txt:34](/data/gpfs/assoc/pgl/tmp/claude-3092957/-data-gpfs-assoc-pgl-devel-bwt-algorithm/ab324600-278d-496b-944b-b0f7ec02f088/scratchpad/review-wt/results/regen/score_table1_regen_v2.txt:34). Elsewhere, however, the Abstract still gives 79.88%, Results calls 32.55% and 79.88% the Table 1b values, Methods says the Table 1b run used four threads, and S2 assigns Table 1b to the native `MAXP=100` configuration ([manuscript.md:21, 80, 113, 473](/data/gpfs/assoc/pgl/tmp/claude-3092957/-data-gpfs-assoc-pgl-devel-bwt-algorithm/ab324600-278d-496b-944b-b0f7ec02f088/scratchpad/review-wt/manuscript.md:21)). [README.md:177](/data/gpfs/assoc/pgl/tmp/claude-3092957/-data-gpfs-assoc-pgl-devel-bwt-algorithm/ab324600-278d-496b-944b-b0f7ec02f088/scratchpad/review-wt/README.md:177) also retains 79.88%. The manifest contains a stale pending Table 1b row and a completed native-period-100 row, but no Table 1b row for the current post-hoc result ([manifest.tsv:8](/data/gpfs/assoc/pgl/tmp/claude-3092957/-data-gpfs-assoc-pgl-devel-bwt-algorithm/ab324600-278d-496b-944b-b0f7ec02f088/scratchpad/review-wt/results/manifest.tsv:8), [manifest.tsv:89](/data/gpfs/assoc/pgl/tmp/claude-3092957/-data-gpfs-assoc-pgl-devel-bwt-algorithm/ab324600-278d-496b-944b-b0f7ec02f088/scratchpad/review-wt/results/manifest.tsv:89)). Minimal fix: consistently define Table 1b as the two-thread full-range run filtered to ≤100 bp, reserve the four-thread 79.88/50.62/32.55/55.51 result for Table 1d and the sensitivity analysis, and add the actual Table 1b scoring row to the manifest.
+
+2. **Supplementary Methods gives the wrong Tier-3 jitter tolerance for every reported run.** S1.3 says the tolerance clamps to 0.04 because the internal maximum is 100 kb ([manuscript.md:440](/data/gpfs/assoc/pgl/tmp/claude-3092957/-data-gpfs-assoc-pgl-devel-bwt-algorithm/ab324600-278d-496b-944b-b0f7ec02f088/scratchpad/review-wt/manuscript.md:440)). At the benchmark commit, Tier 3 is instead bounded by the user’s requested maximum, and the code explicitly documents 0.0204 at `--max-period 2000` ([finder.py:116](/data/gpfs/assoc/pgl/tmp/claude-3092957/-data-gpfs-assoc-pgl-devel-bwt-algorithm/ab324600-278d-496b-944b-b0f7ec02f088/scratchpad/review-wt/bwtandem/finder.py:116)); the formula is `0.02 + 0.02 × max_period/100000` ([tier3.py:38](/data/gpfs/assoc/pgl/tmp/claude-3092957/-data-gpfs-assoc-pgl-devel-bwt-algorithm/ab324600-278d-496b-944b-b0f7ec02f088/scratchpad/review-wt/bwtandem/tier3.py:38)). Thus the period-100 runs used approximately 0.02002, not 0.04. Minimal fix: replace the sentence with the formula and realized values for the 100- and 2,000-bp runs.
+
+3. **The repaired ambiguity threshold is attached to an incorrect autocorrelation equation.** The paper defines raw equality divided by \(w-p\), which would count `N == N` ([manuscript.md:445](/data/gpfs/assoc/pgl/tmp/claude-3092957/-data-gpfs-assoc-pgl-devel-bwt-algorithm/ab324600-278d-496b-944b-b0f7ec02f088/scratchpad/review-wt/manuscript.md:445)). The scalar implementation excludes ambiguous comparisons and divides by the number of valid comparisons ([autocorr.py:60](/data/gpfs/assoc/pgl/tmp/claude-3092957/-data-gpfs-assoc-pgl-devel-bwt-algorithm/ab324600-278d-496b-944b-b0f7ec02f088/scratchpad/review-wt/bwtandem/autocorr.py:60)); the sliding gap-fill and catch-all paths exclude ambiguous matches but divide support by the full window ([finder.py:614](/data/gpfs/assoc/pgl/tmp/claude-3092957/-data-gpfs-assoc-pgl-devel-bwt-algorithm/ab324600-278d-496b-944b-b0f7ec02f088/scratchpad/review-wt/bwtandem/finder.py:614), [finder.py:703](/data/gpfs/assoc/pgl/tmp/claude-3092957/-data-gpfs-assoc-pgl-devel-bwt-algorithm/ab324600-278d-496b-944b-b0f7ec02f088/scratchpad/review-wt/bwtandem/finder.py:703)). Minimal fix: give separate scalar and sliding-window definitions, including their denominators and the 80% valid-comparison gate.
+
+4. **The band-loss repair remains self-contradictory and repeats the rounding error it was intended to eliminate.** The authoritative artefact gives TRASH de novo losses of 0.34 and 18.57 points, and template losses of 0.32 and 18.42 ([table3bc_replacement.md:24](/data/gpfs/assoc/pgl/tmp/claude-3092957/-data-gpfs-assoc-pgl-devel-bwt-algorithm/ab324600-278d-496b-944b-b0f7ec02f088/scratchpad/review-wt/results/regen/table3bc_replacement.md:24)). Table 3B-b’s caption first gives the template pair, then gives 0.35 and 18.56 for the de novo pair by subtracting displayed rounded coverages ([manuscript.md:256](/data/gpfs/assoc/pgl/tmp/claude-3092957/-data-gpfs-assoc-pgl-devel-bwt-algorithm/ab324600-278d-496b-944b-b0f7ec02f088/scratchpad/review-wt/manuscript.md:256)). Results also still says “0.35 or less” and “18.6” ([manuscript.md:282](/data/gpfs/assoc/pgl/tmp/claude-3092957/-data-gpfs-assoc-pgl-devel-bwt-algorithm/ab324600-278d-496b-944b-b0f7ec02f088/scratchpad/review-wt/manuscript.md:282)), while Limitations correctly uses 0.34 and 18.57. Minimal fix: name the TRASH mode every time and copy the exact artefact losses rather than recomputing from rounded cells.
+
+5. **The CEN180 repair contradicts itself and overstates an unmeasured direction.** Methods now correctly names and deposits the 68,840-hit raw BED and the 66,683-hit filtered subset, but the same paragraph later says that the raw set “was not retained” ([manuscript.md:90](/data/gpfs/assoc/pgl/tmp/claude-3092957/-data-gpfs-assoc-pgl-devel-bwt-algorithm/ab324600-278d-496b-944b-b0f7ec02f088/scratchpad/review-wt/manuscript.md:90)). I verified that the files contain 68,840 and 66,683 unique coordinates and that all filtered coordinates occur in the raw set. The claim that recall over excluded monomers “would therefore be lower” does not follow from the within-filter gradient; the raw coordinates can now be scored, although their identities and the original BLAST invocation remain unavailable. Minimal fix: delete the “not retained” statement and either score the raw coordinate set or say that below-80%-identity recall is unknown.
+
+6. **The S4 number is repaired, but its provenance and explanatory README are stale.** The regenerated JSON supports 518,719 scored pairs and 63.53% period exactness ([one_to_one_trf_annot_r50.json:43](/data/gpfs/assoc/pgl/tmp/claude-3092957/-data-gpfs-assoc-pgl-devel-bwt-algorithm/ab324600-278d-496b-944b-b0f7ec02f088/scratchpad/review-wt/results/one_to_one/one_to_one_trf_annot_r50.json:43)). Yet [results/one_to_one/README.md:117](/data/gpfs/assoc/pgl/tmp/claude-3092957/-data-gpfs-assoc-pgl-devel-bwt-algorithm/ab324600-278d-496b-944b-b0f7ec02f088/scratchpad/review-wt/results/one_to_one/README.md:117) still shows a dash, its ordering omits TRF, and it incorrectly labels BWTandem’s 8.68% as the lowest when tantan is 3.42%. Manifest row 114 still attributes the regenerated JSON to the old scorer hash/job, while the README says the manifest keeps both versions. Minimal fix: refresh the README and add a new provenance row for the regenerated TRF annotation result, preserving the historical row as superseded.
+
+7. **The specificity audit is not regenerable as claimed.** Data Availability says the missing dot plots regenerate deterministically from the sampler and sheet ([manuscript.md:336](/data/gpfs/assoc/pgl/tmp/claude-3092957/-data-gpfs-assoc-pgl-devel-bwt-algorithm/ab324600-278d-496b-944b-b0f7ec02f088/scratchpad/review-wt/manuscript.md:336)), while [results/audit11/README.md:20](/data/gpfs/assoc/pgl/tmp/claude-3092957/-data-gpfs-assoc-pgl-devel-bwt-algorithm/ab324600-278d-496b-944b-b0f7ec02f088/scratchpad/review-wt/results/audit11/README.md:20) admits that all 400 renderings are absent. The deposited sampler only selects records and writes TSV files; it contains no dot-plot, unit-shift, or random-reference renderer ([sample_specificity_audit.py:75](/data/gpfs/assoc/pgl/tmp/claude-3092957/-data-gpfs-assoc-pgl-devel-bwt-algorithm/ab324600-278d-496b-944b-b0f7ec02f088/scratchpad/review-wt/scripts/scoring/sample_specificity_audit.py:75)). Minimal fix: deposit the 32 MB render set or the exact renderer, parameters and random seed.
+
+8. **The new implementation limitation converts one chromosome test into an unsupported universal negative.** The underlying documentation reports one 18.8 Mb Chr4 tested under three environment sizes ([CLAUDE.md:77](/data/gpfs/assoc/pgl/tmp/claude-3092957/-data-gpfs-assoc-pgl-devel-bwt-algorithm/ab324600-278d-496b-944b-b0f7ec02f088/scratchpad/review-wt/CLAUDE.md:77)). That does not establish that the unresolved memory-layout-dependent failure “does not reach chromosome-scale output” or that no figure can depend on it ([manuscript.md:326](/data/gpfs/assoc/pgl/tmp/claude-3092957/-data-gpfs-assoc-pgl-devel-bwt-algorithm/ab324600-278d-496b-944b-b0f7ec02f088/scratchpad/review-wt/manuscript.md:326)). The reported ~1,640 runs are traceable only to this narrative summary, not raw logs. Minimal fix: state that no effect was observed on the one chromosome and layouts tested.
+
+9. **Figure 1 mixes shared-range and full-range data.** Panels A/B use native-period-100 BWTandem points but full-range values for the 2026 tools; AniAnn’s is plotted at 0.10/83.79 even though its repaired matched-range row is 0.01/85.09 ([fig1ab_pr_points.csv:2](/data/gpfs/assoc/pgl/tmp/claude-3092957/-data-gpfs-assoc-pgl-devel-bwt-algorithm/ab324600-278d-496b-944b-b0f7ec02f088/scratchpad/review-wt/results/figures/paper_figs/data/fig1ab_pr_points.csv:2)). The overall title nevertheless says “Shared-range accuracy,” and the caption excludes AniAnn’s from panel C because the 2026 tools allegedly lack period bounds, directly contradicting Table 1b and its period column ([plot_fig1_accuracy_tradeoff.py:142](/data/gpfs/assoc/pgl/tmp/claude-3092957/-data-gpfs-assoc-pgl-devel-bwt-algorithm/ab324600-278d-496b-944b-b0f7ec02f088/scratchpad/review-wt/results/figures/paper_figs/plot_fig1_accuracy_tradeoff.py:142)). Minimal fix: restrict all shared-range panels consistently, or clearly separate full-range 2026 points; include AniAnn’s in C or state the real reason for exclusion.
+
+10. **Figure 2’s corrected 3.92% caption applies the number to tools for which it is false.** The exact 3.92% difference applies to the historical GCA FASTA versus the primary chromosome FASTA. longdust and AniAnn’s in panel C used the same primary FASTA as BWTandem, but the caption says generically that “the competitor FASTA” is 3.92% larger ([plot_fig2_range_cost.py:77](/data/gpfs/assoc/pgl/tmp/claude-3092957/-data-gpfs-assoc-pgl-devel-bwt-algorithm/ab324600-278d-496b-944b-b0f7ec02f088/scratchpad/review-wt/results/figures/paper_figs/plot_fig2_range_cost.py:77)). Methods 2.2.4 also still says “~5% larger” ([manuscript.md:99](/data/gpfs/assoc/pgl/tmp/claude-3092957/-data-gpfs-assoc-pgl-devel-bwt-algorithm/ab324600-278d-496b-944b-b0f7ec02f088/scratchpad/review-wt/manuscript.md:99)). Minimal fix: specify that only the historical ULTRA/TRF inputs were 3.92% larger and replace the surviving ~5%. Per instruction, I did not re-report the known superseded Figure 2A data.
+
+11. **Figure 4’s rendered panel and caption describe different quantities.** Panel A plots only `cen_coverage_pct`; its code never plots `cen180_monomer_recall_pct` ([plot_fig4_plant_satellites.py:19](/data/gpfs/assoc/pgl/tmp/claude-3092957/-data-gpfs-assoc-pgl-devel-bwt-algorithm/ab324600-278d-496b-944b-b0f7ec02f088/scratchpad/review-wt/results/figures/paper_figs/plot_fig4_plant_satellites.py:19)). The caption says it shows both coverage and recall. Table 2 has eleven rows and the plot has six; the caption names four omitted rows but silently also omits the second longdust configuration ([plot_fig4_plant_satellites.py:97](/data/gpfs/assoc/pgl/tmp/claude-3092957/-data-gpfs-assoc-pgl-devel-bwt-algorithm/ab324600-278d-496b-944b-b0f7ec02f088/scratchpad/review-wt/results/figures/paper_figs/plot_fig4_plant_satellites.py:97)). Minimal fix: either plot recall or remove it from the caption, and disclose all five omitted rows.
+
+12. **Figure 5 treats selected examples as a species-level conclusion and mislabels cropped subwindows as calls.** The human BWTandem call is chr1:122,257,803–122,338,029, but the figure uses only the first 60 kb; the Arabidopsis call is Chr4:4,985,644–5,079,605 and is likewise cropped. Only the maize interval is an entire call. The caption says the intervals are “taken from BWTandem’s own calls,” and the headline generalizes from one hand-selected locus per species ([plot_fig5_array_structure.py:34](/data/gpfs/assoc/pgl/tmp/claude-3092957/-data-gpfs-assoc-pgl-devel-bwt-algorithm/ab324600-278d-496b-944b-b0f7ec02f088/scratchpad/review-wt/results/figures/paper_figs/plot_fig5_array_structure.py:34), [plot_fig5_array_structure.py:201](/data/gpfs/assoc/pgl/tmp/claude-3092957/-data-gpfs-assoc-pgl-devel-bwt-algorithm/ab324600-278d-496b-944b-b0f7ec02f088/scratchpad/review-wt/results/figures/paper_figs/plot_fig5_array_structure.py:201)). Minimal fix: describe these as selected cropped subwindows, disclose the selection rule, and frame the result as three examples rather than a species comparison.
+
+13. **Unit and scope repairs remain stale in required repository surfaces.** [README.md:181](/data/gpfs/assoc/pgl/tmp/claude-3092957/-data-gpfs-assoc-pgl-devel-bwt-algorithm/ab324600-278d-496b-944b-b0f7ec02f088/scratchpad/review-wt/README.md:181) still calls binary-converted memory values “GB”; `results/comparator_baselines.md` does likewise, and the manifest column remains `peak_rss_gb_sacct`. Supplementary Table S1 also says “except the two noted below,” although the two 2026 tools plus the ULTRA attempt were outside the container and AniAnn’s row does not carry that note. Minimal fix: use GiB consistently, rename or document the manifest column, and make the Singularity exceptions explicit in S1.
+
+## Repaired-number checks that came back clean
+
+- Table 1b’s new row and both new AniAnn’s band rows exactly match their scoring outputs.
+- Both Table 2 TRASH rows are correct: de novo `234 / 74 / 5.79 h / 1.29 GiB`; template-only `397 / 115 / 31.69 h / 2.63 GiB`. The template union and cost arithmetic are documented in [colcen_trash_template_scored.txt:8](/data/gpfs/assoc/pgl/tmp/claude-3092957/-data-gpfs-assoc-pgl-devel-bwt-algorithm/ab324600-278d-496b-944b-b0f7ec02f088/scratchpad/review-wt/results/regen/colcen_trash_template_scored.txt:8).
+- The changed 0.06 GiB longdust and 0.48 GiB AniAnn’s cells, and all other changed GiB conversions, agree with the deposited GNU-time logs.
+- The CEN180 strata recompute exactly to 87.32, 99.08, 99.89, 99.91 and 99.00%; their counts also agree.
+- The exact Table 3 band losses—BWTandem 15.59/33.01/17.84, TRF maximum 1.54, ULTRA maximum 2.82, and tantan 0.51/0.91—match the deposited rescoring artefact.
+- TRF’s repaired S4 period-exact value is exactly 63.53%.
+- The 80% valid-base floor is implemented in both supplementary passes. The added organellar values `k=12/13`, stride 20 and maximum occurrences 200 also follow the code.
+- The FASTA totals independently give 3.9186% additional total sequence, rounding to 3.92%, and 3.8016% additional unambiguous sequence, rounding to 3.80%.
+- The tuning ledger has 44 data records; the Python 3.11.15/numpy 2.4.6 claims match the per-run provenance.
+- The Col-CEN catch-all figures 65.54/60.72 and 99.67/99.68 are traceable to `/data/gpfs/assoc/pgl/devel/exp1_human/filip_repro/catchall_experiment_results.md`, but not to a deposited result or manifest row.
+- The revised Singularity scope, ULTRA last-emitted-record wording, Discussion 4.2 attribution matrix, and Discussion 4.3 mechanism description are supported and appropriately qualified.
+- Figures 3, S1 and S2, Figure 4C’s corrected deltas, and Figure 1C’s numerical values agree with their CSV inputs.
+
+All `results/manifest.sha256` entries passed, HEAD remained `bb64ad5`, and the worktree remained clean.