[martinus]

Closes #322.

The issue's premises, checked first

Two of them do not hold, and the second reverses the story.

"Nobody is told that the correction pass did not happen." A warning has
printed since #223 (2026-08-18), a week before the issue was filed
(2026-08-25).

"widened stays false, which is honest." It does not. confirm set
widened=True whenever nothing was corrected — including when every
confirmation timed out. CONTRIBUTING.md says the flag is true "only when
every survivor in the file really was re-run against the whole suite", and
Report.widened's own docstring says "there is exactly one way to earn it, so
that is what sets it". There were two grant sites and neither checked.

Proven before changing anything — one survivor, one probe, hung on purpose:

confirm said: confirming 1 survivor(s)... | TIMEOUT second hangs -- no answer within 8s
              | 1 confirmation(s) could not be answered; those rows stand as reported.
confirm gave: ['survived']
widened     : True  <- CONTRIBUTING.md:108 says this must be False

So the failure was not quiet-but-honest. It was a documented guarantee claimed
while unmet, written into --json, where tools/reached.py reads it back as a
survivor that had been checked against everything. That is worse than what was
filed, and it is the half the issue could not see.

What changed

1. widened is earned, not assumed. kept = not unsure, applied at both
return paths. One unanswered probe voids the promise however many neighbours
were answered — the promise is about every survivor.

2. The bound is derived. A confirmation probe is one serial pass over the
whole suite; --timeout bounds a mutation against a module or two. One number
cannot be right for both, and 249.7 s of suite under a 300 s bound left every
lane timing out. confirm_timeout derives it from the per-test costs
sweeps/killers.json already persists and refills every run, so it follows
the suite as that grows. --timeout is its floor — someone who raises it means
"give things longer", and a derived bound that came back shorter would answer
the opposite. No remembered costs means the floor, which is today's behaviour: a
guess dressed as a derivation would read as measured in the output and be a
number nobody chose.

The bound is printed (confirming 3 survivor(s) against the whole suite, 300s each...) because a derived number the reader cannot see is one they cannot
check.

Not taken, and why. The issue's option 2 (confirm against a sharded run)
puts a pool inside a lane; run's docstring records that queueing the baseline
alongside the rows was measured at two thirds of the wall clock, so reversing
that is a design call, not a bug fix. Option 3 (confirm against a narrower
module set) is explicitly a weaker claim than CONTRIBUTING.md makes. Both are
beyond one issue.

CONTRIBUTING.md gains the fourth case for widened being false, and the
paragraph on why the bound is derived.

Verification

Hand table — every row a different way to be wrong, run against the whole of
tests.test_mutate:

widened granted regardless     -> FAILED (failures=2)    ['test_a_confirmation_that_timed_out_does_not_earn_widened', 'test_a_correction_beside_a_timeout_still_does_not_earn_widened']
widened on the corrected path  -> FAILED (failures=1)    ['test_a_correction_beside_a_timeout_still_does_not_earn_widened']
no-correction path grants it   -> FAILED (failures=1)    ['test_a_confirmation_that_timed_out_does_not_earn_widened']
confirm reuses --timeout       -> FAILED (failures=1)    ['test_the_derived_bound_is_the_one_the_probe_actually_gets']
slack dropped to one           -> FAILED (failures=2)    ['test_a_measured_suite_raises_the_bound_above_the_floor', 'test_the_derived_bound_is_the_one_the_probe_actually_gets']
floor ignored                  -> FAILED (failures=1)    ['test_the_floor_wins_when_it_is_the_larger']
no costs means unbounded       -> FAILED (failures=1)    ['test_no_measurement_leaves_the_floor_alone']
spec path drops costs          -> FAILED (failures=1)    ['test_the_spec_path_hands_over_what_it_remembers']
generated path drops costs     -> FAILED (failures=1)    ['test_the_generated_path_hands_over_what_it_remembers']

Generated sweep, python -m tools.mutate --base main, 24 of 24 caught:

1 file(s), 75 changed lines -> 24 mutants
  caught    tools/mutate.py:1503 in confirm_timeout() -- the `if` is always taken
  caught    tools/mutate.py:1503 in confirm_timeout() -- the `if` is never taken
  caught    tools/mutate.py:1503 in confirm_timeout() -- the `not` is dropped
  caught    tools/mutate.py:1504 in confirm_timeout() -- returns `None` instead of `floor`
  caught    tools/mutate.py:1505 in confirm_timeout() -- `max` becomes `min`
  caught    tools/mutate.py:1505 in confirm_timeout() -- returns `None` instead of `max(floor, CONFIRM_SLACK * su…`
  caught    tools/mutate.py:1505 in confirm_timeout() -- `*` becomes `//`
  caught    tools/mutate.py:1583 in confirm() -- the call to `print(...)` never happens
  caught    tools/mutate.py:1630 in confirm() -- the `if` is always taken
  caught    tools/mutate.py:1630 in confirm() -- the `if` is never taken
  caught    tools/mutate.py:1631 in confirm() -- the call to `print(...)` never happens
  caught    tools/mutate.py:1641 in confirm() -- the `not` is dropped
  caught    tools/mutate.py:1647 in confirm() -- the `if` is always taken
  caught    tools/mutate.py:1647 in confirm() -- the `if` is never taken
  caught    tools/mutate.py:1648 in confirm() -- returns `None` instead of `report._replace(widened=kept,…`
  caught    tools/mutate.py:1648 in confirm() -- `or` becomes `and`
  caught    tools/mutate.py:1650 in confirm() -- returns `None` instead of `Report([Result(result.mutatio…`
  caught    tools/mutate.py:2086 in _run_spec() -- the `if` is always taken
  caught    tools/mutate.py:2086 in _run_spec() -- the `if` is never taken
  caught    tools/mutate.py:2346 in main() -- the call to `parser.add_argument(...)` never happens
  caught    tools/mutate.py:2422 in main() -- the `if` is always taken
  caught    tools/mutate.py:2422 in main() -- the `if` is never taken
  caught    tools/mutate.py:2452 in main() -- the `if` is always taken
  caught    tools/mutate.py:2452 in main() -- the `if` is never taken

The first sweep had three survivors, and they were real

Worth recording, because "24 of 24" above is the second run:

SURVIVED  tools/mutate.py:1630 in confirm() -- the `if` is always taken
SURVIVED  tools/mutate.py:1647 in confirm() -- the `if` is never taken
SURVIVED  tools/mutate.py:1648 in confirm() -- `or` becomes `and`

All three were output-honesty gaps on lines this diff touches. Making if unsure: unconditional prints "0 confirmation(s) could not be answered ... the
report is not marked widened" beside a report that is marked widened.
Making if not corrected: unreachable prints "0 of them were caught by a test
the selection had not run" after correcting none. And merged or None →
merged and None silently discards the timings Killers budgets the
cheap-first prefix from.

Two false sentences, in the pass whose entire job is to stop a reader trusting a
wrong row. They were killed by asserting what confirm must not say — the
existing suite asserted only what it does say.

Preflight green: ruff check, ruff format --check, mypy, shellcheck, and
CI=true WOSWOAR_HYPOTHESIS_PROFILE=ci python -m tools.run_tests —
OK (0 failures, 0 errors, 4 skipped).

One note on the sweep's own output

This sweep mutates tools/mutate.py, so it ran the changed confirm against
itself: confirming 3 survivor(s) against the whole suite, 300s each... is the
new line, showing the floor because sweeps/killers.json was empty on this
machine. That is the intended fallback, observed rather than argued.

🤖 Generated with Claude Code

[martinus]

CI caught a brittle assertion of mine on test (3.10), now fixed in 88f672d.

FAIL: test_a_measured_suite_raises_the_bound_above_the_floor
AssertionError: 749.100000000002 != 749.0999999999999

3.12 and 3.14 passed and 3.10 did not, which is the interesting part: CPython
3.12 gave sum() compensated (Neumaier) summation for floats. So a fixture of
2.497 × 100 lands on exactly 249.7 there and a few ulps away under 3.10's
naive accumulation, and the assertion compared that against the literal
3.0 * 249.7.

The fixture now uses 2.5, which sums exactly under either algorithm. The story
is unchanged — a ~250 s suite under a 300 s bound — and the test is about the
arithmetic it claims to be about rather than about which summation the
interpreter ships.

Re-verified that it still earns its place:

slack dropped to one   -> FAILED  ['test_a_measured_suite_raises_the_bound_above_the_floor']
slack raised to four   -> FAILED  ['test_a_measured_suite_raises_the_bound_above_the_floor']
floor ignored          -> FAILED  ['test_the_floor_wins_when_it_is_the_larger']

Worth noting against the PR body's claim that the generated sweep came back 24
of 24: that sweep ran on 3.14, where the old fixture passed. It could not have
found this — a mutation table asks whether a test notices the code changing, not
whether it notices the interpreter changing.

---

Generated by Claude Code