feat(sbd-v6): declare the axis a session graded on, and record it where it is resolved

[haishuok0525]

What

Makes the grading axis of a session visible in session_breakdown.json, and records it where the run resolves it rather than re-deriving it at export.

Supersedes #1380, which was written against the pre-#1455 collector export and against the pre-#1444 objective. Both premises have moved; see Why this is not #1380 rebased below.

Why

Nothing in the document said which axis its gains were on. An AgentX replay is ranked on the slow-tail interactivity percentile (e2e_norm_intvty_p90) with total throughput held as a guard; a synthetic run is ranked on output throughput alone. Every throughput field in the breakdown is the output axis by construction, benchmark_mode reaches reports/final.json but never this document, and on the canonical corpus the two axes differ by roughly two orders of magnitude.

A consumer therefore had no way to keep an AgentX session out of a synthetic leaderboard: it would sort one against the other and every number would look plausible.

The two blocks, and why they are two

"metadata": {
  "grading": {
    "benchmark_mode": "agentx",
    "objective": "e2e_norm_intvty_p90",
    "tput_guard": {"enabled": true, "noise_pct": 3.5}
  }
}

metadata.grading is what the session was configured to rank on. outcome.validation.graded_on is what the run actually decided its last promotion on, read off that promotion. The two differ whenever a comparison could not supply the axis pair and fell back to output.

Neither resolves the other, deliberately. Publishing the configured axis in outcome would put an interactivity label on an output number; resolving metadata from outcome would erase what the session asked for. There is a test on each direction.

outcome.final.graded_on agrees with outcome.validation.graded_on by construction -- they are the same figure read twice -- and a test locks it.

Recorded, not re-derived

Resolution reads HYPERLOOM_PERF_METRIC / HYPERLOOM_PERF_NOISE_PCT. Every later reader of those is somewhere the environment is not evidence: a resume is a new process, a re-baseline runs in a subprocess, and CLOSE drives this export from one that frequently never inherited them. So the axis and its band are resolved once at seed and recorded in SharedState.grading, next to benchmark_mode and agentx_epoch, which live there for exactly the same reason.

This fixes a live defect rather than only serving the export. resolve_graded_comparison was re-deriving on every comparison, so a shell that lost the metric var would flip a session back to the output axis mid-run, and a lost noise var would silently widen a 3.5% guard band back to the 5% default. The KEEP/REVERT rule has to be the one the session started with. Both now come from one resolution point, resolved_grading, and the recorded band reaches passes_intvty_gate and passes_tput_guard, which were taking the ambient default despite both already accepting the parameter.

Sessions seeded before the field derive as before and report a null band: the band they actually applied was never recorded, and today's default is not evidence of it.

Recording it also removes the environment from the export path entirely. The metadata collector projects nothing for this block, so it lands from the spool alone through the existing leaf overlay -- collectors/v6.py's metadata path is untouched.

The axes themselves

outcome.baseline.perf, outcome.final.perf and outcome.validation.perf carry the four AgentX axes. They are recorded where the measurement is taken: the baseline round's own measurement block, which is already where outcome.baseline comes from, and the stack validation row that produced the gain.

Not read off state.baseline_perf and current_best at export, for two separate reasons. A second source for one baseline is a second answer to the same question. And a revalidation moves the cumulative gain without re-promoting the recipe, so current_best can be a later measurement than the gain published beside it.

An unmeasured axis is an explicit null. Absent would be indistinguishable from an axis the framework failed to report, and zero reads as "measured, and it was zero", so a synthetic run publishes four nulls. TPOT p90 was already parsed out of the benchmark report and dropped at five call sites; it now lands.

Degrade is a finding, not a field

An adoption that fell off the configured axis is reported through validation.notes, naming the count and the reason. Its contribution sits in the same sum as the axis-graded ones, so the total is not single-axis and the reader has to be told.

The reason is recorded on the adoption because that is the only comparison that can degrade. The whole-stack validation is recorded only after the orchestrator found the comparison comparable, so a degrade_reason on that row would never be non-empty -- a field that reads as protection the document does not have.

Why this is not #1380 rebased

#1380 is stale in two independent ways.

Semantics. It declares objective: "total_throughput" under an intvty_veto, from #1362. #1444 inverted that: interactivity became the objective and throughput the guard, and total_tput_serving_grading_enabled no longer exists. Carrying its block over would have made the document state an axis no session grades on -- precisely the failure it was written to prevent.

Architecture. Its export side patched collectors/optimizations.py, deleted by #1455. outcome.validation now reads the stack ledger timeline event, whose rows the orchestrator records as each adoption lands, so graded_objective was already recorded and simply unpublished. Its 113-line _grading_projection is not needed at all: the author-time snapshot reaches metadata.grading through the overlay with no collector change and no os.environ fallback, which is the weakness #1380's own description acknowledged.

Rollout

Additive to the wire shape. throughput_tok_s_per_gpu, output_throughput and every other existing key keep their output-axis meaning, so nothing downstream has to change to keep working.

Cortex has no column for the interactivity or total axes yet, so the immediate value is that an AgentX session becomes identifiable -- the prerequisite for keeping it off a leaderboard it is not comparable on.

Test

25 new cases in test_sbd_v6_grading.py: both axes at seed, the scriptable-framework carve-out, recorded-beats-environment in both directions, the null band on a legacy session, the block surviving the recorder and the overlay end to end, the metadata/outcome separation locks in both directions, the axis fallback for a session that adopted but never validated, explicit nulls, and the degrade finding.

Two exact-equality assertions on outcome.baseline in test_sbd_v6_stages.py updated for the new key.

3200+ tests pass across the SBD, grading, promotion, writeback, bootstrap and executor suites. ruff check and ruff format clean on src/hyperloom.

docs/reference/session-breakdown.md and breakdown/SKILL.md updated for the new keys.

[haishuok0525]

Closing per request. Branch feat/sbd-v6-agentx-grading-axis retains all commits (SBD grading + conc_sweep alignment).

[luochen-amd]

Reopened as #1520, rebased onto main at ff7c235e4. One conflict, an import line in conc_sweep.py, resolved by keeping both sides; the three commits are otherwise unchanged. I re-verified the three defects against current main first -- all three are still live, and the PR body records which ones and where.