Skip to content

feat(bench): the report answers the question it was built to ask - #935

Draft
wenzowski wants to merge 1 commit into
mainfrom
claude/cloud-1606-refusal-render-perf-k7nr2y
Draft

feat(bench): the report answers the question it was built to ask#935
wenzowski wants to merge 1 commit into
mainfrom
claude/cloud-1606-refusal-render-perf-k7nr2y

Conversation

@wenzowski

Copy link
Copy Markdown
Contributor

Closes CLOUD-1606

CLOUD-1606's last acceptance clause asks the final result to record whether the first-sighting/repeat behaviour should be retained, and to name any successor needed for real harness epoch/residency management. PR #904 landed the benchmark with the numbers and never answered it. This is that answer.

RETAIN, and derived rather than asserted: no measured strategy emits less on a repeat than the shipped one. FirstFullThenCompact is byte-identical to Current on all twelve arms, and FullEveryTime — the control — is the one that costs more. refusal_render_verdict computes that from the records and would print REPLACE, naming the class, if a strategy ever beat it.

That derivation is not a flourish. Two paragraphs of this report were hand-written claims about the measurement and both went false the moment CLOUD-1637 changed the renderer — the ceiling note and the zero-margin note. A retain/replace verdict sitting beside a generated table is the same pairing, so it reads off the same records and is covered by the same drift check.

The successor half is stated as an obligation: the saving is per repeat, Cold/Warm are inputs, and nothing here detects residency — so a successor must own an explicit epoch authority (DecisionRecord.config_epoch is a configuration hash, not a session one) and inherits CLOUD-1386's split, where the ceiling binds the repeat and the headroom is on the sighting side alone.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Ajfos2de2W3zSRywHSvn27


Generated by Claude Code

CLOUD-1606's last acceptance clause asks the final result to record whether the
first-sighting/repeat behaviour should be RETAINED, and to name any successor
needed for real harness epoch/residency management. The report carried the
numbers and never answered it, and the row went to In Review anyway — my own
work spared from a clause I would have held anyone else to.

The answer is RETAIN, and it is derived rather than asserted: no measured
strategy emits less on a repeat than the shipped one. `FirstFullThenCompact` is
byte-identical to `Current` on all twelve arms and `FullEveryTime` is the
control that costs more, so the split is already the cheapest of the three on
the firing that recurs. `refusal_render_verdict` computes that from the records
and would print REPLACE, naming the class, if some strategy ever beat it.

DERIVED IS NOT A FLOURISH HERE. Two paragraphs of this report were hand-written
claims about the measurement and both went false the moment CLOUD-1637 changed
the renderer — the ceiling note and the zero-margin note. A retain/replace
verdict beside a generated table is the same pairing, so it reads off the same
records.

The successor half is stated as an obligation rather than a wish: the saving is
per REPEAT, `Cold`/`Warm` are inputs, and nothing here detects residency — so a
successor must own an explicit epoch authority (`DecisionRecord.config_epoch` is
a configuration hash, not a session one) and inherits CLOUD-1386's split, where
the ceiling binds the repeat and the headroom is on the sighting side alone.

Refs: CLOUD-1606, CLOUD-1386, CLOUD-417, CLOUD-1117
@coderabbitai

coderabbitai Bot commented Sep 9, 2026

Copy link
Copy Markdown

Warning

Review limit reached

Next included review available in 26 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Advanced

Run ID: c3a92f20-befa-4c19-99c2-6320afb41728

📥 Commits

Reviewing files that changed from the base of the PR and between de759a4 and b4d39e0.

📒 Files selected for processing (2)
  • bench/refusal-render/RESULTS.md
  • crates/batten/src/perf.rs

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant