From 19bd222639ea4c7c66ac4567d8b11e97d112c037 Mon Sep 17 00:00:00 2001 From: Alec Wenzowski Date: Sun, 30 Aug 2026 18:27:30 +0000 Subject: [PATCH 1/4] fix(commit-lint): exempt a bot-authored manifest bump from the claim half A Renovate bump carries no `Refs:` trailer and structurally cannot: the row is minted from the pull request, after the commit exists, and the lane never checks the bot's head out under a write token to amend one in. The merge still moves the row, through `Closes CLOUD-` in the PR body. So the gate refused a class of commit it was never aimed at, forever. Measured on #676: seven days Ready and red, 372 workflow runs on one branch, one full matrix per `main` advance, none of which could ever land. Two conjuncts rather than the diff alone, because a workflow-only diff is ordinary human work here: the author address must be one GitHub mints for an App, and every touched path must be a manifest `renovate.json5` can move. Refs: CLOUD-1207 --- mise.toml | 56 +++++++++++++++++++++++++++++++++++++++++++++++-------- 1 file changed, 48 insertions(+), 8 deletions(-) diff --git a/mise.toml b/mise.toml index ae63bf47d..a2b9c74eb 100644 --- a/mise.toml +++ b/mise.toml @@ -2342,14 +2342,42 @@ shell = "bash -c" # uses ${sha # than mentioning, which is the property wanted: a commit citing an issue as # evidence has not thereby become work on it. # -# ONE EXEMPTION, and it is a property of the DIFF rather than of the subject -# line. release-plz authors `chore: release vX.Y.Z` touching exactly the version -# and changelog files, and it has no issue to name because it is not work on one; -# measured on this repo's history, every release commit is keyless and every -# other commit in the last 200 revs carries a key. Keying the exemption to the -# file set rather than the wording means it cannot be claimed by a hand-written -# commit that merely borrows the subject, and it does not encode a message format -# release-plz is free to change. +# TWO EXEMPTIONS, and both are properties of the COMMIT rather than of the +# subject line. +# +# The first: release-plz authors `chore: release vX.Y.Z` touching exactly the +# version and changelog files, and it has no issue to name because it is not work +# on one; measured on this repo's history, every release commit is keyless and +# every other commit in the last 200 revs carries a key. Keying the exemption to +# the file set rather than the wording means it cannot be claimed by a +# hand-written commit that merely borrows the subject, and it does not encode a +# message format release-plz is free to change. +# +# THE SECOND IS THE UPDATE BOT'S, AND IT CLOSES A LOOP THAT COULD NOT TERMINATE +# (CLOUD-1207). A Renovate bump carries no `Refs:` trailer and structurally +# cannot: CLOUD-693's `bot-issue ensure` mints the row FROM the pull request, so +# the key does not exist when the commit is authored, and the lane deliberately +# never checks the bot's head out under a write token to amend one in. The row is +# real and the merge still moves it — through `Closes CLOUD-` in the PR BODY, +# which this gate does not read and should not, since a body is rewritten on +# every rebase (CLOUD-768) and a commit message is not. +# +# So the gate refused, forever, a class of commit it was never aimed at. Measured +# on #676: seven days Ready and red, one full matrix per `main` advance, 372 +# workflow runs, none of which could ever land. +# +# TWO CONJUNCTS, because the diff alone is too wide here where it was sufficient +# for release-plz. A workflow-only diff is ordinary human work in this repository, +# so exempting it by paths would drop the server-side claim (CLOUD-431) for a +# whole class of real changes. The author test is what narrows it: an address of +# the form `…[bot]@users.noreply.github.com` is minted by GitHub for an App, and +# is not a login allowlist — `mise-tasks/bot-issue.sh`'s `BOT_LOGINS_RE` answers a +# different question over a different object (which bot's pull request this lane +# will adopt), so this is not a second spelling of it. A contributor forging that +# address is refused one gate over, by `commit-attribution`'s identity table. +# +# The paths are the manifests `renovate.json5`'s `enabledManagers` can move, and +# no others: a bot commit reaching anything else is not a bump and is not exempt. run = ''' fail=0 keys="$(git rev-parse --show-toplevel)/mise-tasks/claimed-keys.sh" @@ -2364,6 +2392,18 @@ for sha in $(git rev-list --no-merges "${BASE_SHA:?}..${HEAD_SHA:?}"); do echo "REL ${sha:0:8} version/changelog only — no issue to name" continue fi + # The bot exemption's two conjuncts, in the order that costs least: the author + # is one string, the diff is a walk. `%ae` is the AUTHOR's address rather than + # the committer's, because GitHub commits the bot's work as `noreply@github.com` + # and only the author half names which App wrote it. + case "$(git show -s --format='%ae' "$sha")" in + *'[bot]@users.noreply.github.com') + if [ -z "$(printf '%s\n' "$touched" | grep -vE '^(\.github/workflows/[^/]+\.ya?ml|mise\.(toml|lock)|Cargo\.(toml|lock))$' || true)" ]; then + echo "BOT ${sha:0:8} bot-authored manifest bump — its row is minted from the PR, after the commit" + continue + fi + ;; + esac # ` Date: Sun, 30 Aug 2026 18:51:40 +0000 Subject: [PATCH 2/4] fix(renovate): freeze a persistently red bot head instead of rebasing it forever MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `draftPR`, `rebaseWhen` and the lander's ready arm are each right on their own, and none of them covers a head that is Ready and red for a reason no bump can fix. Nothing returns it to draft, so `main` advancing rebases a Ready PR and each rebase is a `synchronize`: a full matrix that cannot land, forever. Measured on #676: seven days open, 372 workflow runs on one branch, 8 rebase cycles in one day. Re-drafting on red does not terminate this, which is why the key is a label and not a retry: drafting stops CI on that sha, the next `main` advance mints a fresh one nothing has graded, and the lander correctly readies it — the identical bill. A label is keyed to the pull request, survives every rebase, and works by stopping the rebases. The retired clause priced freezing against a transiently red head, where two matrices beat one. That arithmetic does not hold when the redness is structural. Refs: CLOUD-1207 --- renovate.json5 | 46 +++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 43 insertions(+), 3 deletions(-) diff --git a/renovate.json5 b/renovate.json5 index 5d21335b9..cb71367c9 100644 --- a/renovate.json5 +++ b/renovate.json5 @@ -91,11 +91,51 @@ // A rebase that arrives mid-CI on a head `auto-bot-land.yml` has readied is // absorbed rather than paid for: `ci.yml`'s `concurrency: cancel-in-progress` // cancels the run the rebase superseded, and the next one grades the head that - // can actually land. Freezing the branch instead (Renovate's - // `stopUpdatingLabel`) costs a matrix that cannot land AND a second one after - // the unfreeze, so it is strictly worse. + // can actually land. + // + // THE CLAUSE THAT USED TO END THIS COMMENT IS RETIRED RATHER THAN DROPPED, and + // the key below is what replaces it. It read: freezing the branch instead + // (Renovate's `stopUpdatingLabel`) "costs a matrix that cannot land AND a + // second one after the unfreeze, so it is strictly worse." That arithmetic is + // right for a head that is TRANSIENTLY red — two matrices beats one — and it + // silently assumed the only alternative was freezing EVERY head. See below for + // what it costs when the redness is not transient. rebaseWhen: "behind-base-branch", + // THE TERMINATOR (CLOUD-1207), and what it terminates is a loop every other key + // in this file was individually right about. + // + // `draftPR: true` makes a rebase free, `rebaseWhen` above keeps the head + // landable, and `auto-bot-land.yml` readies a landable draft so exactly one + // matrix is spent on the SHA about to land. Each holds. What none of them + // covers is a head that is READY and red for a reason no bump can fix: nothing + // returns it to draft, so `main` advancing rebases a Ready PR, and each rebase + // is a `synchronize` — a full matrix that cannot land, forever. + // + // MEASURED ON #676: seven days open, **372 workflow runs on one branch**, 8 + // rebase cycles on 2026-08-30 alone, each buying `CI` (~19 required checks), + // `Rust`, `commit-lint` and `zizmor`. It was red on `commit-lint` — a claim a + // bot commit structurally cannot make, which CLOUD-1207 fixes one file over — + // and on `sbom-check`, whose licence table only a human regenerates + // (CLOUD-1213). Neither is transient, and that is the case the retired clause + // above did not price. + // + // WHY A LABEL AND NOT A SHA-KEYED RETRY, because the obvious fix does not work + // and the reasoning is worth keeping: re-drafting a red head stops CI on THAT + // sha, then the next `main` advance rebases, mints a fresh sha nothing has + // graded, and the lander correctly readies it — one matrix per `main` advance, + // the identical bill. A verdict keyed to a sha cannot bound a loop whose every + // lap mints a new one. The label is keyed to the PULL REQUEST, so it survives + // every rebase, and it works by stopping the rebases themselves: Renovate + // leaves a labelled PR alone, so no new head is minted and no matrix is spent. + // + // THE COST, STATED RATHER THAN DISCOVERED. A frozen head goes stale and cannot + // fast-forward, and it still holds the `prConcurrentLimit: 1` slot below. That + // is the honest shape of the situation: a bot bump that needs a human is not + // making progress whether or not this repository pays CI for it. Removing the + // label is the resume, and the next rebase mints a head the lane readies once. + stopUpdatingLabel: "renovate-frozen", + // ONE HEAD IN FLIGHT. With `rebaseWhen` no longer `never` this is no longer the // second half of a pairing — it is the throttle in its own right, and it is // what makes the lane's cost legible: at most one draft accumulating, at most From ee0f926c84bbd07d3d7a3821e7523d7f0ee7f5d9 Mon Sep 17 00:00:00 2001 From: Alec Wenzowski Date: Sun, 30 Aug 2026 19:05:50 +0000 Subject: [PATCH 3/4] ci(auto-bot-land): draft and freeze a red head instead of readying it again MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The lane readied a landable draft and had no arm for a head that stayed landable and failed. Its only re-draft fires on a stale head, and Renovate rebases within minutes of `main` moving, so that arm never sees one. Three edits. The verdict step moves above the ready arm and drops its `draft == false` gate, so this tick can ask about a draft too. Its outputs go three ways where exit 1 and exit 3 were one `green=false` — collapsing them made a head that failed indistinguishable from one nothing had graded, and the two want opposite treatment. Then the red arm drafts the head, applies the `stopUpdatingLabel` that `renovate.json5` declares, and comments once. The label is what terminates, not the draft: drafting stops CI on that sha, the next `main` advance mints a fresh one nothing has graded, and the ready arm correctly readies it. A verdict keyed to a sha cannot bound a loop whose every lap mints a new sha; a label is keyed to the pull request and stops the rebases. Refs: CLOUD-1207 --- .github/workflows/auto-bot-land.yml | 123 ++++++++++++++++++++++++---- 1 file changed, 105 insertions(+), 18 deletions(-) diff --git a/.github/workflows/auto-bot-land.yml b/.github/workflows/auto-bot-land.yml index e0ee4bcc6..542787a77 100644 --- a/.github/workflows/auto-bot-land.yml +++ b/.github/workflows/auto-bot-land.yml @@ -184,6 +184,10 @@ jobs: # The author half of the same test, applied in the resolve step below where # the PR object is in hand. Named here so both arms read one constant. BOT_LOGIN: renovate[bot] + # The freeze `renovate.json5` declares as `stopUpdatingLabel` (CLOUD-1207). + # The two must agree by string or the freeze is a label nothing reads, so + # this is the one place either is spelled in a workflow. + FREEZE_LABEL: renovate-frozen steps: # One resolution step for both triggers, so everything after it is written # once. `workflow_run` arrives holding the SHA and looks up its PR; the cron @@ -358,29 +362,27 @@ jobs: GH_TOKEN: ${{ secrets.RELEASE_PLZ_TOKEN || secrets.GITHUB_TOKEN }} PR_NUM: ${{ steps.target.outputs.num }} run: gh pr ready "$PR_NUM" --repo "$REPO" --undo - # THE ONE EVENT THAT STARTS CI, and it starts it on the SHA that is about to - # land. Readying with the PAT, not GITHUB_TOKEN: a `ready_for_review` from - # the Actions token fires no workflow, so CI would never run and the landing - # would wait forever on checks nothing creates. - - name: Ready a landable draft so CI runs on the SHA that will land - if: >- - steps.target.outputs.num != '' && - steps.target.outputs.draft == 'true' && - steps.linear.outputs.status == 'ahead' - env: - GH_TOKEN: ${{ secrets.RELEASE_PLZ_TOKEN || secrets.GITHUB_TOKEN }} - PR_NUM: ${{ steps.target.outputs.num }} - run: gh pr ready "$PR_NUM" --repo "$REPO" # The same predicate `ci-wait` blocks on, asked once: at least one required # check graded, none skipped, none pending, each name judged by its latest # run. Exit 3 "no answer yet" and exit 1 "red" are both ordinary here and must # not fail the run — the matrix this tick just started has graded nothing yet, # and a red head waits for its next push. Only exit 2, "could not look", is a # failure. Read with GITHUB_TOKEN: it carries checks:read and the PAT does not. - - name: Is every required check green on the head SHA? + # + # ASKED BEFORE THE READY, AND OF A DRAFT TOO (CLOUD-1207). It ran after it, + # and only for a Ready head, which left this tick's own question — is this + # draft worth readying at all — unasked. A draft's required checks are + # skipped or absent, so `checks-green` answers exit 3 for a fresh head, and + # that reading is what keeps the ready arm below the default rather than a + # new gate on the lane. + # + # AND THE THREE VERDICTS ARE THREE OUTPUTS, where exit 1 and exit 3 were one + # `green=false`. Collapsing them made a head that FAILED indistinguishable + # from one nothing has graded yet, and the two want opposite treatment: + # `pending` waits, `red` stops the lane. Neither fails this run. + - name: What is the verdict on the head SHA? if: >- steps.target.outputs.num != '' && - steps.target.outputs.draft == 'false' && steps.linear.outputs.status == 'ahead' id: checks env: @@ -394,10 +396,95 @@ jobs: mise run checks-green verdict=$? case "$verdict" in - 0) echo "green=true" >> "$GITHUB_OUTPUT" ;; - 1 | 3) echo "green=false" >> "$GITHUB_OUTPUT" ;; + 0) echo "verdict=green" >> "$GITHUB_OUTPUT" ;; + 1) echo "verdict=red" >> "$GITHUB_OUTPUT" ;; + 3) echo "verdict=pending" >> "$GITHUB_OUTPUT" ;; *) exit "$verdict" ;; esac + # A RED HEAD IS DRAFTED **AND FROZEN**, AND THE SECOND HALF IS THE ONE THAT + # TERMINATES (CLOUD-1207). Drafting alone does not, and reasoning that it + # does is the trap this comment exists to disarm: it stops CI on THIS sha, + # then `rebaseWhen: "behind-base-branch"` mints a new one on the next `main` + # advance, that head is a draft nothing has graded, and the ready arm below + # correctly readies it — one full matrix per `main` advance, exactly the bill + # the lane was already paying. A verdict keyed to a sha cannot bound a loop + # whose every lap mints a fresh sha. + # + # So the freeze is what stops the REBASES, and it is keyed to the pull + # request instead: Renovate leaves a head carrying `stopUpdatingLabel` alone, + # so no new sha is minted and no matrix is spent, until a person takes the + # label off. `renovate.json5` declares that label and carries the economy + # this reverses. + # + # MEASURED, AND THIS IS WHY IT IS A FREEZE RATHER THAN A RETRY BUDGET. #676 + # was red on `commit-lint` (a claim no bump can make, fixed in the same + # change as this) and on `sbom-check`, whose licence table only a human + # regenerates (CLOUD-1213). Neither is transient, and 372 workflow runs in + # seven days is what "wait for the next push" costs when the next push + # cannot help. Such a head needs a person; the lane's job is to stop paying + # while it waits for one, and to say so exactly once. + # + # THE LABEL IS ALSO THE ONCE, guarding the comment rather than a marker of + # its own: a tick that finds it set does nothing at all. Pointer-only per + # non-negotiable rule 4 — the failing check NAMES, never their output. + - name: Draft and freeze a red head, and say once why + if: >- + steps.target.outputs.num != '' && + steps.checks.outputs.verdict == 'red' + env: + GH_TOKEN: ${{ secrets.RELEASE_PLZ_TOKEN || secrets.GITHUB_TOKEN }} + PR_NUM: ${{ steps.target.outputs.num }} + SHA: ${{ steps.target.outputs.sha }} + run: | + set -euo pipefail + labels=$(gh pr view "$PR_NUM" --repo "$REPO" --json labels --jq '.labels[].name') + if grep -qxF "$FREEZE_LABEL" <<<"$labels"; then + echo "#$PR_NUM is already frozen; this tick spends nothing" + exit 0 + fi + # `--force` is idempotent creation: the label need not already exist in + # the repository, and a second run neither fails nor rewrites anything. + gh label create "$FREEZE_LABEL" --repo "$REPO" --force \ + --color B60205 \ + --description "Red on a check no bump can fix; Renovate is not rebasing it" >/dev/null + gh pr ready "$PR_NUM" --repo "$REPO" --undo + gh pr edit "$PR_NUM" --repo "$REPO" --add-label "$FREEZE_LABEL" >/dev/null + failing=$(gh api "repos/$REPO/commits/$SHA/check-runs" \ + --jq '[.check_runs[] | select(.conclusion == "failure") | .name] | unique | join(", ")') + { + echo "This head is red on: ${failing:-a required check}." + echo + echo "The lane has returned it to draft and added \`$FREEZE_LABEL\`, so" + echo "Renovate will stop rebasing it and this pull request stops spending" + echo "CI minutes it cannot convert into a landing. Nothing here retries on" + echo "its own: a dependency bump cannot fix a check that needs a change to" + echo "the repository." + echo + echo "To resume, land the fix on \`main\` and remove the label. The next" + echo "rebase mints a fresh head, the lane readies it once, and it lands if" + echo "it is green." + } > "$RUNNER_TEMP/freeze-comment.md" + gh pr comment "$PR_NUM" --repo "$REPO" --body-file "$RUNNER_TEMP/freeze-comment.md" >/dev/null + echo "drafted and froze #$PR_NUM" + # THE ONE EVENT THAT STARTS CI, and it starts it on the SHA that is about to + # land. Readying with the PAT, not GITHUB_TOKEN: a `ready_for_review` from + # the Actions token fires no workflow, so CI would never run and the landing + # would wait forever on checks nothing creates. + # + # `verdict != 'red'` is the narrower of the two guards rather than the + # load-bearing one: it stops THIS tick from re-readying the very head the + # step above just drafted, which would make the freeze an hourly + # oscillation. What stops the next head arriving at all is the freeze. + - name: Ready a landable draft so CI runs on the SHA that will land + if: >- + steps.target.outputs.num != '' && + steps.target.outputs.draft == 'true' && + steps.linear.outputs.status == 'ahead' && + steps.checks.outputs.verdict != 'red' + env: + GH_TOKEN: ${{ secrets.RELEASE_PLZ_TOKEN || secrets.GITHUB_TOKEN }} + PR_NUM: ${{ steps.target.outputs.num }} + run: gh pr ready "$PR_NUM" --repo "$REPO" # `force=false` is the fast-forward assertion: the API refuses the update # unless `main` is an ancestor of this SHA, and branch protection independently # refuses it unless `final` is green on it. So the two things that must hold are @@ -420,7 +507,7 @@ jobs: # refusal below: the next tick re-runs `ensure`, the key comes back, and it # lands then. Only exit 2, "could not look", fails the run. - name: Does the body still close its row? - if: steps.checks.outputs.green == 'true' + if: steps.checks.outputs.verdict == 'green' id: closes env: GH_TOKEN: ${{ github.token }} From 93d3ebc6627a9f178a75a57945f74e2e558d98bc Mon Sep 17 00:00:00 2001 From: Alec Wenzowski Date: Sun, 30 Aug 2026 19:37:47 +0000 Subject: [PATCH 4/4] ci(auto-bot-land): state the bound on "no human in the loop" The header claimed the whole lane, from a draft nobody has looked at to a fast-forward. Two committed gates refuse that for some `github-actions` bumps: `sbom-check` maps each pinned action's licence by repo and sha, so a pin move orphans every row until a person regenerates the table, and `mise-action-floor` reds a pin it denies. Neither is reachable by a bump. What the lane owes in that case is a stop rather than a landing, which the freeze arm now provides. Saying so in the header is the difference between a bounded claim and one a red PR discovers for you. Refs: CLOUD-1207 --- .github/workflows/auto-bot-land.yml | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/.github/workflows/auto-bot-land.yml b/.github/workflows/auto-bot-land.yml index 542787a77..b7e7993dd 100644 --- a/.github/workflows/auto-bot-land.yml +++ b/.github/workflows/auto-bot-land.yml @@ -3,6 +3,17 @@ name: auto-bot-land # Land the bot's pull requests with no human in the loop — the whole lane, from a # draft nobody has looked at to a fast-forward on `main`. # +# THAT CLAIM IS BOUNDED, AND THE BOUND IS STATED HERE RATHER THAN DISCOVERED ON A +# RED PR (CLOUD-1207). Two committed gates need a change to the REPOSITORY that no +# bump can carry: `sbom-check` maps each pinned action's licence by repo AND sha, +# so a `github-actions` bump orphans every row until a person runs `mise run sbom` +# and commits the table, and `mise-action-floor` reds a pin it denies. So the lane +# is unattended for `cargo` and `mise` bumps and attended for some +# `github-actions` ones. What this workflow owes in that case is not a landing — +# it cannot produce one — but a stop: the freeze arm below drafts the head, halts +# the rebases, and says once what failed and what to run. CLOUD-1213 carries +# whether that class should need a person at all. +# # This file is `auto-dependabot-land.yml` repointed, not a second lander beside it # (CLOUD-660). That workflow is deleted in the same commit, along with # `.github/dependabot.yml`: Dependabot version updates moved to Renovate one