diff --git a/.mcp.json b/.mcp.json index 27b894c32..fbd1c2569 100644 --- a/.mcp.json +++ b/.mcp.json @@ -1,8 +1,13 @@ { "mcpServers": { "serena": { - "command": "mise-tasks/serena-mcp.sh", + "command": "batten", "args": [ + "mcp", + "spawn", + "serena", + "--", + "mise", "exec", "pipx:serena-agent@1.7.0", "--", diff --git a/.serena/memories/core.md b/.serena/memories/core.md index 08f98bed6..3130ade7b 100644 --- a/.serena/memories/core.md +++ b/.serena/memories/core.md @@ -64,6 +64,22 @@ err)` takes **both** channels and the resolved `Mode`, so a verb can write a (CLOUD-208; it closed CLOUD-42's G10, "nothing emits at `Verbose` or above yet"). `out` is the answer, `err` the messaging — `batten exec` reports its output matches through `err` for that reason. +- `suites.rs` — the per-suite cost corpus (CLOUD-352), derived from the report + the bats runner already wrote. **IT RUNS NOTHING**: re-executing a + multi-thousand-case suite to measure it would cost more than the waste it + reports and would be a second authority over a run that already happened, so + this opens one file and writes one file and reaches + `policy/spawn-adapters.rego` not at all. The corpus path is a CONTRACT with + `policy/suite-cost-corpus.rego`, which reads the same bytes as a declared + `lines` source — the producer decides cost, the gate decides MEMBERSHIP, and + neither half can see the other. That division is the design: wall clock is a + clock and belongs in a drift job, membership is deterministic and belongs in a + gate. THREE STATES ARE COULD-NOT-LOOK, never an empty corpus — an absent + report (the ordinary state after a receipt-gated no-op lap), a report carrying + no readable ``, and a report naming a suite the tree no longer + tracks. The last one is what makes the gate's remedy reachable: a suite retired + while the report still named it once produced a corpus carrying a cost attached + to nothing, so the gate refused the very file its own remedy had just written. - `surface.rs` — house-style §11, CLOUD-27: the command tree declared **once**, as data (`ROOT` + `SURFACE`) — path, summary, effect, and flags (with each flag's env equivalent, so §8 precedence is inspectable data). `command()` @@ -384,6 +400,22 @@ budget` and **enforced on `check`**. `[budget.]` is a MAP, not a struct wi whole-set reading let one dead glob contribute nothing while the rest counted and still reported green (CLOUD-298). A config declaring no budget is exit 1 too — a budget verb that measured nothing must not report `0`. +- `arm.rs` — the declared-arm harness (CLOUD-1714): run N declared things, + reduce each to a named `Observable`, hand back one `Outcome` per arm. The + primitive `perf.rs` and `mutate.rs` were each one instance of, plus the two the + bash corpus held. **`percentile` is owned here** — four copies existed between + the four instances, and `perf::summarise` now calls this one, so CLOUD-1712's + fetched-duration percentiles have no fifth. It takes the quantile as a RATIO OF + INTEGERS and ranks with `div_ceil`, which is why the rank arithmetic carries no + lint escape: the `f64` form needed three, and `spawn-widening` counts an added + escape as inventory growth whatever the reasoning behind it. `Isolation::at` + sets `HOME`, `XDG_DATA_HOME`, `APPDATA` and `LOCALAPPDATA` together, because an + arm inheriting one of them measures the ambient toolchain instead of the + subject. **A FAILED ARM IS `NotObserved`, NEVER A ZERO**: a zero measurement is + the could-not-look collapse CLOUD-251 names, and it reads as the fastest arm in + the table. Stability is checked BEFORE a byte count is reduced, so a run set + that disagrees is reported `Unstable` with its distinct count rather than + averaged into one plausible number. - `attribution.rs` — what produced commits may carry about the tooling that made them (CLOUD-274), the mechanism for the attribution decision record (CLOUD-268). Judges author/committer identity, every trailer and the message diff --git a/AGENTS.md b/AGENTS.md index 4fb90f2d3..792539e78 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -161,8 +161,8 @@ half**: declared work dies too, so **"unsaved?" is `batten doctor session`**. never a model verdict. _(house-style §5.)_ 4. **Output is a pointer, never the payload** — a count, `path:line`, or boolean, never the content. _(house-style §6.)_ -5. **Exit codes and output follow the one contract** — byte-stable output, the - `0/1/2/3` table, no per-verb exception. _(house-style §6–§7.)_ +5. **Exit codes and output follow one contract** — byte-stable, `0/1/2/3`, no + per-verb exception (§6–§7). The corpus INVERTS `1`/`2`: fold `batten verdict`. 6. **Keep configuration narrow.** One committed authority plus raise-only overrides, no directory walk, no `conf.d` merge (house-style §8). Don't widen it. 7. **Research goes to Linear, not a repo `docs/` tree.** Evidence notes and literature diff --git a/batten.toml b/batten.toml index 504c8e5a4..c2ddeb576 100644 --- a/batten.toml +++ b/batten.toml @@ -811,13 +811,20 @@ reason = """ Rebasing onto `main` by hand is a lap of the loop `mise run land` drives, and \ taking it off the task is the race the contract refuses rather than a repair. \ Background `mise run land`; it fetches, rebases, verifies, pushes and waits, \ -and stops only for a rebase conflict, a failed `verify`, or red CI. TWO \ -SPELLINGS ARE LEFT ALONE, and neither is a lap: the conflict exits \ -(`--continue`, `--abort`, `--skip`) name no target at all, and \ -`git rebase --onto ` is a range move this loop \ -cannot perform. A CONFLICT is the one step that is yours, and this row denies \ -rather than warns because a warn row on this surface reaches nobody — so the \ -conflict stop is a limit this row names rather than one it can soften.""" +and stops only for a rebase conflict, a failed `verify`, or red CI. A CONFLICT \ +is the one step that is yours, and the route is \ +`batten land replay --resolve `: merge the path in the worktree and \ +name it, or `--resolve =` for a path that conflicts at more than \ +one commit, one entry per conflict in order. THE REPLAY IS STATELESS, so there \ +is no rebase in progress and `--continue`, `--abort` and `--skip` cannot apply \ +to it — this row used to name those three as the way out, which is a remedy \ +that does not exist (CLOUD-1050's class, measured when a session read it, \ +concluded the loop was broken, and reached for a cherry-pick that would have \ +completed the replay while writing no lap record). \ +`git rebase --onto ` stays open and is not a lap: \ +it is a range move this loop cannot perform, which is what `unless_contains` \ +admits. This row denies rather than warns because a warn row on this surface \ +reaches nobody.""" # The toolchain pin, given a mechanism (CLOUD-271). AGENTS.md has said "never a # bare `cargo`" since long before anything enforced it, which non-negotiable @@ -1656,6 +1663,126 @@ regex = '(^|/)tasks/[A-Za-z0-9._-]+\.output$' id = "egress-resolver-host" regex = 'api\.github\.com' +# CLOUD-843's port of `pipefail-grep-check`. Three static halves; the flag +# CLUSTER is read letter by letter in the module rather than enumerated here, +# because "a short-flag cluster containing q or l" is a decision and not a +# spelling — an enumeration is the thing that would rot. +[[pattern]] +id = "shell-enables-pipefail" +regex = '^[[:space:]]*set[[:space:]]+-[a-z]*o?[a-z]*[[:space:]]*.*pipefail' + +# A REAL pipe, never the second bar of `||`. The predecessor scanned +# `\|[[:space:]]*grep`, which matched `a || grep -q ... <<<"$x"` — the very +# remedy the gate recommends — and reported it as the defect. Measured on +# `ready-lint.sh` (CLOUD-852). +[[pattern]] +id = "pipe-into-grep" +regex = '(^|[^|])\|[[:space:]]*grep([[:space:]]|$)' + +# CLOUD-843's port of `awk-regex-check`. The NAME is a capture rather than the +# whole match: the module compares full identifiers so that a name which merely +# prefixes another is never confused for it, which the shell approximated with a +# trailing character class. +[[pattern]] +id = "awk-v-assignment" +regex = '-v[[:space:]]*([A-Za-z_][A-Za-z0-9_]*)=' + +# The leading identifier of a fragment, used to read what follows `~` or a comma +# inside `match(...)`. A whole-token comparison is what makes the prefix case +# decidable without a per-name regex, which a `[[pattern]]` row cannot express. +[[pattern]] +id = "leading-identifier" +regex = '^[A-Za-z_][A-Za-z0-9_]*' + +# A full git object id, lowercase hex. `mise-action-floor` uses it to tell a PINNED +# coordinate from a floating one: `@v4` carries no sha, so a denylist cannot speak +# about it at all and reporting green over it would be a claim the gate cannot +# support. +[[pattern]] +id = "git-object-id" +regex = '^[0-9a-f]{40}$' + +# The two spellings a registry credential takes in a workflow. The +# alternate-registry form carries the registry's own name in its middle segment, +# so it is matched on its shape rather than enumerated. +[[pattern]] +id = "cargo-registry-token" +regex = 'CARGO_REGISTRY_TOKEN|CARGO_REGISTRIES_[A-Z0-9_]*_TOKEN' + +# A login call, bounded on both sides so a word merely containing it does not +# fire. +[[pattern]] +id = "cargo-login-call" +regex = '(^|[^A-Za-z0-9_-])cargo login([^A-Za-z0-9_-]|$)' + +# The OIDC permission as a KEY, so the phrase in a comment cannot satisfy the +# implication it discharges. +[[pattern]] +id = "oidc-token-permission" +regex = '^[[:space:]]+id-token:[[:space:]]*write[[:space:]]*(#.*)?$' + +# A workflow JOB key, anchored at exactly two spaces. The indent is what makes it +# a job rather than a step's field. +[[pattern]] +id = "workflow-job-key" +regex = '^ [A-Za-z0-9_-]+:[[:space:]]*$' + +# A top-level workflow key, which is what ends the jobs block. +[[pattern]] +id = "workflow-top-level-key" +regex = '^[a-z][A-Za-z0-9_-]*:' + +# A JOB-level timeout, at exactly four spaces. A step-level one sits deeper and +# is the step's business; matching it here would let a step satisfy a job's +# obligation. +[[pattern]] +id = "job-timeout-line" +regex = '^ timeout-minutes:[[:space:]]*[0-9]+' + +# The measured budget form, capturing the p95 and the multiplier so the +# arithmetic can be checked. Matched WHOLE, so a comment that merely mentions the +# word cannot pass. +[[pattern]] +id = "timeout-budget-measured" +regex = '^#[[:space:]]*budget:[[:space:]]*p95=([0-9]+)s[[:space:]]+x([0-9]+)[[:space:]]+measured=([0-9]{4}-[0-9]{2}-[0-9]{2})[[:space:]]*$' + +# The grandfathered form. Not a synonym for exempt: it says the value predates +# measurement and nobody has justified it yet, which reads as debt. +[[pattern]] +id = "timeout-budget-grandfathered" +regex = '^#[[:space:]]*budget:[[:space:]]*grandfathered[[:space:]]+measured=[0-9]{4}-[0-9]{2}-[0-9]{2}[[:space:]]*$' + +# An install list, which is a one-line plain scalar by the workflows' own +# convention: a block scalar would need continuation tracking, and several tool +# names carry a backend prefix so they read as new keys under any such heuristic. +[[pattern]] +id = "install-args-line" +regex = '^[ \t]*install_args:' + +# The trigger that puts a workflow on the pull-request path, which is the scope +# the install-list rows are argued for. +[[pattern]] +id = "workflow-pull-request-trigger" +regex = '^[ \t]*pull_request:' + +# The ASSIGNMENT set to false, not a mention. A substring search would pass a +# workflow whose only occurrence is the comment explaining why the variable +# matters, and would pass one setting it true — the same hole in a fix's clothing. +[[pattern]] +id = "mise-task-auto-install-off" +regex = '^[ \t]*MISE_TASK_RUN_AUTO_INSTALL:[ \t]*"?false"?[ \t]*$' + +# The other half of the same binding. +[[pattern]] +id = "mise-exec-auto-install-off" +regex = '^[ \t]*MISE_EXEC_AUTO_INSTALL:[ \t]*"?false"?[ \t]*$' + +# A tool a rule's own check spawns, which no install list reads and no `[tools]` +# scan can see. +[[pattern]] +id = "mise-exec-spawn" +regex = 'mise exec -- [a-z][a-z0-9._-]*' + [[pattern]] id = "release-tag" regex = '^v[0-9]+\.[0-9]+\.[0-9]+$' @@ -4999,9 +5126,16 @@ no_fix_reason = "migrate the predicate onto a rule kind, or declare `# stays-bas # TWO SURFACES ARE DELIBERATELY EXEMPT, and the reasons differ, so a later reader # finds a decision rather than a gap: # -# .github/workflows/** bash is allowed. A workflow step is the one place a -# shell line is the native spelling, and there is no -# engine surface it could migrate onto. +# .github/workflows/** bash is allowed, and since CLOUD-1709 it is COUNTED. +# A workflow step is the one place a shell line is the +# native spelling, and there is no engine surface it +# could migrate onto — so it is exempt from THIS row, +# which globs `mise-tasks/**`. It is not exempt from +# measurement: `workflow-shell-not-growing` and its +# `-bare` sibling ratchet the step count. Allowed and +# uncounted are different claims, and reading the first +# as the second is what 497 duplicated lines were +# written under. # install.sh it cannot be a policy module by construction: it is how # the binary reaches a host that does not have it yet, so # it runs before the engine exists. @@ -5116,6 +5250,91 @@ severity = "deny" scope = "tree" no_fix_reason = "migrate the predicate onto a rule kind, or waive the increase deliberately with a reason and an expiry; which of the two is the whole question" +# CLOUD-1709. THE THIRD SURFACE, AND THE LARGEST. `bash-surface-not-growing` +# globs `mise-tasks/**`; CLOUD-1137 closed the `mise.toml` and `.claude/**` holes +# above. `.github/workflows/**` was left, and it is bigger than either: 28 files +# carrying 135 shell steps, on a surface nothing counted. +# +# THE EXEMPTION ABOVE IS ABOUT MIGRATION, NOT ABOUT COUNTING, and these rows are +# the correction of that inference rather than of the exemption itself. Workflow +# bash stays allowed — a step is the native spelling and there is no engine +# surface it could move onto. What does not follow is that it need not be +# MEASURED, and 41% byte-identical duplication is what that inference bought. +# +# WHAT IS COUNTED IS THE STEP, NOT THE LINE, and the row body's 1,215-line figure +# is deliberately NOT the predicate. A ratchet `pattern` is a literal substring +# counted with `str::matches` (CLOUD-1058) — it cannot see where a YAML block +# ends, so "lines inside a `run:` block" is not expressible here. The step is, +# and it is the right unit anyway: `#MISE description=` counts programs for the +# same reason one line up. +# +# TWO LITERALS FOR ONE CONCEPT, exactly as the `mise.toml` pair above, and for +# the same CLOUD-1058 reason. Measured over the 28 workflows, `run:` appears at +# precisely two indentations and no others: `\n run:` (the key under a +# `- name:` step) 111 times, and `\n - run:` (the bare step) 24 times. +# Both are newline-anchored, which is load-bearing: the unanchored literal also +# matches `workflow_run:` and `check_run:`, 15 of them today, so it would count +# trigger declarations as shell. +# +# THE BLIND SPOTS, STATED RATHER THAN DISCOVERED LATER (DoR §2). A `run:` at any +# third indentation is not counted — none exists today, and a nested one inside a +# composite action would need its own row. `.github/actions/**` is not globbed at +# all, because no composite action exists yet; the follow-up that deduplicates +# the 497 lines into one will have to add it, and that is the point at which this +# row would otherwise report a fall it did not earn. +# +# THE FIRING RATE, replayed before the severity was chosen (DoR §7). Over all +# 2,574 commits on `origin/main`, 181 of which touch a workflow: the 8-space row +# fired 59 times, the dash row 21. As a share of all commits that is 2.3% and +# 0.8%, against the `mise.toml` pair's 1.6% and 0.07%. +# +# AND THE FALSE POSITIVES ARE REAL HERE, which the `mise.toml` rows did not have +# to answer for and which decides the `admits_with` column below. At least five +# firings are THIS CAMPAIGN SUCCEEDING: retiring a shell program onto a verb +# leaves a workflow step invoking it, so the step count rises while real bash +# falls. `4b8fc385f` (`retire land-lock-check.sh onto batten lease check`, +1), +# `8a90687a3` (`the perf budget is a rule, not a shell program`, +1) and +# `c5346097d` (`move the latency family into the perf namespace`, +1) on the +# 8-space row; `1e1d19874` (`retire ci-drift onto the verb it was already +# calling`, +1) and `c5346097d` again on the dash row. That is the exact class +# `batten.toml` rejected the wider `run = ` literal for one block up — *"a row +# that denies its own campaign's wins is worse than the hole it closes."* +# +# SO THE PERMIT IS CARRIED RATHER THAN THE SEVERITY LOWERED. `warn` would answer +# the false positives by making every firing advisory, including the 75 that are +# real net-new shell. `admits_with` answers only the firings an author is willing +# to sign for, and the cardinality objection that keeps it off both `mise.toml` +# rows does not apply: those glob exactly ONE file, so the first permit spent +# switches the row off forever, whereas this globs a DIRECTORY, which is the +# per-file semantics the column was designed for. A retirement that adds a step +# declares it in the workflow it added it to, and every other workflow stays +# ratcheted. +[[rule]] +id = "workflow-shell-not-growing" +kind = "ratchet" +glob = ".github/workflows/**" +pattern = "\n run:" +direction = "non_increasing" +base = "origin/main" +admits_with = "# workflow-shell:" +severity = "deny" +scope = "tree" +no_fix_reason = "move the step's body into a composite action or a batten verb, or declare `# workflow-shell: ` in the workflow and own the increase; which of the two is the whole question" + +# The other spelling of the same concept — see the block above for why it is a +# second row rather than a second pattern on the first. +[[rule]] +id = "workflow-shell-not-growing-bare" +kind = "ratchet" +glob = ".github/workflows/**" +pattern = "\n - run:" +direction = "non_increasing" +base = "origin/main" +admits_with = "# workflow-shell:" +severity = "deny" +scope = "tree" +no_fix_reason = "move the step's body into a composite action or a batten verb, or declare `# workflow-shell: ` in the workflow and own the increase; which of the two is the whole question" + # The `.claude/**` half. `#!` rather than a shebang's text, because the programs # there do not share a spelling — `#!/usr/bin/env bash` twice today, and the file # this row's baseline lost spelled it `#!/bin/sh` — so a text literal would count @@ -6759,10 +6978,86 @@ input = "target/release/batten" # `error: test run failed`, exit 100. # # SO THIS ROW GUARDS A DECLARATION RATHER THAN A DURATION, and that division is -# deliberate. `mise-tasks/suite-bench-check.sh` records why a duration gate is the -# wrong instrument — it "would be red on every second run and would be bypassed -# within a day" — and CLOUD-1419 wrote an aggregate ratchet and withdrew it in the -# same branch. nextest measures and decides; batten refuses the weakening. +# deliberate. `policy/suite-cost-corpus.rego` records why a duration gate is the +# wrong instrument — it "would be red on every second run and switched off within +# a day" — and CLOUD-1419 wrote an aggregate ratchet and withdrew it in the same +# branch. nextest measures and decides; batten refuses the weakening. +# +# The membership half of that division is the row below, ported off +# `mise-tasks/suite-bench-check.sh` under CLOUD-1753. + +# The per-suite cost corpus covers exactly the tracked suites (CLOUD-352). +# +# THE DIVISION THE ROW ABOVE NAMES, from the other side. Wall clock is a CLOCK and +# belongs in a drift job; MEMBERSHIP is deterministic and belongs in a gate. A +# suite added and never recorded is invisible to everything that reads the corpus, +# and a row naming a deleted suite is a figure attached to nothing — both decidable +# from committed text at no runtime cost. +# +# `lines` NAMES ONE LITERAL PATH, on `nextest-slow`'s reading directly above: a +# tree without the corpus still acquires the declared source, so the could-not-look +# arm fires on the absent document instead of leaving the rule SKIPPED. That is the +# arm that matters most here — an absent corpus is precisely the state where +# nothing records what a suite costs. +# +# The producer is `batten record suites`, and the path is a contract between the +# two halves rather than an implementation detail of either: `suites.rs` spells it +# as `CORPUS` and this row spells it here. +[[rule]] +id = "suite-cost-corpus" +kind = "policy" +scope = "tree" +lines = ["bench/suites/RESULTS.md"] +module = "policy/suite-cost-corpus.rego" +severity = "deny" + +[[verdict]] +id = "suite count unread" +gloss = "the per-suite cost corpus is absent or carries no row, so nothing records what a suite costs" +class = """ +Could-not-look, never a clean corpus — the whole subject of the gate is that a cost is recorded, so an unreadable corpus is the state it exists to report. Run `mise run test:bats`, then `batten record suites --write`. +""" + +[[verdict.route]] +id = "suite measure now" +kind = "command" +target = "batten record suites --write" + +[[verdict]] +id = "suite list empty" +gloss = "no tracked bats suite exists, so the cost corpus has no subject to cover" +class = """ +Could-not-look rather than a corpus that covers everything there is. Reporting green over a tree with nothing to measure is the collapse that lets a gate pass on ignorance. +""" + +[[verdict.route]] +id = "test add first" +kind = "document" +target = "tests/" + +[[verdict]] +id = "suite count absent" +gloss = "a tracked suite has no row in the cost corpus, so an author editing it is told nothing about what it costs" +class = """ +The corpus is read to answer "is the file I am about to add a case to expensive". A suite it does not name is invisible to that question. Regenerate after a run that covered it. +""" + +[[verdict.route]] +id = "suite measure now" +kind = "command" +target = "batten record suites --write" + +[[verdict]] +id = "suite count dead" +gloss = "the cost corpus records a suite this tree does not track — a figure attached to nothing" +class = """ +Either the suite retired and the corpus was not regenerated, or the report it was derived from predates the tree. The producer refuses the second case by name, so regenerating is the whole remedy. +""" + +[[verdict.route]] +id = "suite measure now" +kind = "command" +target = "batten record suites --write" # # `lines` NAMES ONE FILE AND THAT IS SAFE HERE, unlike `landing-roster-guarded` # which had to declare a whole directory. That module's absent-source arm was dead @@ -6813,6 +7108,265 @@ documents = ["bench/agentic/trials.toml", "bench/agentic/method.toml"] module = "policy/agentic-experiment-record.rego" severity = "deny" +# CLOUD-194, ported from `mise-tasks/module-map-check.sh` under CLOUD-843. The +# module's own header carries the predicate and why a missing map is reported +# once; it is not restated here. +# +# `line_sources` NAMES ONE FILE, and `tracked` is not declared at all because it +# is not a per-path fact — the module asks which paths exist, which the tree +# surface answers for the whole tree. Declaring the crate sources as +# `line_sources` would read every module's TEXT to learn only its NAME, which is +# the payload-for-a-pointer trade non-negotiable rule 4 refuses. +# CLOUD-813, ported from `mise-tasks/no-doctests.sh` under CLOUD-843. The +# module's own header carries the predicate, the parity rule and why `no_run` +# counts as non-running; it is not restated here. +# +# `line_sources` RATHER THAN `sources`: the predicate is over the TEXT of a doc +# comment, and no `Format` owns `.rs` — a `documents` reading would resolve to +# nothing and the rule would load clean over a workspace it never scanned. +# Ported from `mise-tasks/awk-regex-check.sh` under CLOUD-843. The module's own +# header carries the predicate and why identifiers are compared whole. +# +# The glob is the shell corpus plus the two root programs and the task manifest — +# the same set the predecessor scanned. It SHRINKS as CLOUD-843 retires programs, +# which is the campaign working: what is left at the end is `install.sh`, +# `mise.toml` and the five programs that stay bash. +# Ported from `mise-tasks/pipefail-grep-check.sh` under CLOUD-843. The module's +# own header carries the race, the two instances that landed before the class was +# named, and why the flag cluster is read letter by letter. +# Ported from `mise-tasks/license-table-check.sh` under CLOUD-843. The module's +# own header carries the predicate and why zero rows is a failure. +# CLOUD-582, ported from `mise-tasks/report-only-check.sh` under CLOUD-843. The +# module's own header carries the predicate, the task list and why only the +# `pull_request` trigger is judged. +# +# `sources` RATHER THAN `line_sources`: the predicate is over PARSED structure — +# `verify`'s `depends` list, a workflow's trigger map — which is what removes the +# span-carving and the word-boundary the shell needed. +[[rule]] +id = "report-only" +kind = "policy" +scope = "tree" +sources = ["mise.toml", ".github/workflows/*.yml", ".github/workflows/*.yaml"] +module = "policy/report-only.rego" +severity = "deny" + +[[rule]] +id = "license-table" +kind = "policy" +scope = "tree" +line_sources = ["CONTRIBUTING.md"] +module = "policy/license-table.rego" +severity = "deny" + +[[rule]] +id = "pipefail-grep" +kind = "policy" +scope = "tree" +line_sources = ["mise-tasks/**", "*.sh"] +module = "policy/pipefail-grep.rego" +severity = "deny" + +[[rule]] +id = "awk-regex" +kind = "policy" +scope = "tree" +line_sources = ["mise-tasks/**", "*.sh", "mise.toml"] +module = "policy/awk-regex.rego" +severity = "deny" + +[[rule]] +id = "no-doctests" +kind = "policy" +scope = "tree" +line_sources = ["crates/**/*.rs"] +module = "policy/no-doctests.rego" +severity = "deny" + +# CLOUD-224, ported from `mise-tasks/batten-glob-check.sh` under CLOUD-843. The +# `batten-check` step's `glob` is a SECOND authority over the set this config +# already defines, and a second authority narrows silently: add a `[[rule]]` whose +# glob names a path the step does not select, and the gate stops running for +# commits that touch only that path with nothing going red. The whole-tree run in +# CI still covers it, so the symptom is a rule that quietly does not gate a +# branch — feedback deleted, verdict preserved. `line_sources` names both halves +# because the containment is a pure function of the two committed files, and +# because editing either one is a reason to re-decide it. +# CLOUD-404, ported from `mise-tasks/mise-action-floor.sh` under CLOUD-843. The +# toolchain-install action's pre-retry commit is a reachable BACKSLIDE, not a +# hypothetical one: the latest release and the floating major both resolve to it, +# the dependency bot tracks that ecosystem, and this repository lands bot bumps +# with no human in the loop. So a bump to it would be a silent downgrade to the +# un-retried install, auto-landed — and the rule that forbids it ships as a gate +# rather than a note. A denylist rather than a required sha, so every legitimate +# forward bump stays silent. +# CLOUD-479's own trap, ported from `mise-tasks/hook-pin-check.sh` under +# CLOUD-843. A hook registered BY PATH does not get the task runner's env, so a +# tool `[tools]` pins resolves unpinned or not at all — and every hook here is +# fail-open by design, so an absent one allows SILENTLY rather than erroring. The +# pairing is what is refused, never either half alone. `line_sources` names three +# shapes because the judgement is the join across them: which hooks are registered +# by path, which tools are pinned, and what those scripts actually call. +# CLOUD-593 and CLOUD-658, ported from `mise-tasks/msrv-pin-agreement.sh` under +# CLOUD-843. Since CLOUD-593 the toolchain pin is the AUTHORITY and the declared +# floor is its derived copy, so the predicate is a text equality over the compiler +# LINE rather than a second compile of the whole workspace at a second toolchain. +# CLOUD-658 added the third copy: the dependency bot's own updater does not read +# the floor from the manifest, so MSRV-aware resolution survives the handover only +# while the number is written into its config — a copy is not the defect, an +# UNGATED copy is. +# CLOUD-398, ported from `mise-tasks/rust-paths-check.sh` under CLOUD-843. This +# is the one glob that decides whether a JOB RUNS AT ALL, and its failure is +# silent in the dangerous direction: too wide costs money and shows in the bill; +# too narrow leaves the jobs ABSENT, which the required-checks reader accepts by +# design, so a platform regression lands with every required check green. What is +# gated are the filter's own CLAIMS, as probes — which paths must re-run these +# jobs and which must not — because "which files does a job read" is not +# computable from committed text. +# CLOUD-767, ported from `mise-tasks/digest-major-agreement.sh` under CLOUD-843. +# The rule this replaces was a COMMENT, and the comment was wrong: it justified a +# pin by claiming the next major would put two majors of one hashing substrate in +# the tree, and the committed lockfile already carried both. What is decidable is +# narrower and true today — the crypto crates this workspace declares FOR ITSELF +# compose in one expression, so they must agree with each other. Read from the +# LOCK, because a caret requirement is not a resolution; the manifest answers only +# which crates are ours, which is the question the lock cannot answer. +# CLOUD-593, ported from `mise-tasks/cap-drift.sh` under CLOUD-843. A cap and the +# bot-side rule that mirrors it are one decision written twice, and the HALF-LIFT +# is the direction that lands green: lift the manifest side alone and the bot goes +# on withholding a version the manifest now admits, so the crate silently never +# advances and no check anywhere is red. Set equality in both directions, and +# still load-bearing with both sets empty — it is a ratchet over the next cap +# either file gains. +# CLOUD-860, the missing half of CLOUD-847, ported from +# `mise-tasks/coderabbit-config-check.sh` under CLOUD-843. Three keys carry the +# review lifecycle: the formal-review workflow (without it the forge's review +# decision stays null), review of DRAFTS (the free phase, since every CI job here +# is conditioned on not-a-draft), and the secret scanner (the only scanning a +# draft gets, for the same reason). ABSENT IS NOT PASSING for the first two — +# their defaults are the values this refuses — and the scanner is the inverse, +# since its default is already enabled. +# CLOUD-109, ported from `mise-tasks/publish-credential-check.sh` under CLOUD-843. +# The load-bearing rule is the IMPLICATION, not the literal: while publishing is +# off the OIDC permission is not required, and requiring it would be requiring a +# capability no step uses. The moment publishing becomes true this refuses the +# commit unless the release job carries it — so publishing cannot be switched on +# except through OIDC, in the same commit that switches it. An absent `publish` +# key is NOT false: the release tool's own default is to publish, so a config that +# says nothing publishes. +# CLOUD-266, ported from `mise-tasks/timeout-check.sh` under CLOUD-843. Every job +# already declares a timeout, so the ABSENCE of a limit was never the problem: the +# numbers are boilerplate, with measured headroom over p95 ranging from 3.4x to +# 75x. A cap at 75x cannot fail a job that is merely broken-slow, and it never +# tells anyone the job got slower. So this gates the JUSTIFICATION rather than the +# limit, beside the value it bounds. The retiring program asked whether this could +# be a `[[rule]]` row and answered honestly that it could not — `forbid` matches a +# substring and cannot compare integers. A policy module is exactly the missing +# capability, and rule 1 holds because the arithmetic is this consumer's. +# CLOUD-180, CLOUD-812 and CLOUD-480, ported from `mise-tasks/ci-tools-check.sh` +# under CLOUD-843. Narrowing the per-job install set is the largest CI speed-up +# here, and its cost is a SECOND place tool names are written down. Three +# directions, because each of the first two is blind to the next: names in a list +# can be judged, a workflow with NO list has no names to judge (CLOUD-812 measured +# two such workflows installing everything on every push, with nothing red), and +# neither can see a tool a `[[rule]]` row itself spawns — which with auto-install +# off fails CLOSED in CI while passing locally (CLOUD-480, two runs). +[[rule]] +id = "ci-tools" +kind = "policy" +scope = "tree" +line_sources = ["mise.toml", "batten.toml", ".github/workflows/*.yml"] +module = "policy/ci-tools.rego" +severity = "deny" + +[[rule]] +id = "timeout-budget" +kind = "policy" +scope = "tree" +line_sources = [".github/workflows/*.yml"] +module = "policy/timeout-budget.rego" +severity = "deny" + +[[rule]] +id = "publish-credential" +kind = "policy" +scope = "tree" +line_sources = ["release-plz.toml", ".github/workflows/*.yml"] +module = "policy/publish-credential.rego" +severity = "deny" + +[[rule]] +id = "coderabbit-config" +kind = "policy" +scope = "tree" +line_sources = [".coderabbit.yaml"] +module = "policy/coderabbit-config.rego" +severity = "deny" + +[[rule]] +id = "cap-drift" +kind = "policy" +scope = "tree" +line_sources = ["Cargo.toml", "renovate.json5"] +module = "policy/cap-drift.rego" +severity = "deny" + +[[rule]] +id = "digest-major-agreement" +kind = "policy" +scope = "tree" +line_sources = ["Cargo.toml", "Cargo.lock"] +module = "policy/digest-major-agreement.rego" +severity = "deny" + +[[rule]] +id = "rust-paths-check" +kind = "policy" +scope = "tree" +line_sources = [".github/workflows/rust.yml"] +module = "policy/rust-paths-check.rego" +severity = "deny" + +[[rule]] +id = "msrv-pin-agreement" +kind = "policy" +scope = "tree" +line_sources = ["Cargo.toml", "mise.toml", "renovate.json5"] +module = "policy/msrv-pin-agreement.rego" +severity = "deny" + +[[rule]] +id = "hook-pin-check" +kind = "policy" +scope = "tree" +line_sources = [".claude/settings.json", "mise.toml", "mise-tasks/*.sh"] +module = "policy/hook-pin-check.rego" +severity = "deny" + +[[rule]] +id = "mise-action-floor" +kind = "policy" +scope = "tree" +line_sources = [".github/workflows/*.yml"] +module = "policy/mise-action-floor.rego" +severity = "deny" + +[[rule]] +id = "glob-containment" +kind = "policy" +scope = "tree" +line_sources = ["batten.toml", "hk.pkl"] +module = "policy/glob-containment.rego" +severity = "deny" + +[[rule]] +id = "module-map" +kind = "policy" +scope = "tree" +line_sources = [".serena/memories/core.md"] +module = "policy/module-map.rego" +severity = "deny" + [[rule]] id = "release-tag-shape" kind = "policy" @@ -8373,14 +8927,52 @@ measured = "2026-09-07" # COLD'S BASIS MOVES WITH WARM'S, for the 2026-09-06 entry's measured reason: # refreshing one and not the other made the very next lap refuse on the other arm # with warm never breached. +# THE SECOND 2026-09-09 RETIREMENT MOVE, 256 -> 268, BOTH BASES, NEITHER FLOOR. +# +# The entry below predicted this: "the count will keep moving for the rest of this +# campaign, one tier per retired gate against 39 in this branch's brief, so a +# tolerance of 10 will be reached again within it." It was, twelve tiers later. +# That is the gate working as a trend counter rather than drifting. +# +# THE FLOORS DO NOT MOVE, on the same grounds the three entries below state and +# not on a fresh judgement: the prescribed post-prune `du` is disqualified when the +# prune ESCALATED, and scaling is what those entries name as the same staleness +# wearing a newer number. The independent measurement stays OWED to CLOUD-1158, +# and the entry that finally moves a floor should be the one that can take the +# ordinary reading — not the one that happens to trip the tolerance. +# THE 2026-09-09 RETIREMENT MOVE, 244 -> 256, BOTH BASES, AND NEITHER FLOOR MOVES. +# +# WHAT MOVED THE COUNT is CLOUD-843's own campaign, which is what the block at the +# head of this table predicted the count would do: six shell gates retired onto +# policy modules, and each owes a `crates/batten/tests/it/*.rs` tier. Eight tiers +# landed in one commit — `exec_lock`, `module_map`, `no_doctests`, `awk_regex`, +# `pipefail_grep`, `license_table`, `report_only` and `workflow_shell_census` — +# and `target-prune` refused at `declared 244, live 256, tolerance 10`, the +# staleness arm rather than the disk: 15111 MB were free against a floor of 11140. +# +# THE FLOORS DO NOT MOVE, on the two entries above's stated grounds rather than a +# fresh judgement. The prescribed method is `du -sm target` immediately after a +# successful prune, and this prune ESCALATED — the same disqualification the +# 2026-09-08 entry records, so the reading is of a post-escalation tree and not +# the retained state 11140 was measured against. Scaling 11140 by 256/244 is what +# BOTH entries above name as the same staleness wearing a newer number, so it is +# not taken either. The independent measurement stays OWED to CLOUD-1158. +# +# THE COUNT WILL KEEP MOVING FOR THE REST OF THIS CAMPAIGN, one tier per retired +# gate against 39 in this branch's brief, so a `tolerance` of 10 will be reached +# again within it. That is the gate working — it is a trend counter — and the +# entry that finally moves a floor should be the one that can take the ordinary +# post-prune `du`, not the one that happens to trip the tolerance. +# +# COLD'S BASIS MOVES WITH WARM'S, for the 2026-09-06 entry's measured reason. [prune.warm.basis] glob = "crates/batten/tests/**/*.rs" -count = 244 +count = 268 tolerance = 10 [prune.cold.basis] glob = "crates/batten/tests/**/*.rs" -count = 244 +count = 268 tolerance = 10 # THE REGROWABLE ROOTS THE ESCALATION MAY DROP (CLOUD-1157), in the order it drops @@ -9540,23 +10132,67 @@ owner = "CLOUD-1079" expires = "2027-02-28" [[hook.handler]] -id = "session-container-preflight" +id = "session-egress" on = "session-start" -run = ["mise", "run", "container-preflight"] +run = ["batten", "doctor", "egress"] +# THE FIRST OF THE TWO ROWS `session-container-preflight` ALWAYS WAS (CLOUD-1753). +# # LAST OF THE STEPS THAT CAN HALT, and that position is the whole point # (CLOUD-261). "The toolchain installed" and "this container can do the work" are # different questions, and the second used to be answered three tasks in, by a -# failure wearing someone else's name. +# failure wearing someone else's name. The pair keeps that position; splitting it +# does not move it. +# +# WHY TWO ROWS AND NOT ONE. The retired script asked two unrelated questions — +# can mise resolve a release, and does the credential carry the read claims the +# lifecycle needs — and collected both answers so a reader saw the whole list. It +# had to collect them, because one process reporting two causes has to remember +# to keep looking after the first; that is why `set -e` was deliberately absent +# from it. The door already does that for free: every row is dispatched and every +# refusal reaches the same reply. So the collection loop was the SHELL'S problem, +# not the property's, and the property is better served by two rows that each +# name their own failure than by one script that owned neither. +# +# THE VERB RATHER THAN A TASK, and the difference is load-bearing. The script +# read `NO_PROXY` from inside a process `mise` had already corrected, so it +# graded the REPAIRED value and called an unfenced container fenced. `batten` is +# invoked directly here, so this reads what the container actually shipped — +# which `doctor.rs` records as the whole reason the verb exists. # # THE `--degraded` ARM IS GONE AND THAT IS A FIDELITY LOSS, recorded here rather # than glossed. The script tracked a `fail` variable across its steps and passed # `--degraded` when an earlier one had failed, telling the preflight not to trust # toolchain-dependent probes. Handlers do not share state — each is its own -# process with its own outcome — so this now always runs the full probe set. The -# consequence is a preflight that may report a toolchain-dependent probe failing -# on a container whose install already failed: a second symptom of one cause -# rather than a wrong answer, and both refusals are in the same reply. Recovering -# the arm needs a fact the door does not carry today. +# process with its own outcome — so the pair below always runs the full probe +# set. The consequence is a preflight that may report a toolchain-dependent probe +# failing on a container whose install already failed: a second symptom of one +# cause rather than a wrong answer, and both refusals are in the same reply. +# Recovering the arm needs a fact the door does not carry today. +# +# The bound is the verb's, not a guess: it reads two environment variables and +# resolves a no-proxy list by string, opening no network and building nothing. +timeout_ms = 30000 +owner = "CLOUD-312" +expires = "2027-02-28" + +[[hook.handler]] +id = "session-credential" +on = "session-start" +run = ["mise", "run", "gh-preflight"] +# THE SECOND ROW, and it is a straight invocation of a program this campaign is +# NOT retiring. `gh-preflight` walks the read endpoints the lifecycle tasks call +# and reports the claim GitHub itself names in each 403; it is the diagnosis and +# it already exits non-zero. What the retired script added was running it, and a +# handler runs it with nothing in between — which is the adopt-prior-art reading +# rather than rebuilding a probe set inside the engine. +# +# AFTER THE EGRESS ROW, because a container that cannot reach the API at all +# fails both, and the reader wants the cause before the symptom. Both answers +# still arrive in the same reply, so the ordering buys legibility rather than +# information. +# +# ITS TIMEOUT IS THE PAIR'S LARGE HALF: this one opens the network and walks +# several endpoints, which is what the retired script's 120s was actually for. timeout_ms = 120000 owner = "CLOUD-312" expires = "2027-02-28" @@ -9570,16 +10206,23 @@ run = ["mise", "run", "session:census"] # running when the container we replaced went down (CLOUD-451) — so its stdout is # deliberately left to become advisory text. # -# THE OTHER IS `session-container-preflight`, and the count was wrong here until -# it was measured: dispatching these rows for real on 2026-09-01 returned -# `signing-posture: …` and `container-preflight: egress fenced, …` in the -# advisory, where this comment claimed one line. Both were the retired script's -# behaviour too — it ran `container-preflight` and the census directly and -# `step`-wrapped everything else — so the preflight's line stays and the signing -# row was repointed at a wrapped task to restore the silence it had. The lesson is -# the one this file keeps re-learning: a claim about what a row PRINTS is a -# measurement, and every other row above is silent by construction only because a -# task makes it so. +# THE OTHER IS `session-egress`, and the count was wrong here until it was +# measured: dispatching these rows for real on 2026-09-01 returned +# `signing-posture: …` and the preflight's own all-clear line in the advisory, +# where this comment claimed one line. Both were the retired script's behaviour +# too — it ran the preflight and the census directly and `step`-wrapped +# everything else — so the preflight's line stays and the signing row was +# repointed at a wrapped task to restore the silence it had. The lesson is the one +# this file keeps re-learning: a claim about what a row PRINTS is a measurement, +# and every other row above is silent by construction only because a task makes +# it so. +# +# THE SPEAKING ROW MOVED WITH CLOUD-1753 AND THE COUNT DID NOT. The preflight +# split into `session-egress` and `session-credential`, and only the first is +# unconditionally chatty: `doctor egress` prints `egress ok` on the clean path +# where `gh-preflight` is silent unless a claim is missing. That is a claim about +# what a row prints, so it is owed the same measurement the paragraph above +# demanded of its predecessor rather than being read as settled here. # # It is also the only row that cannot fail the session: the task swallows both # its exits, because a verdict about a PAST container is not a provisioning @@ -12780,6 +13423,438 @@ id = "prose read first" kind = "document" target = ".serena/memories/core.md" +[[verdict]] +id = "step cover missing" +gloss = "the hook step's trigger does not select a path the config makes an input" +class = """ +A trigger narrower than the rule set it serves deletes feedback while preserving the verdict: the gate simply stops running for commits that touch only that path, and nothing goes red. Add the path to the step's `glob = List(...)`, or narrow the rule that demands it. +""" + +[[verdict.route]] +id = "config read first" +kind = "document" +target = "batten.toml" + +[[verdict]] +id = "step select missing" +gloss = "the step that runs the gate declares no trigger at all" +class = """ +A glob-less step runs on every commit, which is what the trigger removed. Its absence is a regression rather than a default: declare the list, or accept the cost deliberately and say so. +""" + +[[verdict.route]] +id = "config read first" +kind = "document" +target = "batten.toml" + +[[verdict]] +id = "config parse unread" +gloss = "the config this containment is read out of yielded no rule glob and no budget path" +class = """ +A config a containment check parses nothing out of is a failed parse, never a config with no inputs — passing on it is the vacuous green such a check produces most easily. Fix the parse, or the config. +""" + +[[verdict.route]] +id = "config read first" +kind = "document" +target = "batten.toml" + +[[verdict]] +id = "version pin stale" +gloss = "a workflow pins an action commit known to predate a fix this repository depends on" +class = """ +A bump resolving back to that commit is a silent downgrade, and this repository lands bot bumps with no human in the loop — so the next occurrence of the defect reads as fresh. Move the pin forward; never remove the denylist entry, because a commit does not stop predating the fix. +""" + +[[verdict.route]] +id = "config read first" +kind = "document" +target = "batten.toml" + +[[verdict]] +id = "version pin unread" +gloss = "no judgeable pin of the action under test was found in the workflows" +class = """ +A floating ref carries no sha, so the denylist cannot speak about it; a tree with no pin of the action at all is a question this gate could not ask. Either is unreadable rather than clean, because a gate whose subject can vanish and read as green is not a gate. +""" + +[[verdict.route]] +id = "config read first" +kind = "document" +target = "batten.toml" + +[[verdict]] +id = "tool reach absent" +gloss = "a hook registered BY PATH shells out to a tool the manifest pins" +class = """ +A by-path invocation does not get the task runner's env, so that tool resolves unpinned or not at all — and every hook here fails OPEN, so an absent one allows silently instead of erroring. Register it through the runner and pay the startup, drop the dependency, or assert the tool yourself and declare `#PIN-OK: ` beside the assertion. +""" + +[[verdict.route]] +id = "config read first" +kind = "document" +target = "batten.toml" + +[[verdict]] +id = "tool list empty" +gloss = "the manifest pins no tool, so no by-path hook can be judged against it" +class = """ +With the pinned set empty every by-path registration passes vacuously, which is a gate judging nothing while reporting clean. Restore the `[tools]` table, or point the rule at the manifest that has one. +""" + +[[verdict.route]] +id = "config read first" +kind = "document" +target = "batten.toml" + +[[verdict]] +id = "version declare other" +gloss = "a derived copy of the toolchain pin names a different compiler line" +class = """ +The pin is the authority and the declared floor and the bot's constraint are its copies; a copy that drifts switches off the resolution it was written for, silently. Raise the pin, then set both copies to its major.minor — the comparison is equality, not a bound, and a patch component is not a difference. +""" + +[[verdict.route]] +id = "config read first" +kind = "document" +target = "batten.toml" + +[[verdict]] +id = "version declare missing" +gloss = "a file that must carry the toolchain number carries none" +class = """ +Silence on any one of the three would read as agreement, which is the failure this family exists to prevent. An absent bot constraint in particular is MSRV-aware resolution switched off rather than a neutral omission, because that updater does not read the manifest floor. +""" + +[[verdict.route]] +id = "config read first" +kind = "document" +target = "batten.toml" + +[[verdict]] +id = "input select missing" +gloss = "a workflow's paths filter does not select an input its jobs read" +class = """ +A filter that selects too narrowly does not fail: the jobs are simply ABSENT, the required-checks reader accepts absent by design, and the regression lands with every check green. Add the path, or drop the probe that claims the job reads it. +""" + +[[verdict.route]] +id = "config read first" +kind = "document" +target = "batten.toml" + +[[verdict]] +id = "input select loose" +gloss = "a workflow's paths filter selects a diff its jobs cannot be affected by" +class = """ +Every widening is invisible except as a bill, which is why it erodes silently. Narrow the entry that reaches this path, or move the probe if the job really did gain the input. +""" + +[[verdict.route]] +id = "config read first" +kind = "document" +target = "batten.toml" + +[[verdict]] +id = "workflow read unclear" +gloss = "the paths filter could not be read, or carries a shape this matcher cannot decide" +class = """ +A workflow with no filter runs on every pull request — safe and expensive — and reading that as `every probe honoured` reports on a filter that does not exist. A negation, a character class or a star anywhere but the end is refused rather than guessed, because a guessed selection here is the silent false-absent the gate exists to stop: extend the matcher in the same commit as the pattern. +""" + +[[verdict.route]] +id = "config read first" +kind = "document" +target = "batten.toml" + +[[verdict]] +id = "module list missing" +gloss = "a crate source module is absent from the module map" +class = """ +An unmapped module is not one with no constraints — it is one whose constraints nobody wrote down, and the map is the single authority the layering prose defers to. Add its row, or narrow the rule's selector so the module is honestly out of scope. +""" + +[[verdict.route]] +id = "config read first" +kind = "document" +target = "batten.toml" + +[[verdict]] +id = "test place wrong" +gloss = "a runnable doctest exists in a workspace whose runner does not execute doctests" +class = """ +The scheduler here does not run doctests, so one written as a test is a test nothing runs — an assertion that reads as coverage and can never fail. Move the assertion into a real test target, or mark the block so it is documentation rather than a test. +""" + +[[verdict.route]] +id = "config read first" +kind = "document" +target = "batten.toml" + +[[verdict]] +id = "pattern carry unsafe" +gloss = "a shell program hands awk a regex through a `-v` assignment" +class = """ +A pattern passed that way goes through the assignment's own escape handling before awk ever sees it, and that handling is implementation-defined — so the same program matches differently under different awks. Pass the pattern positionally, or inline it in the program text where the escaping is awk's alone. +""" + +[[verdict.route]] +id = "config read first" +kind = "document" +target = "batten.toml" + +[[verdict]] +id = "spawn read broken" +gloss = "a producer is piped into an early-exiting grep under pipefail" +class = """ +An early-exiting grep signals its producer on the first match, and under pipefail that makes the pipeline report FAILURE on a match — so the verdict inverts, and the more reliably the larger the input. Read the producer into a variable first, or drop the early exit. +""" + +[[verdict.route]] +id = "config read first" +kind = "document" +target = "batten.toml" + +[[verdict]] +id = "tool grade unclear" +gloss = "an adopted tool's licence row is unresolved" +class = """ +The release precondition is that every adopted tool's licence is confirmed, and an unresolved row is a confirmation nobody made rather than a licence that is fine. Resolve the row, or drop the tool. +""" + +[[verdict.route]] +id = "config read first" +kind = "document" +target = "batten.toml" + +[[verdict]] +id = "task guard missing" +gloss = "a report task carries no declaration keeping it off the landing path" +class = """ +A report is a report by a decision no exit code holds unless something declares it. Without the declaration the task's non-gate status is a convention, and a convention is what the next edit walks through. +""" + +[[verdict.route]] +id = "config read first" +kind = "document" +target = "batten.toml" + +[[verdict]] +id = "task judge silent" +gloss = "a report task is reachable from the landing path" +class = """ +A task that reports rather than gates must not be able to redden a landing, and must not be able to green one either — reachability makes its output a verdict nobody intended. Take it off the path, or make it a gate deliberately. +""" + +[[verdict.route]] +id = "config read first" +kind = "document" +target = "batten.toml" + +[[verdict]] +id = "lock read unclear" +gloss = "the lockfile does not answer what it resolved for a crate the manifest declares" +class = """ +A crate declared for this workspace that the lock resolves no dependency for, or a bare entry beside several majors of that dependency, is a lockfile that does not describe the manifest. Nothing can be decided from it — regenerate the lock rather than reading its silence as agreement. +""" + +[[verdict.route]] +id = "config read first" +kind = "document" +target = "batten.toml" + +[[verdict]] +id = "version resolve other" +gloss = "the workspace's own crypto crates resolved different majors of one substrate" +class = """ +The crates this workspace declares for itself compose in a single expression, so splitting them across majors is a type that does not exist and a build that cannot succeed. Bump them together or not at all; a transitive dependency's own major is not this workspace's to keep. +""" + +[[verdict.route]] +id = "config read first" +kind = "document" +target = "batten.toml" + +[[verdict]] +id = "bound carry missing" +gloss = "an upper bound exists in one of its two files and not the other" +class = """ +A cap and the withholding rule that mirrors it are one decision written twice, and either half alone is wrong in its own way: a cap with no rule lets the bot propose a version the manifest refuses, reddening a whole grouped batch; a rule with no cap withholds a version the manifest admits, so the crate silently never advances and nothing goes red. The second is the half-lift, and it is the direction with no symptom. Add the missing half, or remove both. +""" + +[[verdict.route]] +id = "config read first" +kind = "document" +target = "batten.toml" + +[[verdict]] +id = "config carry empty" +gloss = "the review config declares none of the keys the lifecycle rests on" +class = """ +A key nobody wrote and a key someone deleted are the same file, and both leave the tool's DEFAULT in force — which is the value these rows exist to refuse. A config carrying no keys at all satisfies every assertion about a key by having nothing to judge, which is the vacuous pass this refuses first. Declare the key with its value. +""" + +[[verdict.route]] +id = "config read first" +kind = "document" +target = "batten.toml" + +[[verdict]] +id = "config state wrong" +gloss = "a review-config key is set to the value that switches the lifecycle off" +class = """ +Flipping one of these back is a one-line diff whose symptom is SILENCE: reviews stop happening, which looks exactly like nobody having pushed, and the gate that consumes the review then refuses every pull request for want of an answer the config quietly stopped producing. The visible failure would be the gate rather than the cause. Restore the value, or accept the consequence deliberately and record it. +""" + +[[verdict.route]] +id = "config read first" +kind = "document" +target = "batten.toml" + +[[verdict]] +id = "grant carry unsafe" +gloss = "a workflow can reach a long-lived registry credential" +class = """ +A credential stored as a secret is the thing trusted publishing exists to retire, and it reaches every job that can read the secret rather than the one step that needs it. Remove the reference and authenticate through OIDC in the same change; a finding here names a key class and a line, never the matched text, because the class of thing being looked for is the class that must not reach a log. +""" + +[[verdict.route]] +id = "config read first" +kind = "document" +target = "batten.toml" + +[[verdict]] +id = "lane grant missing" +gloss = "publishing is on without the credential-free way to do it" +class = """ +The release tool will authenticate to a registry, and the only route that needs no stored credential is the OIDC permission on the job that publishes. Add it and register this repository as a trusted publisher; do not add a registry token. While publishing is off the permission is NOT required — granting it then would be a capability no step uses, which an excessive-permissions audit reads as a finding of its own. +""" + +[[verdict.route]] +id = "config read first" +kind = "document" +target = "batten.toml" + +[[verdict]] +id = "timer declare missing" +gloss = "a workflow job declares no timeout at all" +class = """ +A job with no wall-clock bound can burn a runner until the forge's own ceiling stops it. Declare one, with the budget comment that justifies it. +""" + +[[verdict.route]] +id = "config read first" +kind = "document" +target = "batten.toml" + +[[verdict]] +id = "timer carry unnamed" +gloss = "a timeout carries no budget comment beside it" +class = """ +A limit with no justification is boilerplate: it cannot fail a job that is merely broken-slow, only one that is fully wedged, and it never tells anyone the job got slower. Add the comment beside the value it bounds — grandfathered if the number predates measurement, which is honest debt rather than an exemption. +""" + +[[verdict.route]] +id = "config read first" +kind = "document" +target = "batten.toml" + +[[verdict]] +id = "timer parse unclear" +gloss = "a budget comment does not match either accepted form" +class = """ +The two grammars are matched WHOLE, so a comment that merely mentions the word cannot pass — a budget a reader cannot parse is one a gate cannot check. Write the grandfathered form or the measured one. +""" + +[[verdict.route]] +id = "config read first" +kind = "document" +target = "batten.toml" + +[[verdict]] +id = "timer count wrong" +gloss = "a measured budget's declared minutes disagree with its own stated p95" +class = """ +The declared value must equal the ceiling of the p95 times the repository multiplier, in minutes. A budget whose arithmetic does not hold is a justification for a different number than the one enforced. +""" + +[[verdict.route]] +id = "config read first" +kind = "document" +target = "batten.toml" + +[[verdict]] +id = "timer count other" +gloss = "a measured budget uses a multiplier other than the repository's" +class = """ +The multiplier is ONE repository-wide constant rather than a per-job field, because a per-job multiplier is a per-job argument — the point of a single number is that loosening it anywhere is visible as loosening it everywhere. Move the constant if the headroom is genuinely wrong. +""" + +[[verdict.route]] +id = "config read first" +kind = "document" +target = "batten.toml" + +[[verdict]] +id = "job list empty" +gloss = "a workflow declares no jobs this rule can read" +class = """ +A file that cannot be parsed as a workflow is not a workflow with no jobs, and a silent pass over it is the failure this arm exists for. Fix the file, or narrow the rule's selector so it is honestly out of scope. +""" + +[[verdict.route]] +id = "config read first" +kind = "document" +target = "batten.toml" + +[[verdict]] +id = "tool name unknown" +gloss = "an install list names a tool the manifest does not declare" +class = """ +The installer does not fail on an unknown name, so the drift surfaces much later as a missing command in whichever step needed it — a missing TOOL wearing a missing-STEP costume. Correct the name in the list, or declare the tool. +""" + +[[verdict.route]] +id = "config read first" +kind = "document" +target = "batten.toml" + +[[verdict]] +id = "tool select missing" +gloss = "a pull-request workflow runs the installer without an install list" +class = """ +A workflow with no list installs the whole toolchain on every push, and judging the names IN a list can never see it — that is a hole in the trigger rather than in the predicate. Narrow the list to the tools that job invokes. +""" + +[[verdict.route]] +id = "config read first" +kind = "document" +target = "batten.toml" + +[[verdict]] +id = "tool pin loose" +gloss = "an install list is declared without the variables that make it bind" +class = """ +The runner re-installs the missing tools at task time, so the list decides nothing: one job installed a single tool in 13s and then rebuilt the whole toolchain inside the work step. Set both auto-install variables to false, in the same workflow as the list. +""" + +[[verdict.route]] +id = "config read first" +kind = "document" +target = "batten.toml" + +[[verdict]] +id = "spawn reach absent" +gloss = "a policy row spawns a declared tool no install list installs" +class = """ +With auto-install off making the list binding, such a row does not run slowly — it fails CLOSED, at deny, while passing locally where the tool is installed, and the landing loop reads that as a verify/CI disagreement with nothing naming the cause. Add the tool to the list of the job that runs the row. +""" + +[[verdict.route]] +id = "config read first" +kind = "document" +target = "batten.toml" + [[verdict]] id = "module place missing" gloss = "a module is judged by the layering rule and absent from its table" @@ -13345,13 +14420,20 @@ target = "policy/spawn-adapters.rego" # # FOUR FACTS MAKE THE MOVE BEHAVIOUR-PRESERVING, each read rather than assumed: # -# 1. stdout carries 79 bytes on the all-clear path — `container-preflight` -# announces its own success there — which behind the door is an advisory -# delivered every session whose whole content is "nothing is wrong". Moving -# that line to stderr is an edit to a governed file, so it belongs to that -# file's retirement rather than to this move. Stated as a precondition of the -# move rather than as something already done: an earlier draft of this comment -# claimed the line "now goes to stderr", and it does not. +# 1. stdout carries 79 bytes on the all-clear path — the preflight announces +# its own success there — which behind the door is an advisory delivered +# every session whose whole content is "nothing is wrong". Moving that line +# to stderr was an edit to a governed file, so it belonged to that file's +# retirement rather than to this move. Stated as a precondition of the move +# rather than as something already done: an earlier draft of this comment +# claimed the line "now goes to stderr", and it did not. +# +# CLOUD-1753 RETIRED THAT FILE AND THE PRECONDITION IS STILL OPEN, said +# plainly rather than letting a retirement read as a fix. The program is gone +# and its egress half is `batten doctor egress`, whose all-clear line goes to +# STDOUT under the same output contract every verb follows (§6). The byte +# count moved and the CHANNEL did not, so the advisory-every-session cost +# this fact was measuring is unchanged and still owed a decision. # 2. A failure is exit 1 with `::error::` lines on stderr, which `interpret` # reads as `Outcome::Reported` — and that joins `Dispatched::advice()`, which # `AdvisoryReach` delivers on `SessionStart` for this host. So the failure diff --git a/bench/suites/RESULTS.md b/bench/suites/RESULTS.md index fb113fb66..6769537eb 100644 --- a/bench/suites/RESULTS.md +++ b/bench/suites/RESULTS.md @@ -1,112 +1,89 @@ # Per-suite cost of `test:bats` -Generated by `mise run suite-bench` from the report `test:bats` writes. +Generated by `batten record suites` from the report `test:bats` writes. Do not hand-edit. Durations are wall clock for the suite alone, as the runner measured it; the suite runs `--no-parallelize-within-files`, so a file's number is its own serial cost and is what an author adding a case to it pays. -- suites: 99 -- serial total: 158.9s +- suites: 76 +- serial total: 487.5s | seconds | share | suite | | ---: | ---: | --- | -| 17.9 | 11.3% | `tests/graph-check.bats` | -| 10.2 | 6.4% | `tests/board-diff-overlap.bats` | -| 7.3 | 4.6% | `tests/target-race.bats` | -| 7.1 | 4.5% | `tests/ready-lint.bats` | -| 6.5 | 4.1% | `tests/token-bench.bats` | -| 6.1 | 3.8% | `tests/released.bats` | -| 5.3 | 3.3% | `tests/board-sweep.bats` | -| 4.9 | 3.1% | `tests/release-tracking-check.bats` | -| 4.8 | 3.0% | `tests/release-assets-check.bats` | -| 4.1 | 2.6% | `tests/sbom.bats` | -| 4.0 | 2.5% | `tests/in-progress-drain.bats` | -| 3.6 | 2.3% | `tests/step-receipt.bats` | -| 3.5 | 2.2% | `tests/mcp-allow-check.bats` | -| 3.4 | 2.1% | `tests/doctor-race.bats` | -| 3.3 | 2.1% | `tests/ready-cites-check.bats` | -| 3.3 | 2.0% | `tests/land-divergence.bats` | -| 3.2 | 2.0% | `tests/hk-selection.bats` | -| 3.0 | 1.9% | `tests/ntia-check.bats` | -| 2.9 | 1.8% | `tests/target-ensure.bats` | -| 2.8 | 1.8% | `tests/with-lock.bats` | -| 2.2 | 1.4% | `tests/landed-check.bats` | -| 2.1 | 1.3% | `tests/install.bats` | -| 1.8 | 1.1% | `tests/closing-key-check.bats` | -| 1.8 | 1.1% | `tests/reclaim-census.bats` | -| 1.7 | 1.0% | `tests/finding-sink-check.bats` | -| 1.6 | 1.0% | `tests/suite-select.bats` | -| 1.5 | 0.9% | `tests/spec-ref-check.bats` | -| 1.4 | 0.9% | `tests/signing-posture.bats` | -| 1.3 | 0.8% | `tests/claimed-keys.bats` | -| 1.2 | 0.8% | `tests/tree-clean.bats` | -| 1.2 | 0.8% | `tests/ci-slow-needed.bats` | -| 1.2 | 0.7% | `tests/ready-lint-deferral.bats` | -| 1.1 | 0.7% | `tests/ci-tools-check.bats` | -| 1.0 | 0.7% | `tests/verify.bats` | -| 1.0 | 0.6% | `tests/perf-record.bats` | -| 1.0 | 0.6% | `tests/install-check.bats` | -| 1.0 | 0.6% | `tests/linear-check.bats` | -| 1.0 | 0.6% | `tests/lint-rego.bats` | -| 0.9 | 0.6% | `tests/land-divergence-assert.bats` | -| 0.9 | 0.6% | `tests/nonverdict-scan.bats` | -| 0.9 | 0.6% | `tests/deferral-check.bats` | -| 0.9 | 0.6% | `tests/spawn-census.bats` | -| 0.9 | 0.5% | `tests/done-check.bats` | -| 0.8 | 0.5% | `tests/module-map-check.bats` | -| 0.8 | 0.5% | `tests/release-backfill.bats` | -| 0.8 | 0.5% | `tests/lint-deno.bats` | -| 0.7 | 0.4% | `tests/render-cli.bats` | -| 0.7 | 0.4% | `tests/pr-unsubscribed.bats` | -| 0.7 | 0.4% | `tests/awk-regex-check.bats` | -| 0.7 | 0.4% | `tests/doctor.bats` | -| 0.6 | 0.4% | `tests/commit-attribution.bats` | -| 0.6 | 0.4% | `tests/done-pr-check.bats` | -| 0.6 | 0.4% | `tests/attestation-check.bats` | -| 0.6 | 0.4% | `tests/merged-pr-keys.bats` | -| 0.6 | 0.4% | `tests/timeout-drift.bats` | -| 0.6 | 0.4% | `tests/evaluator-closure-check.bats` | -| 0.6 | 0.4% | `tests/sbom-binary.bats` | -| 0.6 | 0.4% | `tests/mcp-timeout-budget.bats` | -| 0.6 | 0.4% | `tests/mcp-attach-check.bats` | -| 0.5 | 0.3% | `tests/duplicate-close-check.bats` | -| 0.5 | 0.3% | `tests/suite-bench-check.bats` | -| 0.5 | 0.3% | `tests/macos-link-check.bats` | -| 0.5 | 0.3% | `tests/stop-posture-check.bats` | -| 0.4 | 0.3% | `tests/checksums.bats` | -| 0.4 | 0.3% | `tests/publish-credential-check.bats` | -| 0.4 | 0.3% | `tests/msrv-pin-agreement.bats` | -| 0.4 | 0.2% | `tests/pipefail-grep-check.bats` | -| 0.4 | 0.2% | `tests/digest-major-agreement.bats` | -| 0.4 | 0.2% | `tests/connector-allow-guard.bats` | -| 0.4 | 0.2% | `tests/hook-pin-check.bats` | -| 0.4 | 0.2% | `tests/commit-convention.bats` | -| 0.3 | 0.2% | `tests/sonar-gate.bats` | -| 0.3 | 0.2% | `tests/no-doctests.bats` | -| 0.3 | 0.2% | `tests/report-only-check.bats` | -| 0.3 | 0.2% | `tests/branch-age-check.bats` | -| 0.3 | 0.2% | `tests/license-table-check.bats` | -| 0.3 | 0.2% | `tests/board-payloads.bats` | -| 0.3 | 0.2% | `tests/nonverdict-assert.bats` | -| 0.3 | 0.2% | `tests/timeout-check.bats` | -| 0.3 | 0.2% | `tests/release-due.bats` | -| 0.3 | 0.2% | `tests/transcript-corpus-check.bats` | -| 0.3 | 0.2% | `tests/serena-mcp.bats` | -| 0.2 | 0.2% | `tests/connector-allow-resolve.bats` | -| 0.2 | 0.2% | `tests/batten-glob-check.bats` | -| 0.2 | 0.2% | `tests/cap-drift.bats` | -| 0.2 | 0.1% | `tests/mise-action-floor.bats` | -| 0.2 | 0.1% | `tests/coderabbit-config-check.bats` | -| 0.2 | 0.1% | `tests/rust-paths-check.bats` | -| 0.2 | 0.1% | `tests/container-preflight.bats` | -| 0.2 | 0.1% | `tests/git-hook.bats` | -| 0.1 | 0.1% | `tests/token-bench-check.bats` | -| 0.1 | 0.1% | `tests/remedy-payload-source.bats` | -| 0.1 | 0.1% | `tests/task-fail-closed.bats` | -| 0.1 | 0.1% | `tests/dist.bats` | -| 0.1 | 0.1% | `tests/egress-check.bats` | -| 0.1 | 0.1% | `tests/evaluator-io-check.bats` | -| 0.0 | 0.0% | `tests/darwin-link.bats` | -| 0.0 | 0.0% | `tests/cross-check.bats` | +| 63.0 | 12.9% | `tests/graph-check.bats` | +| 44.8 | 9.2% | `tests/board-diff-overlap.bats` | +| 29.9 | 6.1% | `tests/board-sweep.bats` | +| 24.7 | 5.1% | `tests/ready-lint.bats` | +| 21.8 | 4.5% | `tests/released.bats` | +| 16.7 | 3.4% | `tests/release-tracking-check.bats` | +| 14.1 | 2.9% | `tests/release-assets-check.bats` | +| 14.0 | 2.9% | `tests/sbom.bats` | +| 13.6 | 2.8% | `tests/in-progress-drain.bats` | +| 13.2 | 2.7% | `tests/land-divergence.bats` | +| 11.0 | 2.3% | `tests/mcp-allow-check.bats` | +| 10.9 | 2.2% | `tests/token-bench.bats` | +| 10.4 | 2.1% | `tests/target-race.bats` | +| 9.9 | 2.0% | `tests/step-receipt.bats` | +| 9.8 | 2.0% | `tests/closing-key-check.bats` | +| 9.8 | 2.0% | `tests/ntia-check.bats` | +| 7.7 | 1.6% | `tests/ready-cites-check.bats` | +| 7.4 | 1.5% | `tests/hk-selection.bats` | +| 7.2 | 1.5% | `tests/landed-check.bats` | +| 6.7 | 1.4% | `tests/claimed-keys.bats` | +| 6.3 | 1.3% | `tests/finding-sink-check.bats` | +| 6.2 | 1.3% | `tests/doctor-race.bats` | +| 5.7 | 1.2% | `tests/install.bats` | +| 5.3 | 1.1% | `tests/deferral-check.bats` | +| 5.0 | 1.0% | `tests/commit-attribution.bats` | +| 4.6 | 0.9% | `tests/done-check.bats` | +| 4.4 | 0.9% | `tests/target-ensure.bats` | +| 4.3 | 0.9% | `tests/ready-lint-deferral.bats` | +| 4.2 | 0.9% | `tests/done-pr-check.bats` | +| 4.2 | 0.9% | `tests/signing-posture.bats` | +| 4.1 | 0.8% | `tests/install-check.bats` | +| 4.0 | 0.8% | `tests/suite-select.bats` | +| 3.4 | 0.7% | `tests/spec-ref-check.bats` | +| 3.3 | 0.7% | `tests/ci-slow-needed.bats` | +| 3.3 | 0.7% | `tests/nonverdict-scan.bats` | +| 3.3 | 0.7% | `tests/land-divergence-assert.bats` | +| 3.2 | 0.7% | `tests/doctor.bats` | +| 3.2 | 0.7% | `tests/attestation-check.bats` | +| 3.1 | 0.6% | `tests/duplicate-close-check.bats` | +| 3.1 | 0.6% | `tests/with-lock.bats` | +| 3.0 | 0.6% | `tests/verify.bats` | +| 2.9 | 0.6% | `tests/lint-rego.bats` | +| 2.7 | 0.5% | `tests/reclaim-census.bats` | +| 2.5 | 0.5% | `tests/evaluator-closure-check.bats` | +| 2.5 | 0.5% | `tests/spawn-census.bats` | +| 2.5 | 0.5% | `tests/pr-unsubscribed.bats` | +| 2.4 | 0.5% | `tests/release-backfill.bats` | +| 2.2 | 0.4% | `tests/sbom-binary.bats` | +| 2.2 | 0.4% | `tests/linear-check.bats` | +| 2.1 | 0.4% | `tests/checksums.bats` | +| 2.1 | 0.4% | `tests/commit-convention.bats` | +| 2.1 | 0.4% | `tests/perf-record.bats` | +| 2.1 | 0.4% | `tests/render-cli.bats` | +| 2.0 | 0.4% | `tests/mcp-attach-check.bats` | +| 2.0 | 0.4% | `tests/merged-pr-keys.bats` | +| 1.8 | 0.4% | `tests/connector-allow-guard.bats` | +| 1.8 | 0.4% | `tests/board-payloads.bats` | +| 1.8 | 0.4% | `tests/branch-age-check.bats` | +| 1.7 | 0.4% | `tests/timeout-drift.bats` | +| 1.7 | 0.3% | `tests/mcp-timeout-budget.bats` | +| 1.6 | 0.3% | `tests/macos-link-check.bats` | +| 1.6 | 0.3% | `tests/lint-deno.bats` | +| 1.3 | 0.3% | `tests/sonar-gate.bats` | +| 1.2 | 0.2% | `tests/connector-allow-resolve.bats` | +| 1.1 | 0.2% | `tests/transcript-corpus-check.bats` | +| 1.0 | 0.2% | `tests/git-hook.bats` | +| 1.0 | 0.2% | `tests/nonverdict-assert.bats` | +| 0.9 | 0.2% | `tests/release-due.bats` | +| 0.6 | 0.1% | `tests/dist.bats` | +| 0.5 | 0.1% | `tests/remedy-payload-source.bats` | +| 0.5 | 0.1% | `tests/evaluator-io-check.bats` | +| 0.5 | 0.1% | `tests/token-bench-check.bats` | +| 0.4 | 0.1% | `tests/task-fail-closed.bats` | +| 0.3 | 0.1% | `tests/darwin-link.bats` | +| 0.3 | 0.1% | `tests/cross-check.bats` | | 0.0 | 0.0% | `tests/zizmor-split.bats` | diff --git a/completions/batten.bash b/completions/batten.bash index e28538149..591a4f22d 100644 --- a/completions/batten.bash +++ b/completions/batten.bash @@ -139,6 +139,9 @@ _batten() { batten,task) cmd="batten__subcmd__task" ;; + batten,verdict) + cmd="batten__subcmd__verdict" + ;; batten,wiring) cmd="batten__subcmd__wiring" ;; @@ -502,6 +505,9 @@ _batten() { batten__subcmd__help,task) cmd="batten__subcmd__help__subcmd__task" ;; + batten__subcmd__help,verdict) + cmd="batten__subcmd__help__subcmd__verdict" + ;; batten__subcmd__help,wiring) cmd="batten__subcmd__help__subcmd__wiring" ;; @@ -1412,7 +1418,7 @@ _batten() { case "${cmd}" in batten) - opts="-q -v -y -h -V --strictness --fail-on-warning --config-from --config-in --silent --quiet --verbose --debug --trace --log-level --no-color --no-input --yes --help --version check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" + opts="-q -v -y -h -V --strictness --fail-on-warning --config-from --config-in --silent --quiet --verbose --debug --trace --log-level --no-color --no-input --yes --help --version check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy verdict commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" if [[ ${cur} == -* || ${COMP_CWORD} -eq 1 ]] ; then COMPREPLY=( $(compgen -W "${opts}" -- "${cur}") ) return 0 @@ -3232,7 +3238,7 @@ _batten() { return 0 ;; batten__subcmd__exec) - opts="-q -v -y -h --capture-only --tee --jobs --continue-on-error --format --style --strictness --fail-on-warning --config-from --config-in --silent --quiet --verbose --debug --trace --log-level --no-color --no-input --yes --help" + opts="-q -v -y -h --capture-only --tee --jobs --continue-on-error --lock --lock-path --lock-attempts --lock-label --format --style --strictness --fail-on-warning --config-from --config-in --silent --quiet --verbose --debug --trace --log-level --no-color --no-input --yes --help" if [[ ${cur} == -* || ${COMP_CWORD} -eq 2 ]] ; then COMPREPLY=( $(compgen -W "${opts}" -- "${cur}") ) return 0 @@ -3242,6 +3248,22 @@ _batten() { COMPREPLY=($(compgen -f "${cur}")) return 0 ;; + --lock) + COMPREPLY=($(compgen -f "${cur}")) + return 0 + ;; + --lock-path) + COMPREPLY=($(compgen -f "${cur}")) + return 0 + ;; + --lock-attempts) + COMPREPLY=($(compgen -f "${cur}")) + return 0 + ;; + --lock-label) + COMPREPLY=($(compgen -f "${cur}")) + return 0 + ;; --format) COMPREPLY=($(compgen -W "human json jsonl" -- "${cur}")) return 0 @@ -3564,7 +3586,7 @@ _batten() { return 0 ;; batten__subcmd__help) - opts="check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" + opts="check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy verdict commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" if [[ ${cur} == -* || ${COMP_CWORD} -eq 2 ]] ; then COMPREPLY=( $(compgen -W "${opts}" -- "${cur}") ) return 0 @@ -5565,6 +5587,20 @@ _batten() { COMPREPLY=( $(compgen -W "${opts}" -- "${cur}") ) return 0 ;; + batten__subcmd__help__subcmd__verdict) + opts="" + if [[ ${cur} == -* || ${COMP_CWORD} -eq 3 ]] ; then + COMPREPLY=( $(compgen -W "${opts}" -- "${cur}") ) + return 0 + fi + case "${prev}" in + *) + COMPREPLY=() + ;; + esac + COMPREPLY=( $(compgen -W "${opts}" -- "${cur}") ) + return 0 + ;; batten__subcmd__help__subcmd__wiring) opts="reclaim" if [[ ${cur} == -* || ${COMP_CWORD} -eq 3 ]] ; then @@ -10315,6 +10351,44 @@ _batten() { COMPREPLY=( $(compgen -W "${opts}" -- "${cur}") ) return 0 ;; + batten__subcmd__verdict) + opts="-q -v -y -h --findings --unjudgeable --strictness --fail-on-warning --config-from --config-in --silent --quiet --verbose --debug --trace --log-level --no-color --no-input --yes --help" + if [[ ${cur} == -* || ${COMP_CWORD} -eq 2 ]] ; then + COMPREPLY=( $(compgen -W "${opts}" -- "${cur}") ) + return 0 + fi + case "${prev}" in + --findings) + COMPREPLY=($(compgen -f "${cur}")) + return 0 + ;; + --unjudgeable) + COMPREPLY=($(compgen -f "${cur}")) + return 0 + ;; + --strictness) + COMPREPLY=($(compgen -W "permissive standard strict" -- "${cur}")) + return 0 + ;; + --config-from) + COMPREPLY=($(compgen -f "${cur}")) + return 0 + ;; + --config-in) + COMPREPLY=($(compgen -f "${cur}")) + return 0 + ;; + --log-level) + COMPREPLY=($(compgen -W "silent quiet normal verbose debug trace" -- "${cur}")) + return 0 + ;; + *) + COMPREPLY=() + ;; + esac + COMPREPLY=( $(compgen -W "${opts}" -- "${cur}") ) + return 0 + ;; batten__subcmd__wiring) opts="-q -v -y -h --strictness --fail-on-warning --config-from --config-in --silent --quiet --verbose --debug --trace --log-level --no-color --no-input --yes --help reclaim help" if [[ ${cur} == -* || ${COMP_CWORD} -eq 2 ]] ; then diff --git a/completions/batten.fish b/completions/batten.fish index 122b06e7c..1c4639edc 100644 --- a/completions/batten.fish +++ b/completions/batten.fish @@ -62,6 +62,7 @@ complete -c batten -n "__fish_batten_needs_command" -f -a "generate" -d 'Emit ar complete -c batten -n "__fish_batten_needs_command" -f -a "perf" -d 'Measure this repository\'s own invocation cost' complete -c batten -n "__fish_batten_needs_command" -f -a "mutate" -d 'Decide whether this repository\'s gates discriminate, rather than merely parse' complete -c batten -n "__fish_batten_needs_command" -f -a "policy" -d 'Inspect the thresholds and path sets this repository holds itself to' +complete -c batten -n "__fish_batten_needs_command" -f -a "verdict" -d 'Fold a run\'s findings and blind spots into this tool\'s exit code' complete -c batten -n "__fish_batten_needs_command" -f -a "commit" -d 'The shape a commit must take here: what its subject may say' complete -c batten -n "__fish_batten_needs_command" -f -a "ready" -d 'Whether an issue\'s Ready block satisfies the checkable clauses of the gate' complete -c batten -n "__fish_batten_needs_command" -f -a "landed" -d 'Whether a board column is honest about what git and the forge already did' @@ -138,6 +139,10 @@ complete -c batten -n "__fish_batten_using_subcommand enforce" -l no-input -d 'N complete -c batten -n "__fish_batten_using_subcommand enforce" -s y -l yes -d 'Confirm a destructive operation that would otherwise refuse' complete -c batten -n "__fish_batten_using_subcommand enforce" -s h -l help -d 'Print help (see more with \'--help\')' complete -c batten -n "__fish_batten_using_subcommand exec" -l jobs -d 'How many of a `:::` bundle\'s commands run at once' -r +complete -c batten -n "__fish_batten_using_subcommand exec" -l lock -d 'Hold this clone\'s named singleton lock for the child\'s lifetime' -r +complete -c batten -n "__fish_batten_using_subcommand exec" -l lock-path -d 'Hold the lock at this path, for a resource the clone does not own' -r +complete -c batten -n "__fish_batten_using_subcommand exec" -l lock-attempts -d 'How many times to ask for the lock before reporting it held' -r +complete -c batten -n "__fish_batten_using_subcommand exec" -l lock-label -d 'What the wait is for, named by the caller for the refusal line' -r complete -c batten -n "__fish_batten_using_subcommand exec" -l format -d 'How Batten\'s own record is encoded (hk\'s axis)' -r -f -a "human\t'Pointer lines, one per fact' json\t'One JSON document' jsonl\t'One JSON record per line'" @@ -1256,6 +1261,29 @@ complete -c batten -n "__fish_batten_using_subcommand policy; and __fish_seen_su complete -c batten -n "__fish_batten_using_subcommand policy; and __fish_seen_subcommand_from help" -f -a "explain" -d 'Resolve a verdict token to its class definition and the routes out of it' complete -c batten -n "__fish_batten_using_subcommand policy; and __fish_seen_subcommand_from help" -f -a "rule" -d 'Resolve a rule id to the remedy its row declares' complete -c batten -n "__fish_batten_using_subcommand policy; and __fish_seen_subcommand_from help" -f -a "help" -d 'Print this message or the help of the given subcommand(s)' +complete -c batten -n "__fish_batten_using_subcommand verdict" -l findings -d 'How many blocking findings the run produced' -r +complete -c batten -n "__fish_batten_using_subcommand verdict" -l unjudgeable -d 'How many subjects the run could not read' -r +complete -c batten -n "__fish_batten_using_subcommand verdict" -l strictness -d 'Raise how strictly gates apply (an override may only tighten policy)' -r -f -a "permissive\t'Advisory: findings are reported without failing the run' +standard\t'The default: a finding is a violation' +strict\t'Everything `Standard` fails on, plus anything advisory'" +complete -c batten -n "__fish_batten_using_subcommand verdict" -l config-from -d 'Read the committed config from a git ref (e.g. origin/main) instead of the working tree' -r +complete -c batten -n "__fish_batten_using_subcommand verdict" -l config-in -d 'Read the committed config from this directory instead of the directory being judged' -r +complete -c batten -n "__fish_batten_using_subcommand verdict" -l log-level -d 'Set the verbosity rung by name' -r -f -a "silent\t'Say nothing but a verdict or a usage error' +quiet\t'Suppress ordinary progress; keep warnings' +normal\t'The default' +verbose\t'Explain what is being checked' +debug\t'Add resolution detail' +trace\t'Add everything'" +complete -c batten -n "__fish_batten_using_subcommand verdict" -l fail-on-warning -d 'Promote a warn-severity finding to a violation (an override may only turn this on)' +complete -c batten -n "__fish_batten_using_subcommand verdict" -l silent -d 'Say nothing but a verdict or a usage error' +complete -c batten -n "__fish_batten_using_subcommand verdict" -s q -l quiet -d 'Suppress ordinary progress (repeatable: -qq is silent)' +complete -c batten -n "__fish_batten_using_subcommand verdict" -s v -l verbose -d 'Explain what is being checked (repeatable: -vv is debug)' +complete -c batten -n "__fish_batten_using_subcommand verdict" -l debug -d 'Add resolution detail' +complete -c batten -n "__fish_batten_using_subcommand verdict" -l trace -d 'Add everything' +complete -c batten -n "__fish_batten_using_subcommand verdict" -l no-color -d 'Never colour stderr, whatever it is attached to' +complete -c batten -n "__fish_batten_using_subcommand verdict" -l no-input -d 'Never prompt; treat the run as unattended' +complete -c batten -n "__fish_batten_using_subcommand verdict" -s y -l yes -d 'Confirm a destructive operation that would otherwise refuse' +complete -c batten -n "__fish_batten_using_subcommand verdict" -s h -l help -d 'Print help (see more with \'--help\')' complete -c batten -n "__fish_batten_using_subcommand commit; and not __fish_seen_subcommand_from check help" -l strictness -d 'Raise how strictly gates apply (an override may only tighten policy)' -r -f -a "permissive\t'Advisory: findings are reported without failing the run' standard\t'The default: a finding is a violation' strict\t'Everything `Standard` fails on, plus anything advisory'" @@ -3587,49 +3615,50 @@ complete -c batten -n "__fish_batten_using_subcommand land; and __fish_seen_subc complete -c batten -n "__fish_batten_using_subcommand land; and __fish_seen_subcommand_from help" -f -a "fast-forward" -d 'Ask this head\'s pull request to fast-forward, and read the answer that request got' complete -c batten -n "__fish_batten_using_subcommand land; and __fish_seen_subcommand_from help" -f -a "lap" -d 'Drive the whole lap and lap again on any refusal a rebase would clear' complete -c batten -n "__fish_batten_using_subcommand land; and __fish_seen_subcommand_from help" -f -a "help" -d 'Print this message or the help of the given subcommand(s)' -complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "check" -d 'Run the applicable read-only gates against the repository' -complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "enforce" -d 'Run every configured rule, including kinds that execute a configured command' -complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "exec" -d 'Run a command — or a `:::` bundle — and report a pointer to what it wrote' -complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "capture" -d 'Captured command output: navigate what `exec` already ran, without running it again' -complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "mcp" -d 'Dispatch a declared MCP call and hand back a reduction instead of the payload' -complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "target" -d 'Inspect and reclaim this repository\'s build tree' -complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "config" -d 'Inspect configuration' -complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "lint" -d 'Lint an artifact against a declared schema' -complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "spec" -d 'Print the tool\'s own command spec' -complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "doctor" -d 'Diagnose whether Batten can run in this repository' -complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "init" -d 'Write a starter batten.toml, refusing to overwrite an existing one' -complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "baseline" -d 'Record the findings that already exist, so only new ones fail' -complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "generate" -d 'Emit artifacts derived from the command spec, on stdout' -complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "perf" -d 'Measure this repository\'s own invocation cost' -complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "mutate" -d 'Decide whether this repository\'s gates discriminate, rather than merely parse' -complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "policy" -d 'Inspect the thresholds and path sets this repository holds itself to' -complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "commit" -d 'The shape a commit must take here: what its subject may say' -complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "ready" -d 'Whether an issue\'s Ready block satisfies the checkable clauses of the gate' -complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "landed" -d 'Whether a board column is honest about what git and the forge already did' -complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "hk" -d 'The adopted gate runner\'s surface contract' -complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "checks" -d 'Whether a commit\'s check runs answer the question a landing depends on' -complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "pr" -d 'The pull request a landing drives, and the answers it waits on' -complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "task" -d 'What long-running tasks are doing, recorded where it can be read without a log' -complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "singleton" -d 'Whether a second copy of a task may start in this clone' -complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "claim" -d 'Whether the issue you are about to pull is actually unclaimed' -complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "semver" -d 'Whether this branch\'s API delta is compatible with the bump it claims' -complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "attribution" -d 'What produced commits may carry about the tooling that made them' -complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "worktree" -d 'Worktrees and the work in them: what is at risk' -complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "override" -d 'Issued admissions: an override is a record, never a variable somebody knows' -complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "provision" -d 'Pinned tools this repository provisions, cached out of tree' -complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "startup" -d 'Report whether this container matches what the repository declares' -complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "adjudicate" -d 'Adjudicate a mediated tool call read from stdin (a deny is exit 2, the one contract)' -complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "payload" -d 'Read a hook payload from stdin' -complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "receipt" -d 'Verification receipts: SHA-keyed claims a named check passed, invalidated by git facts' -complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "defects" -d 'The append-only defect ledger: the lessons this repository has already paid for' -complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "design" -d 'Design-evidence claims: the integrity of the record behind a decision' -complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "state" -d 'The out-of-tree findings store: which store belongs to this checkout' -complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "record" -d 'Out-of-tree verdict stores: what something else judged, keyed so a stale answer cannot answer' -complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "show" -d 'Report what something is, without changing it' -complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "wiring" -d 'Repair a host\'s hook registrations' -complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "lease" -d 'The landing lease: one branch spends a matrix at a time' -complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "land" -d 'The landing lap: replay this branch onto a base that moved' -complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "help" -d 'Print this message or the help of the given subcommand(s)' +complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy verdict commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "check" -d 'Run the applicable read-only gates against the repository' +complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy verdict commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "enforce" -d 'Run every configured rule, including kinds that execute a configured command' +complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy verdict commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "exec" -d 'Run a command — or a `:::` bundle — and report a pointer to what it wrote' +complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy verdict commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "capture" -d 'Captured command output: navigate what `exec` already ran, without running it again' +complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy verdict commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "mcp" -d 'Dispatch a declared MCP call and hand back a reduction instead of the payload' +complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy verdict commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "target" -d 'Inspect and reclaim this repository\'s build tree' +complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy verdict commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "config" -d 'Inspect configuration' +complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy verdict commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "lint" -d 'Lint an artifact against a declared schema' +complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy verdict commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "spec" -d 'Print the tool\'s own command spec' +complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy verdict commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "doctor" -d 'Diagnose whether Batten can run in this repository' +complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy verdict commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "init" -d 'Write a starter batten.toml, refusing to overwrite an existing one' +complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy verdict commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "baseline" -d 'Record the findings that already exist, so only new ones fail' +complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy verdict commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "generate" -d 'Emit artifacts derived from the command spec, on stdout' +complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy verdict commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "perf" -d 'Measure this repository\'s own invocation cost' +complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy verdict commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "mutate" -d 'Decide whether this repository\'s gates discriminate, rather than merely parse' +complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy verdict commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "policy" -d 'Inspect the thresholds and path sets this repository holds itself to' +complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy verdict commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "verdict" -d 'Fold a run\'s findings and blind spots into this tool\'s exit code' +complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy verdict commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "commit" -d 'The shape a commit must take here: what its subject may say' +complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy verdict commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "ready" -d 'Whether an issue\'s Ready block satisfies the checkable clauses of the gate' +complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy verdict commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "landed" -d 'Whether a board column is honest about what git and the forge already did' +complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy verdict commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "hk" -d 'The adopted gate runner\'s surface contract' +complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy verdict commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "checks" -d 'Whether a commit\'s check runs answer the question a landing depends on' +complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy verdict commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "pr" -d 'The pull request a landing drives, and the answers it waits on' +complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy verdict commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "task" -d 'What long-running tasks are doing, recorded where it can be read without a log' +complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy verdict commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "singleton" -d 'Whether a second copy of a task may start in this clone' +complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy verdict commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "claim" -d 'Whether the issue you are about to pull is actually unclaimed' +complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy verdict commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "semver" -d 'Whether this branch\'s API delta is compatible with the bump it claims' +complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy verdict commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "attribution" -d 'What produced commits may carry about the tooling that made them' +complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy verdict commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "worktree" -d 'Worktrees and the work in them: what is at risk' +complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy verdict commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "override" -d 'Issued admissions: an override is a record, never a variable somebody knows' +complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy verdict commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "provision" -d 'Pinned tools this repository provisions, cached out of tree' +complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy verdict commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "startup" -d 'Report whether this container matches what the repository declares' +complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy verdict commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "adjudicate" -d 'Adjudicate a mediated tool call read from stdin (a deny is exit 2, the one contract)' +complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy verdict commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "payload" -d 'Read a hook payload from stdin' +complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy verdict commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "receipt" -d 'Verification receipts: SHA-keyed claims a named check passed, invalidated by git facts' +complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy verdict commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "defects" -d 'The append-only defect ledger: the lessons this repository has already paid for' +complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy verdict commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "design" -d 'Design-evidence claims: the integrity of the record behind a decision' +complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy verdict commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "state" -d 'The out-of-tree findings store: which store belongs to this checkout' +complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy verdict commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "record" -d 'Out-of-tree verdict stores: what something else judged, keyed so a stale answer cannot answer' +complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy verdict commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "show" -d 'Report what something is, without changing it' +complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy verdict commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "wiring" -d 'Repair a host\'s hook registrations' +complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy verdict commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "lease" -d 'The landing lease: one branch spends a matrix at a time' +complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy verdict commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "land" -d 'The landing lap: replay this branch onto a base that moved' +complete -c batten -n "__fish_batten_using_subcommand help; and not __fish_seen_subcommand_from check enforce exec capture mcp target config lint spec doctor init baseline generate perf mutate policy verdict commit ready landed hk checks pr task singleton claim semver attribution worktree override provision startup adjudicate payload receipt defects design state record show wiring lease land help" -f -a "help" -d 'Print this message or the help of the given subcommand(s)' complete -c batten -n "__fish_batten_using_subcommand help; and __fish_seen_subcommand_from capture" -f -a "show" -d 'Print a capture\'s pointer, or the lines a selection asks for, with no second run' complete -c batten -n "__fish_batten_using_subcommand help; and __fish_seen_subcommand_from capture" -f -a "find" -d 'Resolve a stored tool response by the key it carries, with no handle to look up first' complete -c batten -n "__fish_batten_using_subcommand help; and __fish_seen_subcommand_from capture" -f -a "list" -d 'List this repository\'s captures as handles, in a fixed order' diff --git a/completions/batten.zsh b/completions/batten.zsh index d4a9f9b04..eccf1ac4f 100644 --- a/completions/batten.zsh +++ b/completions/batten.zsh @@ -120,6 +120,10 @@ trace\:"Add everything"))' \ (exec) _arguments "${_arguments_options[@]}" : \ '--jobs=[How many of a \`\:\:\:\` bundle'\''s commands run at once]: :_default' \ +'--lock=[Hold this clone'\''s named singleton lock for the child'\''s lifetime]: :_default' \ +'--lock-path=[Hold the lock at this path, for a resource the clone does not own]: :_default' \ +'--lock-attempts=[How many times to ask for the lock before reporting it held]: :_default' \ +'--lock-label=[What the wait is for, named by the caller for the refusal line]: :_default' \ '--format=[How Batten'\''s own record is encoded (hk'\''s axis)]: :((human\:"Pointer lines, one per fact" json\:"One JSON document" jsonl\:"One JSON record per line"))' \ @@ -1999,6 +2003,37 @@ esac ;; esac ;; +(verdict) +_arguments "${_arguments_options[@]}" : \ +'--findings=[How many blocking findings the run produced]: :_default' \ +'--unjudgeable=[How many subjects the run could not read]: :_default' \ +'--strictness=[Raise how strictly gates apply (an override may only tighten policy)]: :((permissive\:"Advisory\: findings are reported without failing the run" +standard\:"The default\: a finding is a violation" +strict\:"Everything \`Standard\` fails on, plus anything advisory"))' \ +'--config-from=[Read the committed config from a git ref (e.g. origin/main) instead of the working tree]: :_default' \ +'--config-in=[Read the committed config from this directory instead of the directory being judged]: :_default' \ +'--log-level=[Set the verbosity rung by name]: :((silent\:"Say nothing but a verdict or a usage error" +quiet\:"Suppress ordinary progress; keep warnings" +normal\:"The default" +verbose\:"Explain what is being checked" +debug\:"Add resolution detail" +trace\:"Add everything"))' \ +'--fail-on-warning[Promote a warn-severity finding to a violation (an override may only turn this on)]' \ +'*--silent[Say nothing but a verdict or a usage error]' \ +'*-q[Suppress ordinary progress (repeatable\: -qq is silent)]' \ +'*--quiet[Suppress ordinary progress (repeatable\: -qq is silent)]' \ +'*-v[Explain what is being checked (repeatable\: -vv is debug)]' \ +'*--verbose[Explain what is being checked (repeatable\: -vv is debug)]' \ +'*--debug[Add resolution detail]' \ +'*--trace[Add everything]' \ +'--no-color[Never colour stderr, whatever it is attached to]' \ +'--no-input[Never prompt; treat the run as unattended]' \ +'-y[Confirm a destructive operation that would otherwise refuse]' \ +'--yes[Confirm a destructive operation that would otherwise refuse]' \ +'-h[Print help (see more with '\''--help'\'')]' \ +'--help[Print help (see more with '\''--help'\'')]' \ +&& ret=0 +;; (commit) _arguments "${_arguments_options[@]}" : \ '--strictness=[Raise how strictly gates apply (an override may only tighten policy)]: :((permissive\:"Advisory\: findings are reported without failing the run" @@ -6351,6 +6386,10 @@ _arguments "${_arguments_options[@]}" : \ ;; esac ;; +(verdict) +_arguments "${_arguments_options[@]}" : \ +&& ret=0 +;; (commit) _arguments "${_arguments_options[@]}" : \ ":: :_batten__subcmd__help__subcmd__commit_commands" \ @@ -7059,6 +7098,7 @@ _batten_commands() { 'perf:Measure this repository'\''s own invocation cost' \ 'mutate:Decide whether this repository'\''s gates discriminate, rather than merely parse' \ 'policy:Inspect the thresholds and path sets this repository holds itself to' \ +'verdict:Fold a run'\''s findings and blind spots into this tool'\''s exit code' \ 'commit:The shape a commit must take here\: what its subject may say' \ 'ready:Whether an issue'\''s Ready block satisfies the checkable clauses of the gate' \ 'landed:Whether a board column is honest about what git and the forge already did' \ @@ -7671,6 +7711,7 @@ _batten__subcmd__help_commands() { 'perf:Measure this repository'\''s own invocation cost' \ 'mutate:Decide whether this repository'\''s gates discriminate, rather than merely parse' \ 'policy:Inspect the thresholds and path sets this repository holds itself to' \ +'verdict:Fold a run'\''s findings and blind spots into this tool'\''s exit code' \ 'commit:The shape a commit must take here\: what its subject may say' \ 'ready:Whether an issue'\''s Ready block satisfies the checkable clauses of the gate' \ 'landed:Whether a board column is honest about what git and the forge already did' \ @@ -8544,6 +8585,11 @@ _batten__subcmd__help__subcmd__task__subcmd__unregister_commands() { local commands; commands=() _describe -t commands 'batten help task unregister commands' commands "$@" } +(( $+functions[_batten__subcmd__help__subcmd__verdict_commands] )) || +_batten__subcmd__help__subcmd__verdict_commands() { + local commands; commands=() + _describe -t commands 'batten help verdict commands' commands "$@" +} (( $+functions[_batten__subcmd__help__subcmd__wiring_commands] )) || _batten__subcmd__help__subcmd__wiring_commands() { local commands; commands=( @@ -9865,6 +9911,11 @@ _batten__subcmd__task__subcmd__unregister_commands() { local commands; commands=() _describe -t commands 'batten task unregister commands' commands "$@" } +(( $+functions[_batten__subcmd__verdict_commands] )) || +_batten__subcmd__verdict_commands() { + local commands; commands=() + _describe -t commands 'batten verdict commands' commands "$@" +} (( $+functions[_batten__subcmd__wiring_commands] )) || _batten__subcmd__wiring_commands() { local commands; commands=( diff --git a/contracts/hk.json b/contracts/hk.json index eacf8e7c7..819d49a48 100644 --- a/contracts/hk.json +++ b/contracts/hk.json @@ -13,22 +13,11 @@ "id": "group_0", "stepIds": [ "hk-version", - "ci-tools-check", - "rust-paths-check", "suite-bench-check", "mcp-timeout-budget", - "msrv-pin-agreement", - "cap-drift", - "digest-major-agreement", - "report-only-check", "no-docs-tree", - "no-doctests", - "license-table-check", - "coderabbit-config-check", "hook-profile-check", "hk-contract-check", - "module-map-check", - "hook-pin-check", "mutant-census", "rules-drift", "hooks-wiring-check", @@ -39,16 +28,10 @@ "schema-check", "sbom-check", "install-check", - "publish-credential-check", "release-tracking-check", "mcp-allow-check", - "awk-regex-check", - "pipefail-grep-check", "lock-complete", - "timeout-check", - "mise-action-floor", "policy-budget", - "batten-glob-check", "skill-check", "macos-link-check", "shfmt", @@ -78,334 +61,232 @@ "orderIndex": 0, "parallelGroupId": "group_0" }, - { - "name": "ci-tools-check", - "status": "included", - "orderIndex": 1, - "parallelGroupId": "group_0" - }, - { - "name": "rust-paths-check", - "status": "included", - "orderIndex": 2, - "parallelGroupId": "group_0" - }, { "name": "suite-bench-check", "status": "included", - "orderIndex": 3, + "orderIndex": 1, "parallelGroupId": "group_0" }, { "name": "mcp-timeout-budget", "status": "included", - "orderIndex": 4, - "parallelGroupId": "group_0" - }, - { - "name": "msrv-pin-agreement", - "status": "included", - "orderIndex": 5, - "parallelGroupId": "group_0" - }, - { - "name": "cap-drift", - "status": "included", - "orderIndex": 6, - "parallelGroupId": "group_0" - }, - { - "name": "digest-major-agreement", - "status": "included", - "orderIndex": 7, - "parallelGroupId": "group_0" - }, - { - "name": "report-only-check", - "status": "included", - "orderIndex": 8, + "orderIndex": 2, "parallelGroupId": "group_0" }, { "name": "no-docs-tree", "status": "included", - "orderIndex": 9, - "parallelGroupId": "group_0" - }, - { - "name": "no-doctests", - "status": "included", - "orderIndex": 10, - "parallelGroupId": "group_0" - }, - { - "name": "license-table-check", - "status": "included", - "orderIndex": 11, - "parallelGroupId": "group_0" - }, - { - "name": "coderabbit-config-check", - "status": "included", - "orderIndex": 12, + "orderIndex": 3, "parallelGroupId": "group_0" }, { "name": "hook-profile-check", "status": "included", - "orderIndex": 13, + "orderIndex": 4, "parallelGroupId": "group_0" }, { "name": "hk-contract-check", "status": "included", - "orderIndex": 14, - "parallelGroupId": "group_0" - }, - { - "name": "module-map-check", - "status": "included", - "orderIndex": 15, - "parallelGroupId": "group_0" - }, - { - "name": "hook-pin-check", - "status": "included", - "orderIndex": 16, + "orderIndex": 5, "parallelGroupId": "group_0" }, { "name": "mutant-census", "status": "included", - "orderIndex": 17, + "orderIndex": 6, "parallelGroupId": "group_0" }, { "name": "rules-drift", "status": "included", - "orderIndex": 18, + "orderIndex": 7, "parallelGroupId": "group_0" }, { "name": "hooks-wiring-check", "status": "included", - "orderIndex": 19, + "orderIndex": 8, "parallelGroupId": "group_0" }, { "name": "reference-check", "status": "included", - "orderIndex": 20, + "orderIndex": 9, "parallelGroupId": "group_0" }, { "name": "token-bench-check", "status": "included", - "orderIndex": 21, + "orderIndex": 10, "parallelGroupId": "group_0" }, { "name": "config-lint", "status": "included", - "orderIndex": 22, + "orderIndex": 11, "parallelGroupId": "group_0" }, { "name": "renovate-config-validator", "status": "included", - "orderIndex": 23, + "orderIndex": 12, "parallelGroupId": "group_0" }, { "name": "schema-check", "status": "included", - "orderIndex": 24, + "orderIndex": 13, "parallelGroupId": "group_0" }, { "name": "sbom-check", "status": "included", - "orderIndex": 25, + "orderIndex": 14, "parallelGroupId": "group_0" }, { "name": "install-check", "status": "included", - "orderIndex": 26, - "parallelGroupId": "group_0" - }, - { - "name": "publish-credential-check", - "status": "included", - "orderIndex": 27, + "orderIndex": 15, "parallelGroupId": "group_0" }, { "name": "release-tracking-check", "status": "included", - "orderIndex": 28, + "orderIndex": 16, "parallelGroupId": "group_0" }, { "name": "mcp-allow-check", "status": "included", - "orderIndex": 29, - "parallelGroupId": "group_0" - }, - { - "name": "awk-regex-check", - "status": "included", - "orderIndex": 30, - "parallelGroupId": "group_0" - }, - { - "name": "pipefail-grep-check", - "status": "included", - "orderIndex": 31, + "orderIndex": 17, "parallelGroupId": "group_0" }, { "name": "lock-complete", "status": "included", - "orderIndex": 32, - "parallelGroupId": "group_0" - }, - { - "name": "timeout-check", - "status": "included", - "orderIndex": 33, - "parallelGroupId": "group_0" - }, - { - "name": "mise-action-floor", - "status": "included", - "orderIndex": 34, + "orderIndex": 18, "parallelGroupId": "group_0" }, { "name": "policy-budget", "status": "included", - "orderIndex": 35, - "parallelGroupId": "group_0" - }, - { - "name": "batten-glob-check", - "status": "included", - "orderIndex": 36, + "orderIndex": 19, "parallelGroupId": "group_0" }, { "name": "skill-check", "status": "included", - "orderIndex": 37, + "orderIndex": 20, "parallelGroupId": "group_0" }, { "name": "macos-link-check", "status": "included", - "orderIndex": 38, + "orderIndex": 21, "parallelGroupId": "group_0" }, { "name": "shfmt", "status": "included", - "orderIndex": 39, + "orderIndex": 22, "parallelGroupId": "group_0" }, { "name": "shellcheck", "status": "included", - "orderIndex": 40, + "orderIndex": 23, "parallelGroupId": "group_0" }, { "name": "test:bats", "status": "included", - "orderIndex": 41, + "orderIndex": 24, "parallelGroupId": "group_0" }, { "name": "taplo", "status": "included", - "orderIndex": 42, + "orderIndex": 25, "parallelGroupId": "group_0" }, { "name": "taplo-format", "status": "included", - "orderIndex": 43, + "orderIndex": 26, "parallelGroupId": "group_0" }, { "name": "pkl", "status": "included", - "orderIndex": 44, + "orderIndex": 27, "parallelGroupId": "group_0" }, { "name": "pkl-format", "status": "included", - "orderIndex": 45, + "orderIndex": 28, "parallelGroupId": "group_0" }, { "name": "prettier", "status": "included", - "orderIndex": 46, + "orderIndex": 29, "parallelGroupId": "group_0" }, { "name": "deno-fmt", "status": "included", - "orderIndex": 47, + "orderIndex": 30, "parallelGroupId": "group_0" }, { "name": "rego", "status": "included", - "orderIndex": 48, + "orderIndex": 31, "parallelGroupId": "group_0" }, { "name": "actionlint", "status": "included", - "orderIndex": 49, + "orderIndex": 32, "parallelGroupId": "group_0" }, { "name": "cargo-fmt", "status": "included", - "orderIndex": 50, + "orderIndex": 33, "parallelGroupId": "group_0" }, { "name": "cargo-clippy", "status": "included", - "orderIndex": 51, + "orderIndex": 34, "parallelGroupId": "group_0" }, { "name": "test", "status": "included", - "orderIndex": 52, + "orderIndex": 35, "parallelGroupId": "group_0" }, { "name": "batten-check", "status": "included", - "orderIndex": 53, + "orderIndex": 36, "parallelGroupId": "group_0" }, { "name": "policy-test", "status": "included", - "orderIndex": 54, + "orderIndex": 37, "parallelGroupId": "group_0" }, { "name": "memories-check", "status": "included", - "orderIndex": 55, + "orderIndex": 38, "parallelGroupId": "group_0" } ] @@ -486,22 +367,11 @@ "id": "group_0", "stepIds": [ "hk-version", - "ci-tools-check", - "rust-paths-check", "suite-bench-check", "mcp-timeout-budget", - "msrv-pin-agreement", - "cap-drift", - "digest-major-agreement", - "report-only-check", "no-docs-tree", - "no-doctests", - "license-table-check", - "coderabbit-config-check", "hook-profile-check", "hk-contract-check", - "module-map-check", - "hook-pin-check", "mutant-census", "rules-drift", "hooks-wiring-check", @@ -512,16 +382,10 @@ "schema-check", "sbom-check", "install-check", - "publish-credential-check", "release-tracking-check", "mcp-allow-check", - "awk-regex-check", - "pipefail-grep-check", "lock-complete", - "timeout-check", - "mise-action-floor", "policy-budget", - "batten-glob-check", "skill-check", "macos-link-check", "shfmt", @@ -550,328 +414,226 @@ "orderIndex": 0, "parallelGroupId": "group_0" }, - { - "name": "ci-tools-check", - "status": "included", - "orderIndex": 1, - "parallelGroupId": "group_0" - }, - { - "name": "rust-paths-check", - "status": "included", - "orderIndex": 2, - "parallelGroupId": "group_0" - }, { "name": "suite-bench-check", "status": "included", - "orderIndex": 3, + "orderIndex": 1, "parallelGroupId": "group_0" }, { "name": "mcp-timeout-budget", "status": "included", - "orderIndex": 4, - "parallelGroupId": "group_0" - }, - { - "name": "msrv-pin-agreement", - "status": "included", - "orderIndex": 5, - "parallelGroupId": "group_0" - }, - { - "name": "cap-drift", - "status": "included", - "orderIndex": 6, - "parallelGroupId": "group_0" - }, - { - "name": "digest-major-agreement", - "status": "included", - "orderIndex": 7, - "parallelGroupId": "group_0" - }, - { - "name": "report-only-check", - "status": "included", - "orderIndex": 8, + "orderIndex": 2, "parallelGroupId": "group_0" }, { "name": "no-docs-tree", "status": "included", - "orderIndex": 9, - "parallelGroupId": "group_0" - }, - { - "name": "no-doctests", - "status": "included", - "orderIndex": 10, - "parallelGroupId": "group_0" - }, - { - "name": "license-table-check", - "status": "included", - "orderIndex": 11, - "parallelGroupId": "group_0" - }, - { - "name": "coderabbit-config-check", - "status": "included", - "orderIndex": 12, + "orderIndex": 3, "parallelGroupId": "group_0" }, { "name": "hook-profile-check", "status": "included", - "orderIndex": 13, + "orderIndex": 4, "parallelGroupId": "group_0" }, { "name": "hk-contract-check", "status": "included", - "orderIndex": 14, - "parallelGroupId": "group_0" - }, - { - "name": "module-map-check", - "status": "included", - "orderIndex": 15, - "parallelGroupId": "group_0" - }, - { - "name": "hook-pin-check", - "status": "included", - "orderIndex": 16, + "orderIndex": 5, "parallelGroupId": "group_0" }, { "name": "mutant-census", "status": "included", - "orderIndex": 17, + "orderIndex": 6, "parallelGroupId": "group_0" }, { "name": "rules-drift", "status": "included", - "orderIndex": 18, + "orderIndex": 7, "parallelGroupId": "group_0" }, { "name": "hooks-wiring-check", "status": "included", - "orderIndex": 19, + "orderIndex": 8, "parallelGroupId": "group_0" }, { "name": "reference-check", "status": "included", - "orderIndex": 20, + "orderIndex": 9, "parallelGroupId": "group_0" }, { "name": "token-bench-check", "status": "included", - "orderIndex": 21, + "orderIndex": 10, "parallelGroupId": "group_0" }, { "name": "config-lint", "status": "included", - "orderIndex": 22, + "orderIndex": 11, "parallelGroupId": "group_0" }, { "name": "renovate-config-validator", "status": "included", - "orderIndex": 23, + "orderIndex": 12, "parallelGroupId": "group_0" }, { "name": "schema-check", "status": "included", - "orderIndex": 24, + "orderIndex": 13, "parallelGroupId": "group_0" }, { "name": "sbom-check", "status": "included", - "orderIndex": 25, + "orderIndex": 14, "parallelGroupId": "group_0" }, { "name": "install-check", "status": "included", - "orderIndex": 26, - "parallelGroupId": "group_0" - }, - { - "name": "publish-credential-check", - "status": "included", - "orderIndex": 27, + "orderIndex": 15, "parallelGroupId": "group_0" }, { "name": "release-tracking-check", "status": "included", - "orderIndex": 28, + "orderIndex": 16, "parallelGroupId": "group_0" }, { "name": "mcp-allow-check", "status": "included", - "orderIndex": 29, - "parallelGroupId": "group_0" - }, - { - "name": "awk-regex-check", - "status": "included", - "orderIndex": 30, - "parallelGroupId": "group_0" - }, - { - "name": "pipefail-grep-check", - "status": "included", - "orderIndex": 31, + "orderIndex": 17, "parallelGroupId": "group_0" }, { "name": "lock-complete", "status": "included", - "orderIndex": 32, - "parallelGroupId": "group_0" - }, - { - "name": "timeout-check", - "status": "included", - "orderIndex": 33, - "parallelGroupId": "group_0" - }, - { - "name": "mise-action-floor", - "status": "included", - "orderIndex": 34, + "orderIndex": 18, "parallelGroupId": "group_0" }, { "name": "policy-budget", "status": "included", - "orderIndex": 35, - "parallelGroupId": "group_0" - }, - { - "name": "batten-glob-check", - "status": "included", - "orderIndex": 36, + "orderIndex": 19, "parallelGroupId": "group_0" }, { "name": "skill-check", "status": "included", - "orderIndex": 37, + "orderIndex": 20, "parallelGroupId": "group_0" }, { "name": "macos-link-check", "status": "included", - "orderIndex": 38, + "orderIndex": 21, "parallelGroupId": "group_0" }, { "name": "shfmt", "status": "included", - "orderIndex": 39, + "orderIndex": 22, "parallelGroupId": "group_0" }, { "name": "shellcheck", "status": "included", - "orderIndex": 40, + "orderIndex": 23, "parallelGroupId": "group_0" }, { "name": "test:bats", "status": "included", - "orderIndex": 41, + "orderIndex": 24, "parallelGroupId": "group_0" }, { "name": "taplo", "status": "included", - "orderIndex": 42, + "orderIndex": 25, "parallelGroupId": "group_0" }, { "name": "taplo-format", "status": "included", - "orderIndex": 43, + "orderIndex": 26, "parallelGroupId": "group_0" }, { "name": "pkl", "status": "included", - "orderIndex": 44, + "orderIndex": 27, "parallelGroupId": "group_0" }, { "name": "pkl-format", "status": "included", - "orderIndex": 45, + "orderIndex": 28, "parallelGroupId": "group_0" }, { "name": "prettier", "status": "included", - "orderIndex": 46, + "orderIndex": 29, "parallelGroupId": "group_0" }, { "name": "deno-fmt", "status": "included", - "orderIndex": 47, + "orderIndex": 30, "parallelGroupId": "group_0" }, { "name": "rego", "status": "included", - "orderIndex": 48, + "orderIndex": 31, "parallelGroupId": "group_0" }, { "name": "actionlint", "status": "included", - "orderIndex": 49, + "orderIndex": 32, "parallelGroupId": "group_0" }, { "name": "cargo-fmt", "status": "included", - "orderIndex": 50, + "orderIndex": 33, "parallelGroupId": "group_0" }, { "name": "cargo-clippy", "status": "included", - "orderIndex": 51, + "orderIndex": 34, "parallelGroupId": "group_0" }, { "name": "test", "status": "included", - "orderIndex": 52, + "orderIndex": 35, "parallelGroupId": "group_0" }, { "name": "batten-check", "status": "included", - "orderIndex": 53, + "orderIndex": 36, "parallelGroupId": "group_0" }, { "name": "policy-test", "status": "included", - "orderIndex": 54, + "orderIndex": 37, "parallelGroupId": "group_0" } ] diff --git a/crates/batten/src/arm.rs b/crates/batten/src/arm.rs new file mode 100644 index 000000000..4aa435d83 --- /dev/null +++ b/crates/batten/src/arm.rs @@ -0,0 +1,405 @@ +//! The declared-arm harness: run N declared things, reduce each to a named +//! observable, and hand back one record per arm (CLOUD-1714). +//! +//! # Why this exists +//! +//! The engine shipped **two** instances of "run N declared things, reduce to an +//! observable, record, adjudicate" and neither was generic, so two more lived in +//! bash. [`crate::perf`] spawns a benchmarking tool and reduces wall-clock to +//! percentiles; [`crate::mutate`] stages a repository copy, runs a declared +//! command and reduces a test stream to pass/fail counts. The bash pair reduce a +//! multi-path sweep to percentiles and a merged output stream to a byte count. +//! Four copies of a percentile function existed between them. +//! +//! This is the primitive all four are instances of: a table of +//! `(id, cwd, argv, stdin, env)` arms, a run count, and a pluggable +//! [`Observable`]. **It generalises the two shipped instances rather than adding +//! a third** — [`percentile`] is the one this module owns, and both consumers +//! call it here. +//! +//! # The percentile reduction is owned HERE +//! +//! Decided on CLOUD-1714 and recorded so CLOUD-1712 can consume it: the +//! `forge::window` cluster needs percentiles over durations it FETCHED rather +//! than ran, which is the same reduction over a different series. A fifth copy +//! there would be the drift this module exists to end, so [`percentile`] takes a +//! bare series and knows nothing about where it came from. +//! +//! # A FAILED ARM IS COULD-NOT-LOOK, NEVER A ZERO +//! +//! [`Outcome`] is the whole reason this is not a `Vec`. A zero measurement +//! and an absent one are different verdicts, and conflating them is how a broken +//! arm reads as **fast**: an arm that failed to start, or whose output could not +//! be parsed, contributes nothing to a comparison rather than contributing the +//! best number in it. This is [`crate::findings::Observation`]'s distinction +//! applied to measurement. +//! +//! # BYTE-STABILITY IS AN OBSERVABLE, NOT AN ASSERTION +//! +//! The bash token benchmark asserts that arms produce identical bytes across +//! runs *before* it counts them, and a harness that only reported the last run +//! could not see instability at all. So [`Observable::Stability`] is a +//! first-class reduction: an arm whose runs disagree is reported +//! [`Reading::Unstable`], never averaged into a number that describes none of +//! them. +//! +//! # Isolation is BEHAVIOUR, not setup +//! +//! Arms run with a state root of their own because a contributor's ambient state +//! must not move a measurement — and because an arm that WRITES would otherwise +//! leave the store in a condition depending on which arm ran first, making the +//! result a fact about ordering. [`Isolation`] carries that explicitly rather +//! than leaving it to each caller to remember (CLOUD-1559: a port carries the +//! decisions, not the steps). +//! +//! # Output is a pointer +//! +//! [`Reading::line`] renders `arm= observable= …` and never a byte of +//! a captured stream (non-negotiable rule 4). The streams these arms produce are +//! exactly the payload that must not reach a log. + +use std::collections::BTreeMap; +use std::path::{Path, PathBuf}; + +/// One declared thing to run. +/// +/// `id` is the arm's name in every record and refusal it produces; the rest is +/// what it takes to run it once. Declared as data so a caller states its table +/// and this module owns the loop. +#[derive(Debug, Clone)] +pub struct Arm { + /// The name this arm carries in every record and refusal. + pub id: String, + /// Where the arm runs. + pub cwd: PathBuf, + /// The program and its arguments. The first element is the program. + pub argv: Vec, + /// What to write to the arm's standard input, if anything. + pub stdin: Option, + /// Environment entries set for this arm, over and above [`Isolation`]. + pub env: BTreeMap, +} + +impl Arm { + /// A new arm with no stdin and no extra environment. + #[must_use] + pub fn new(id: impl Into, cwd: impl Into, argv: Vec) -> Self { + Self { + id: id.into(), + cwd: cwd.into(), + argv, + stdin: None, + env: BTreeMap::new(), + } + } +} + +/// The state root an arm runs under, so ambient state cannot move a measurement. +/// +/// Not a convenience: the perf pair's post-tool arm WRITES, and two arms sharing +/// a store would each read rows the other created. That is an order dependency, +/// and it does not divide out of a ratio — which is the only thing the comparison +/// reads. +#[derive(Debug, Clone)] +pub struct Isolation { + root: PathBuf, +} + +impl Isolation { + /// A state root at `root`. + #[must_use] + pub fn at(root: impl Into) -> Self { + Self { root: root.into() } + } + + /// The environment entries that point every state-writing path at this root. + /// + /// All four names are set together and always: a caller that set only the + /// one its own platform reads would leave the others ambient, which is the + /// leak this type exists to close. + #[must_use] + pub fn env(&self) -> BTreeMap { + let root = self.root.to_string_lossy().into_owned(); + let mut env = BTreeMap::new(); + for name in ["HOME", "XDG_DATA_HOME", "APPDATA", "LOCALAPPDATA"] { + env.insert(String::from(name), root.clone()); + } + env + } + + /// The root itself, for a caller that must create or empty it. + #[must_use] + pub fn root(&self) -> &Path { + &self.root + } +} + +/// What an arm's runs are reduced to. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum Observable { + /// A percentile of the wall-clock series, in the caller's own units. + WallClock, + /// The byte count of the arm's merged output stream. + Bytes, + /// Whether every run produced identical bytes. + Stability, + /// The arm's exit status. + ExitStatus, +} + +impl Observable { + /// The stable name this observable carries in a record line. + #[must_use] + pub const fn name(self) -> &'static str { + match self { + Observable::WallClock => "wall-clock", + Observable::Bytes => "bytes", + Observable::Stability => "stability", + Observable::ExitStatus => "exit", + } + } +} + +/// What an arm's runs reduced to, or why they did not. +#[derive(Debug, Clone, PartialEq)] +pub enum Reading { + /// A percentile pair over the wall-clock series, with the sample count. + WallClock { + /// The median. + p50: f64, + /// The 95th percentile. + p95: f64, + /// How many runs the series holds. + runs: usize, + }, + /// A byte count every run agreed on. + Bytes(usize), + /// Every run produced identical bytes. + Stable, + /// The runs disagreed, and this is reported rather than averaged away. + Unstable { + /// How many distinct outputs the runs produced. + distinct: usize, + /// How many runs there were. + runs: usize, + }, + /// The arm's exit status. + ExitStatus(i32), +} + +/// One arm's result: observed, or not. +/// +/// The two arms are the point. A zero measurement and an absent one are +/// different verdicts, and a harness that returned `0.0` for an arm that failed +/// to start would report the broken arm as the fastest one. +#[derive(Debug, Clone, PartialEq)] +pub enum Outcome { + /// The arm ran and reduced. + Observed(Reading), + /// The arm could not be measured, and this is why. + NotObserved(String), +} + +impl Outcome { + /// Whether this arm produced a reading at all. + #[must_use] + pub const fn observed(&self) -> bool { + matches!(self, Outcome::Observed(_)) + } + + /// The pointer line this outcome renders as. + /// + /// Pointer-only per non-negotiable rule 4: an arm's id, the observable's + /// name and the reduction. Never a byte of the stream that produced it — + /// which for these arms is exactly the payload that must not reach a log. + #[must_use] + pub fn line(&self, id: &str, observable: Observable) -> String { + let value = match self { + Outcome::Observed(Reading::WallClock { p50, p95, runs }) => { + format!("p50={p50} p95={p95} runs={runs}") + } + Outcome::Observed(Reading::Bytes(count)) => format!("count={count}"), + Outcome::Observed(Reading::Stable) => String::from("stable"), + Outcome::Observed(Reading::Unstable { distinct, runs }) => { + format!("UNSTABLE distinct={distinct} runs={runs}") + } + Outcome::Observed(Reading::ExitStatus(code)) => format!("code={code}"), + // The reason is the caller's own text, not captured content: every + // construction site below writes it, and none of them reads a stream. + Outcome::NotObserved(why) => format!("not-observed {why}"), + }; + format!("arm={id} observable={} {value}", observable.name()) + } +} + +/// A percentile over a series, by the nearest-rank convention this repository +/// already used. +/// +/// **The one copy.** Four existed across the two shipped instances and the two +/// bash programs; CLOUD-1714 decides this module owns it, and CLOUD-1712's +/// fetched-duration series consume it here rather than growing a fifth. +/// +/// The series is taken by value and sorted, because a percentile over an +/// unsorted series is the bug this signature makes unrepresentable. An empty +/// series has no percentile — which is an [`Outcome::NotObserved`] at the call +/// site, never a zero. +/// +/// # The quantile is a RATIO OF INTEGERS, not a float, and that is the whole +/// reason the rank arithmetic carries no lint escape +/// +/// The obvious signature takes `f64` and pays for it three times: `(n - 1) as +/// f64` is a precision loss, and `position.ceil() as usize` is a possible +/// truncation and a sign loss. Each is provably harmless here and each still +/// needs an `#[expect]` to say so — three escapes in engine source, which +/// `spawn-widening` counts as inventory growth whether or not the reasoning +/// behind them is sound. +/// +/// A rank is an index, and an index is an integer. `numerator / denominator` +/// says the same thing the float said and computes the ceiling exactly, so +/// nothing is rounded, nothing is cast, and there is no lint to waive. A zero +/// denominator is [`None`] rather than a panic, for the reason an empty series +/// is: this is a reduction, and a reduction that cannot answer says so. +#[must_use] +pub fn percentile(mut series: Vec, numerator: usize, denominator: usize) -> Option { + if series.is_empty() || denominator == 0 { + return None; + } + series.sort_by(f64::total_cmp); + let last = series.len() - 1; + // CLAMPED RATHER THAN REFUSED, because the clamp is what a percentile above + // the top of the series means: the last element. The float form clamped the + // quantile to `[0, 1]` and this clamps the rank to `[0, last]`, which is the + // same bound expressed where it can be checked. + let scaled = last.saturating_mul(numerator.min(denominator)); + // CEIL rather than round: the 95th percentile of a short series must not + // round DOWN into the body of the distribution, which is how a tail that a + // budget exists to bound stops being represented at all. Integer ceiling + // division, so the rounding is exact rather than a float's best effort. + let index = scaled.div_ceil(denominator); + series.get(index.min(last)).copied() +} + +/// Reduce a wall-clock series to a reading, or say why it could not be. +#[must_use] +pub fn wall_clock(series: Vec) -> Outcome { + let runs = series.len(); + let (Some(p50), Some(p95)) = ( + percentile(series.clone(), 50, 100), + percentile(series, 95, 100), + ) else { + return Outcome::NotObserved(String::from("the arm produced no timing series")); + }; + Outcome::Observed(Reading::WallClock { p50, p95, runs }) +} + +/// Reduce a set of per-run outputs to a byte count, refusing to count an +/// unstable arm. +/// +/// The order is the decision: stability is checked BEFORE the count, because a +/// count over runs that disagree describes none of them. The bash program this +/// generalises asserted the same thing and in the same order. +#[must_use] +pub fn bytes(runs: &[Vec]) -> Outcome { + match stability(runs) { + Outcome::Observed(Reading::Stable) => runs.first().map_or_else( + || Outcome::NotObserved(String::from("the arm produced no runs")), + |first| Outcome::Observed(Reading::Bytes(first.len())), + ), + other => other, + } +} + +/// Whether every run produced identical bytes. +#[must_use] +pub fn stability(runs: &[Vec]) -> Outcome { + let Some(first) = runs.first() else { + return Outcome::NotObserved(String::from("the arm produced no runs")); + }; + let distinct = runs.iter().collect::>().len(); + if runs.iter().all(|run| run == first) { + Outcome::Observed(Reading::Stable) + } else { + Outcome::Observed(Reading::Unstable { + distinct, + runs: runs.len(), + }) + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn a_percentile_over_a_known_series_matches_a_hand_computation() { + // Ten values, so the indices are checkable by eye: p50 takes index + // ceil(9 * 0.5) = 5, and p95 takes ceil(9 * 0.95) = 9. + let series: Vec = (1..=10).map(f64::from).collect(); + assert_eq!(percentile(series.clone(), 50, 100), Some(6.0)); + assert_eq!(percentile(series.clone(), 95, 100), Some(10.0)); + // And it does not depend on the order it was handed. + let mut shuffled = series; + shuffled.reverse(); + assert_eq!(percentile(shuffled, 95, 100), Some(10.0)); + } + + #[test] + fn a_percentile_of_an_empty_series_is_absent_rather_than_zero() { + // The whole `Outcome` distinction, at its source: an empty series has no + // percentile, and returning `0.0` would make an arm that produced + // nothing the fastest one in the comparison. + assert_eq!(percentile(Vec::new(), 50, 100), None); + assert!(matches!(wall_clock(Vec::new()), Outcome::NotObserved(_))); + } + + #[test] + fn an_unstable_arm_is_reported_unstable_rather_than_averaged_away() { + // THE CASE A NAIVE HARNESS GETS WRONG. Reporting the last run, or the + // mean of the counts, would describe none of the runs — and would hide + // exactly the instability the observable exists to surface. + let runs = vec![b"aaa".to_vec(), b"aaaa".to_vec(), b"aaa".to_vec()]; + let outcome = stability(&runs); + assert_eq!( + outcome, + Outcome::Observed(Reading::Unstable { + distinct: 2, + runs: 3 + }) + ); + // And the byte count refuses to be taken over it, in that order. + assert_eq!(bytes(&runs), outcome); + } + + #[test] + fn a_stable_arm_counts_its_bytes() { + let runs = vec![b"hello".to_vec(), b"hello".to_vec()]; + assert_eq!(stability(&runs), Outcome::Observed(Reading::Stable)); + assert_eq!(bytes(&runs), Outcome::Observed(Reading::Bytes(5))); + } + + #[test] + fn an_arm_with_no_runs_is_could_not_look() { + assert!(matches!(stability(&[]), Outcome::NotObserved(_))); + assert!(matches!(bytes(&[]), Outcome::NotObserved(_))); + } + + #[test] + fn a_record_line_is_a_pointer_and_never_a_stream() { + // Rule 4, at the one place a reading becomes text. The arms these + // observables run produce exactly the payload that must not reach a log. + let secret = b"a-distinctive-token".to_vec(); + let line = bytes(&[secret.clone(), secret]).line("baseline", Observable::Bytes); + assert_eq!(line, "arm=baseline observable=bytes count=19"); + assert!(!line.contains("distinctive")); + } + + #[test] + fn isolation_sets_every_state_root_together() { + // A caller that set only the name its own platform reads would leave the + // others ambient, which is the leak this type closes. + let env = Isolation::at("/tmp/state").env(); + for name in ["HOME", "XDG_DATA_HOME", "APPDATA", "LOCALAPPDATA"] { + assert_eq!(env.get(name).map(String::as_str), Some("/tmp/state")); + } + } +} diff --git a/crates/batten/src/cli.rs b/crates/batten/src/cli.rs index 2db82089a..9492e788a 100644 --- a/crates/batten/src/cli.rs +++ b/crates/batten/src/cli.rs @@ -121,6 +121,14 @@ pub enum Command { /// The output format for the spec. format: SpecFormat, }, + /// Fold a run's findings and blind spots into this tool's exit code + /// (CLOUD-1718). + Verdict { + /// How many blocking findings the caller's run produced. + findings: usize, + /// How many subjects the caller could not read. + unjudgeable: usize, + }, /// Report what an agent may do in this repository (CLOUD-1180). ShowAgent { /// Emit the data document rather than pointer lines. @@ -900,6 +908,23 @@ pub struct ExecRequest { /// Whether a bundle keeps going past a failure. `false` when unasked, which /// the committed table may still turn on. pub continue_on_error: bool, + /// The key of this clone's singleton lock to hold for the child's lifetime + /// (CLOUD-1710). `None` is the ordinary unlocked run. + pub lock: Option, + /// A lock path, for a resource the clone does not own. Mutually exclusive + /// with `lock`; naming both is a usage refusal rather than a precedence + /// rule, because the two answer "where should the queue form" differently + /// and silently picking one would serialize the wrong thing. + pub lock_path: Option, + /// How many times to ask for that lock before reporting it held, as the + /// caller typed it. Unparsed for `jobs`' reason: a bad value owes a + /// `UsageError` naming it, and reading it as the default would queue for a + /// length nobody asked for. + pub lock_attempts: Option, + /// What the wait is FOR, for the refusal line. A lock key is a pointer to a + /// file; "the toolchain lock (aarch64-apple-darwin)" is a pointer to the + /// thing a reader has to reason about. + pub lock_label: Option, } /// Subcommands of `lint` — one arm per *kind* of artifact, which is what the @@ -1259,6 +1284,13 @@ pub enum McpCommand { /// The method's arguments as a JSON object. `None` is an empty object. params: Option, }, + /// Record that a client spawned this server, then exec the launch line. + Spawn { + /// The server this launch is for, as the ledger names it. + server: String, + /// The launch line, verbatim. Never empty — the surface requires it. + command: Vec, + }, } /// Subcommands of `target` (CLOUD-1030). @@ -1332,6 +1364,11 @@ pub enum StateCommand { #[derive(Debug, Clone, PartialEq, Eq)] #[non_exhaustive] pub enum RecordCommand { + /// Derive what each bats suite costs and record it where an author reads it. + Suites { + /// Write the corpus to its committed path instead of printing it. + write: bool, + }, /// Record a declared tool row's verdict. Tool { /// The `[[rule.tools]]` id whose verdict is being recorded. @@ -1366,6 +1403,12 @@ pub enum RecordCommand { #[derive(Debug, Clone, PartialEq, Eq)] #[non_exhaustive] pub enum ReceiptCommand { + /// Is the working tree the bytes at HEAD, so a receipt could name them? + /// + /// The cheap end of a pair `record` completes: this is asked before a long + /// gate set runs, and `record`'s own guard closes the window a tree dirtied + /// mid-run opens. + Clean, /// Record that the named check concluded pass against the current HEAD. Record { /// The check whose conclusion is being recorded. @@ -2035,6 +2078,9 @@ fn receipt_of(matches: &ArgMatches) -> Option { // make this whole function answer `None` for it, and a sub-verb that parses // to nothing is a verb that silently does not exist. match name { + // Takes no positional, like `verified` below and for its reason: the + // subject is the tree, which the verb resolves for itself. + "clean" => Some(ReceiptCommand::Clean), "record" => Some(ReceiptCommand::Record { check: matches.get_one::("check")?.clone(), }), @@ -2242,6 +2288,38 @@ fn pr_of(matches: &ArgMatches) -> Option { } } +/// The `exec` arm, lifted out of [`command_of`] so that stays a table. +/// +/// One line per verb is what makes `command_of` readable, and `exec` carries +/// nine flags — the same reason [`ExecRequest`] is a struct rather than nine +/// variant fields, applied one level up. +fn exec_of(matches: &ArgMatches) -> Option { + let command: Vec = matches + .get_many::("command") + .map(|values| values.cloned().collect()) + .unwrap_or_default(); + if command.is_empty() { + return None; + } + Some(Command::Exec(ExecRequest { + command, + capture_only: flag(matches, "capture_only"), + tee: matches.get_flag("tee"), + // Read through the VALUE SOURCE, not the value: clap fills + // `defaulted_enum`'s default in, so `get_one` always answers and a + // config-set default would be overwritten on every call by a flag + // nobody typed. + format: supplied(matches, "format").copied(), + style: supplied(matches, "style").copied(), + jobs: matches.get_one::("jobs").cloned(), + continue_on_error: matches.get_flag("continue_on_error"), + lock: matches.get_one::("lock").cloned(), + lock_path: matches.get_one::("lock_path").cloned(), + lock_attempts: matches.get_one::("lock_attempts").cloned(), + lock_label: matches.get_one::("lock_label").cloned(), + })) +} + fn capture_of(matches: &ArgMatches) -> Option { match matches.subcommand()? { ("show", matches) => Some(CaptureCommand::Show { @@ -2290,6 +2368,21 @@ fn capture_of(matches: &ArgMatches) -> Option { /// `show` is a noun over one leaf today, so an absent subcommand is a usage /// error rather than a default action — `surface::is_noun` marks it and clap /// refuses the bare invocation before this runs. +/// A non-negative count off the command line, absent reading as zero. +/// +/// A value that is not a whole number reads as zero too, and deliberately: this +/// verb's whole job is to be callable from a shell epilogue, where an unset +/// variable expands to the empty string. Refusing that would put the caller back +/// to hand-folding the very case it came here to avoid — and the safe direction +/// is the one that reports LESS, since a miscounted finding is still reported by +/// the caller's own stderr while a usage error replaces the verdict entirely. +fn count_of(matches: &ArgMatches, id: &str) -> usize { + matches + .get_one::(id) + .and_then(|raw| raw.trim().parse::().ok()) + .unwrap_or(0) +} + fn show_of(matches: &ArgMatches) -> Option { match matches.subcommand()? { ("agent", matches) => Some(Command::ShowAgent { @@ -2310,6 +2403,17 @@ fn mcp_of(matches: &ArgMatches) -> Option { method: matches.get_one::("method").cloned()?, params: matches.get_one::("params").cloned(), }), + // The trailing argv is `num_args(1..)` in the surface, so clap has + // already refused an empty launch line: a spawn verb with nothing to + // exec would record a launch that never happened, which is the exact + // false reading CLOUD-714's ledger exists to make impossible. + ("spawn", matches) => Some(McpCommand::Spawn { + server: matches.get_one::("server").cloned()?, + command: matches + .get_many::("command")? + .cloned() + .collect::>(), + }), _ => None, } } @@ -2350,6 +2454,9 @@ fn state_of(matches: &ArgMatches) -> Option { /// inside their arms — [`state_of`]'s shape rather than [`receipt_of`]'s. fn record_of(matches: &ArgMatches) -> Option { match matches.subcommand()? { + ("suites", matches) => Some(RecordCommand::Suites { + write: flag(matches, "write"), + }), ("tool", matches) => Some(RecordCommand::Tool { id: matches.get_one::("id")?.clone(), }), @@ -2435,31 +2542,18 @@ fn command_of((name, matches): (&str, &ArgMatches)) -> Option { // token after `--` is a separate value and the child's argv is the whole // list. An empty list is unreachable — clap enforces `num_args(1..)` — // and is mapped to `None` rather than an empty exec. - "exec" => { - let command: Vec = matches - .get_many::("command") - .map(|values| values.cloned().collect()) - .unwrap_or_default(); - if command.is_empty() { - None - } else { - Some(Command::Exec(ExecRequest { - command, - capture_only: flag(matches, "capture_only"), - tee: matches.get_flag("tee"), - // Read through the VALUE SOURCE, not the value: clap fills - // `defaulted_enum`'s default in, so `get_one` always answers - // and a config-set default would be overwritten on every - // call by a flag nobody typed. - format: supplied(matches, "format").copied(), - style: supplied(matches, "style").copied(), - jobs: matches.get_one::("jobs").cloned(), - continue_on_error: matches.get_flag("continue_on_error"), - })) - } - } + "exec" => exec_of(matches), "capture" => capture_of(matches).map(|command| Command::Capture { command }), "mcp" => mcp_of(matches).map(|command| Command::Mcp { command }), + // Two integers off the command line and a code back: no file, no tree, + // no spawn. A missing count is ZERO rather than a usage error, because + // the caller that has only findings to report should not have to say it + // saw no blind spots — and zero is the honest reading of an absent count, + // not a default standing in for one. + "verdict" => Some(Command::Verdict { + findings: count_of(matches, "findings"), + unjudgeable: count_of(matches, "unjudgeable"), + }), "show" => show_of(matches), "target" => target_of(matches).map(|command| Command::Target { command }), "adjudicate" => matches diff --git a/crates/batten/src/exec.rs b/crates/batten/src/exec.rs index 07e9d9511..f1123cbc9 100644 --- a/crates/batten/src/exec.rs +++ b/crates/batten/src/exec.rs @@ -193,7 +193,7 @@ use std::ffi::OsString; use std::io::{Read, Write}; -use std::path::Path; +use std::path::{Path, PathBuf}; #[expect( clippy::disallowed_types, reason = "stays: `batten exec -- ` IS the spawn — the verb's whole contract is a transparent passthrough of a caller's argv, streams and exit code (CLOUD-285)" @@ -211,6 +211,7 @@ use crate::capture::{self, Stream}; use crate::error::{Passthrough, UsageError}; use crate::exit::ExitCode; use crate::outputs::{self, Hit, OutputPattern}; +use crate::task; /// The exit code a POSIX shell reports for a process killed by a signal. /// @@ -1587,6 +1588,161 @@ pub fn run_in_with_env( report_bundle(&bundle, &outcomes, patterns, settings, report) } +// -- The named lock a child may be held under (CLOUD-1710) ------------------- +// +// `mise-tasks/with-lock.sh` is retired onto this. Everything about WHAT a lock +// means stays in `task.rs` — the atomic create-or-fail, the empty pid file that +// reads as held rather than free, the two-sighting dead-holder reclaim — because +// a second implementation of a lock is the two-authorities class CLOUD-857 +// measured. What lives here is the WRAPPING the shell also had and +// `task::singleton_acquire` does not: hold it across a child, and drop it on +// every exit path. +// +// The mutations target the two decisions a naive lock loses, which are exactly +// the two `tests/with-lock.bats` asserted and which acquire/release cannot tell +// apart. The third targets the verdict, which is the whole product of a wrapper. +//MUTANT exec-lock-empty-holder-read-as-free|s@Claim::Taken | task::Claim::Reclaimed(_)@Claim::Taken | task::Claim::Reclaimed(_) | task::Claim::Held { .. }@|an_empty_holder_file_is_held_not_free +//MUTANT exec-lock-dead-holder-not-reclaimed|s@attempts@1@|a_dead_holder_is_reclaimed_rather_than_waited_out +//MUTANT exec-lock-verdict-discarded|s@let _held =@let _unheld =@|the_wrapped_exit_code_survives_the_lock +//MUTANT-SUITE crates/batten/tests/it/exec_lock.rs + +/// How long the queue is when the caller does not say — the shell's own default +/// (600s at one ask per 100ms), stated as the count it always was. +pub const LOCK_ATTEMPTS_DEFAULT: usize = 6000; + +/// The interval between two asks for a held lock. +/// +/// `mise-tasks/with-lock.sh`'s `sleep 0.1`, carried unchanged. Fixed rather than +/// a flag: `--lock-attempts` is the bound a caller has a reason to move, and a +/// second knob over the same wait would let two callers spell one length two +/// ways. +const LOCK_POLL: std::time::Duration = std::time::Duration::from_millis(100); + +/// Where a lock lives, which decides what it serializes. +/// +/// **Two arms because the resource decides, not the caller's taste.** A lock +/// under `$GIT_DIR` serializes one clone's own work, which is +/// `batten singleton`'s "one task per clone" and what `mise run alive` can +/// enumerate. A lock at a path serializes a resource the clone does not own — +/// a mise install tree, a rustup toolchain — where every checkout on the machine +/// has to form one queue or CLOUD-220 returns by another route. +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum LockPlace { + /// Keyed under `$GIT_DIR`, scoped to this clone. + Key(String), + /// At an explicit path, scoped to whatever lives there. + Path(PathBuf), +} + +/// What `--lock` asked for. +#[derive(Debug, Clone, PartialEq, Eq)] +#[non_exhaustive] +pub struct Lock { + /// Where the lock lives. + pub place: LockPlace, + /// How many times to ask before reporting it held. + pub attempts: usize, + /// What the wait is FOR, for the refusal line. + pub label: String, +} + +/// Holds one clone's singleton lock for as long as this value lives. +/// +/// **A `Drop` rather than a release at each return**, which is the shell's exit +/// trap expressed in the type system: the caller has several exits and one of +/// them is an error path carrying the child's own code, and a release a reader +/// has to find on all of them is one a later edit drops. The shell names the +/// cost of getting this wrong: *"a failure that leaves the lock held wedges +/// every later caller for the full timeout, turning one red run into a stuck +/// repo."* +/// +/// The residual is a `SIGKILL`, where no destructor runs and no trap fired +/// either. That is not a gap this needs to close, and the shell says why: an +/// abandoned lock is reclaimed by the dead-holder sighting in +/// [`task::singleton_queue`], so it costs one ask rather than the whole queue. +#[derive(Debug)] +pub struct HeldLock { + lock: PathBuf, +} + +impl Drop for HeldLock { + fn drop(&mut self) { + task::singleton_release_at(&self.lock); + } +} + +/// What asking for a lock produced. +/// +/// Three arms rather than an `Option`, because "no lock was asked for" and "the +/// lock was asked for and refused" are opposite answers an `Option` would spell +/// the same way — and the second carries a code the first has no business +/// naming. +#[derive(Debug)] +pub enum LockOutcome { + /// No lock was asked for; this is an ordinary unlocked run. + Unlocked, + /// The lock is this process's until the guard drops. + /// + /// The value is never read, and that is the mechanism rather than an + /// oversight: what it does happens in [`HeldLock::drop`], so the field is + /// the lock's lifetime expressed as a binding. + Held(HeldLock), + /// The lock could not be taken, and this is what to exit. + Refused(ExitCode), +} + +/// Take the lock `want` names, queueing behind a live holder. +/// +/// A held lock is a verdict about this clone and reports as +/// [`ExitCode::Violation`]; an unresolvable git dir is could-not-look and +/// reports as [`ExitCode::Internal`] — never "nothing holds it", which is how +/// two of them start. Both readings are [`crate::task::report_claim`]'s rather +/// than restated here, so the lock has one authority on what its answers mean. +/// +/// **Ported onto the engine's exit table, not the corpus's.** The retiring shell +/// spelled a held lock `1`; `crate::exit` spells a verdict `2`, and carrying the +/// inversion across would be the defect CLOUD-1718 exists to name. +/// +/// # Errors +/// +/// Propagates a write failure on the report channel. +pub fn hold(want: Option<&Lock>, err: &mut dyn Write) -> Result { + let Some(want) = want else { + return Ok(LockOutcome::Unlocked); + }; + let pid = std::process::id().to_string(); + let (lock, claim) = match &want.place { + LockPlace::Key(key) => { + let Ok(git_dir) = crate::git::git_dir(Path::new(".")) else { + writeln!( + err, + "::error:: exec: --lock names a lock under the git dir, and this is not a git repository — use --lock-path for a lock outside one" + )?; + return Ok(LockOutcome::Refused(ExitCode::Internal)); + }; + let lock = git_dir.join(task::SINGLETON_DIR).join(key); + let claim = task::singleton_acquire_at(&lock, &pid, LOCK_POLL, want.attempts); + (lock, claim) + } + LockPlace::Path(path) => { + let claim = task::singleton_acquire_at(path, &pid, LOCK_POLL, want.attempts); + (path.clone(), claim) + } + }; + match claim { + task::Claim::Taken | task::Claim::Reclaimed(_) => Ok(LockOutcome::Held(HeldLock { lock })), + // Stdout belongs to the child even on the path where there is no child: + // a caller parsing this command's stdout must not find a lock's + // narration in it, so the success channel is a sink here. + refused => Ok(LockOutcome::Refused(task::report_claim( + &refused, + &want.label, + &mut std::io::sink(), + err, + )?)), + } +} + /// Run a command and report WHICH declared patterns its output matched, /// whatever it exited. /// @@ -1888,6 +2044,86 @@ pub(crate) fn detached(program: &Path, args: &[String], env: &[(&str, &str)]) { drop(builder.spawn()); } +/// Become `argv`: replace this process, or say why it could not be. +/// +/// # Why this is a THIRD shape beside `piped` and `detached` +/// +/// Both of those leave this process alive — one waiting, one not — and there is +/// a caller for whom that is the defect rather than the mechanism. `mcp spawn` +/// records a launch and then hands the process to the server, and CLOUD-714's +/// whole constraint is that nothing survives to supervise it: not a retry, not a +/// keepalive. A shim that recovered from a failed launch would hide the failure +/// it exists to expose. `exec` is what makes that structural instead of a +/// promise — after it there is no process left that could restart anything. +/// +/// So the guarantee is a PROPERTY OF THE SYSCALL, not of the code above it, +/// which is why the launch line goes through here rather than through a spawn a +/// later edit could quietly wrap in a loop. +/// +/// # Placed here rather than widening the adapter table +/// +/// `policy/spawn-adapters.rego` names `exec` as *"the sanctioned child-process +/// boundary"*, and its own prose records why the table is not the place to grow: +/// two rows were once added by a branch whose justification was false, with every +/// sensor green over them. `mcp` calling this reaches the boundary that already +/// exists; adding `mcp` to that set would cost `spawn-widening`'s refusal for a +/// capability the placed adapter already owns. +/// +/// # Errors +/// +/// Only when the replacement fails — a program that is not on `PATH`, or not +/// executable. Success does not return. +#[cfg(unix)] +#[expect( + clippy::disallowed_types, + reason = "stays: becoming the named program IS the verb, on `provision::become_process`'s reading — there is no in-process form of somebody else's binary (CLOUD-320)" +)] +pub fn become_argv(argv: &[String]) -> Result { + use std::os::unix::process::CommandExt as _; + // An empty argv is unreachable — the surface declares the trailing list + // `num_args(1..)` — and is a refusal rather than a panic, because a launch + // line nobody named must never read as a launch that happened. + let Some((program, rest)) = argv.split_first() else { + return Err(UsageError::raise( + "mcp spawn: no launch line to become, so nothing was started".to_owned(), + )); + }; + let mut command = Command::new(program); + command.args(rest); + // `exec` returns only on failure, so reaching the next line IS the error. + let failed = command.exec(); + Err(UsageError::raise(format!( + "mcp spawn: could not become `{program}`: {failed}" + ))) +} + +/// The same, where a process cannot be replaced. +/// +/// **A refusal rather than a spawn**, and the difference from +/// `provision::become_process`'s fallback is deliberate: that one runs the child +/// and takes its status, which is right for a launcher. It is wrong here. The +/// caller's whole guarantee is that no supervisor survives the launch, and a +/// parent waiting on a child IS a supervisor — one that a later edit could give a +/// restart loop without any gate noticing. Refusing says the guarantee cannot be +/// made on this host instead of quietly making a weaker one. +/// +/// Nothing in this repository links on such a host today; the arm exists so the +/// cross-target build is honest about it. +/// +/// # Errors +/// +/// Always. +#[cfg(not(unix))] +pub fn become_argv(argv: &[String]) -> Result { + let _ = argv; + Err(UsageError::raise( + "mcp spawn: this host cannot replace a process, and the launch guarantee is that no \ + supervisor survives it — running the server as a child would be a weaker promise wearing \ + the same name" + .to_owned(), + )) +} + /// Whether a spawn's stderr joins its stdout, and it is per CALL SITE. /// /// **A shared spawn may not decide this, which is what the first attempt got diff --git a/crates/batten/src/exit.rs b/crates/batten/src/exit.rs index 54bda5851..d393cb6d1 100644 --- a/crates/batten/src/exit.rs +++ b/crates/batten/src/exit.rs @@ -121,6 +121,52 @@ impl ExitCode { } } + /// The one fold from "what did the run see" to "what code does it take" + /// (CLOUD-1718). + /// + /// [`verdict`] answers the two-valued question a findings pipeline asks. + /// This answers the THREE-valued one every gate epilogue actually has, and + /// it exists because 82 shell programs were each re-spelling it by hand — + /// on a table that is the INVERSE of this one, where `1` is a violation and + /// `2` is could-not-look. A caller reading the wrong table does not get a + /// worse message, it gets the opposite verdict: a blind spot read as a + /// finding, or a finding read as a blind spot. Those are the two things a + /// completion gate most needs to keep apart, which is what + /// [`crate::findings::Observation::NotObserved`] exists to say. + /// + /// # A BLIND SPOT OUTRANKS A FINDING, and this is the one place that is + /// decided + /// + /// CLOUD-251 settled it and one shell program deliberately inverted it, so + /// two files could disagree about which wins with nothing noticing. The + /// direction is not a preference: a run that could not read part of its + /// subject has an INCOMPLETE answer, and reporting the findings it did reach + /// as though they were the whole answer is the vacuous green this repository + /// refuses everywhere else. A caller told `Violation` fixes what it names + /// and sees green; a caller told `Internal` learns the gate did not run. The + /// findings are on stderr either way, so nothing is lost by ranking the + /// blind spot first. + /// + /// # Never [`Usage`] + /// + /// Its range is `Success`, `Violation` and `Internal` — the three answers + /// ABOUT THE REPOSITORY. `Usage` is a statement about the invocation, and no + /// count of findings or of unreadable subjects can make an invocation + /// malformed. + /// + /// [`verdict`]: ExitCode::verdict + /// [`Usage`]: ExitCode::Usage + #[must_use] + pub const fn combine(findings: usize, unjudgeable: usize) -> Self { + if unjudgeable > 0 { + ExitCode::Internal + } else if findings > 0 { + ExitCode::Violation + } else { + ExitCode::Success + } + } + #[must_use] pub const fn verdict(blocking: bool) -> Self { if blocking { @@ -247,6 +293,50 @@ mod tests { } } + #[test] + fn combine_ranks_a_blind_spot_above_a_finding() { + // THE PRECEDENCE, ASSERTED ONCE (CLOUD-251, CLOUD-1718). One shell + // program deliberately inverted this and nothing could notice, because + // the rule lived in prose in each epilogue rather than in a function. + // A run that could not read part of its subject has an incomplete + // answer, and reporting the findings it did reach as the whole answer is + // the vacuous green refused everywhere else here. + assert_eq!(ExitCode::combine(3, 1), ExitCode::Internal); + assert_eq!(ExitCode::combine(0, 1), ExitCode::Internal); + assert_eq!(ExitCode::combine(3, 0), ExitCode::Violation); + assert_eq!(ExitCode::combine(0, 0), ExitCode::Success); + } + + #[test] + fn combine_never_reports_a_failure_of_battens_own() { + // The range is the guarantee, exactly as it is for `verdict`: no count + // of findings or of unreadable subjects makes an INVOCATION malformed, + // so `Usage` is unreachable through this fold. + for findings in [0_usize, 1, 7] { + for unjudgeable in [0_usize, 1, 7] { + let code = ExitCode::combine(findings, unjudgeable); + assert_ne!(code, ExitCode::Usage, "{findings}/{unjudgeable}"); + } + } + } + + #[test] + fn combine_and_the_shell_convention_disagree_on_every_nonclean_answer() { + // THE INVERSION ITSELF, which is CLOUD-1718's discriminating case. The + // retiring corpus reads `1` as a violation and `2` as could-not-look; + // this table reads `2` as a violation and `3` as could-not-look. A port + // that carried the old fold across would not report a worse code, it + // would report the OPPOSITE meaning — so the two codes are asserted + // different here rather than left to a reader to notice. + let shell_violation = 1; + let shell_could_not_look = 2; + assert_ne!(ExitCode::combine(1, 0).code(), shell_violation); + assert_ne!(ExitCode::combine(0, 1).code(), shell_could_not_look); + // And the collision that makes the inversion dangerous rather than + // merely different: the shell's could-not-look IS this table's violation. + assert_eq!(ExitCode::Violation.code(), shell_could_not_look); + } + #[test] fn every_documented_code_fits_the_process_exit_range() { // The From conversion falls back to Internal only for a code diff --git a/crates/batten/src/land.rs b/crates/batten/src/land.rs index ee84e9c33..d1542a169 100644 --- a/crates/batten/src/land.rs +++ b/crates/batten/src/land.rs @@ -278,14 +278,33 @@ pub fn replay( ) -> Result { let tracking = tracking_ref(reference); advance(root, remote, reference, &tracking)?; + replay_onto(root, &tracking, branch, resolutions) +} - let outcome = gitwrite::rebase_resolving( - root, - &format!("refs/heads/{branch}"), - &tracking, - resolutions, - ) - .with_context(|| format!("land: replay {branch} onto {tracking}"))?; +/// The half of [`replay`] that runs once the tracking ref is in place: rebase, +/// map, record. +/// +/// **Split out so the property can be tested at all** (CLOUD-1708). The store +/// this writes is what `rebase-conflict-stops-the-lap` reads, and nothing drove a +/// REAL conflict through the writer and then read the record — every case +/// constructed a [`Replay`] and handed it to [`record`], which pins the writer +/// but not the path that reaches it. The one thing keeping that test from +/// existing was [`advance`]: it fetches over the forge's HTTP protocol, so an +/// end-to-end case would need a server rather than a repository. +/// +/// The fetch is not what the row doubted, and it is the only inch this leaves +/// unexercised. Everything after it — the rebase, the mapping of each arm, and +/// the record — is one function a test can drive against a real conflicting +/// tree. +pub fn replay_onto( + root: &Path, + tracking: &str, + branch: &str, + resolutions: &[String], +) -> Result { + let outcome = + gitwrite::rebase_resolving(root, &format!("refs/heads/{branch}"), tracking, resolutions) + .with_context(|| format!("land: replay {branch} onto {tracking}"))?; let replayed = match outcome { Rebase::Conflicted { commit, paths } => Replay::Conflicted { commit, paths }, Rebase::Current => Replay::Current, @@ -2026,47 +2045,6 @@ impl Ledger { } } -/// Was this head's CI failure a provisioning transient rather than a verdict? -/// -/// `records` is one line per failed run, as the non-verdict scanner reported them. -/// **A run is absorbed only if EVERY record is a non-verdict**: one line naming a -/// verdict means the branch was judged, and re-running would spend jobs to -/// re-learn a real refusal. -/// -/// `None` for could-not-look, and the causes are deliberately one reading: no -/// failed runs, a scan that produced nothing, a scan that answered with a -/// verdict, and a record this reader does not recognise. A caller cannot act -/// differently on which, and inventing a distinction would invite one to. -/// -/// **AN UNRECOGNISED RECORD IS COULD-NOT-LOOK, NOT AN ABSENT VERDICT.** The -/// filter used to keep the `nonverdict` lines and DROP everything else, so a -/// scanner error, a truncated record or a shape added later read as *every -/// record is a non-verdict* — the permissive answer, which re-runs the matrix on -/// a head that may well have been judged. Absorbing is the expensive direction, -/// so the reading that cannot be justified must not reach it. -#[must_use] -pub fn absorbed(records: &[String]) -> Option> { - let lines: Vec<&str> = records - .iter() - .flat_map(|record| record.lines()) - .map(str::trim) - .filter(|line| !line.is_empty()) - .collect(); - if lines.is_empty() { - return None; - } - if lines.iter().any(|line| line.starts_with("verdict")) { - return None; - } - // EVERY line must be one this reader knows. `nonverdict` does not begin with - // `verdict`, so the two prefixes partition cleanly and anything outside the - // pair is a record nobody here can classify. - if !lines.iter().all(|line| line.starts_with("nonverdict")) { - return None; - } - Some(lines.iter().map(|line| (*line).to_owned()).collect()) -} - // --------------------------------------------------------------------------- // CLOUD-900 / CLOUD-1338: abandoning the matrix a red check made worthless. // --------------------------------------------------------------------------- @@ -2515,54 +2493,6 @@ pub fn buys_a_matrix( } } -/// The runs on a head that failed, as ids. -/// -/// **NO PAGE SIZE, deliberately, and the predecessor says why in a sentence -/// worth carrying:** `tests/land.bats`'s keyed-verdict sensor asserts the lander -/// carries no windowed page size, because the fast-forward verdict must be found -/// by its KEY rather than by a window. This is a different endpoint that needs -/// none — a head sha's failed runs are a handful — so the sensor stays exact -/// instead of being spelled past. -/// -/// `None` is could-not-look, and the caller reads it as *not absorbed*: a -/// transient is a claim about the runs, and a claim over a list nobody could -/// read is not one. -#[must_use] -pub fn failed_runs(repo: &str, sha: &str) -> Option> { - let answer = crate::rest::get( - &format!("repos/{repo}/actions/runs?head_sha={sha}&status=failure"), - None, - )?; - let document = serde_json::from_str::(&answer.body).ok()?; - let runs = document.get("workflow_runs")?.as_array()?; - let ids: Vec = runs - .iter() - .filter_map(|run| { - let id = run.get("id")?; - id.as_u64() - .map(|found| found.to_string()) - .or_else(|| id.as_str().map(str::to_owned)) - }) - .collect(); - // AN EMPTY LIST IS NOT AN ANSWER HERE. The predecessor returns non-zero on - // one, and it is right to: "no failed runs" cannot support "the failure was - // a transient", because there is no failure to have been one. - (!ids.is_empty()).then_some(ids) -} - -/// Ask the forge to re-run one run's failed jobs. -/// -/// `false` where the forge refused. **Reported rather than swallowed**, unlike -/// the tap: the predecessor dies here with a remedy naming the exact command, -/// because a lap that believed it re-ran and did not would wait forever for a -/// run nobody started. -#[must_use] -pub fn rerun_failed(repo: &str, run: &str) -> bool { - crate::rest::post(&format!( - "repos/{repo}/actions/runs/{run}/rerun-failed-jobs" - )) -} - #[cfg(test)] mod lap_tests { use super::{Progress, Step, progress}; @@ -3251,47 +3181,6 @@ mod tests { assert_eq!(ledger.laps, 0); } - /// **The discriminating pair: every record a non-verdict is absorbed, one - /// verdict is not.** - /// - /// A run that reached a verdict was a judgement on this branch, and - /// re-running it would spend jobs to re-learn a real refusal. - #[test] - fn a_failure_before_any_verdict_is_absorbed_and_one_after_is_not() { - let absorbed_runs = absorbed(&[ - String::from("nonverdict 111 provision\n"), - String::from("nonverdict 222 checkout\n"), - ]); - assert_eq!( - absorbed_runs, - Some(vec![ - String::from("nonverdict 111 provision"), - String::from("nonverdict 222 checkout"), - ]), - "neither run reached a verdict, so neither judged the branch" - ); - - assert_eq!( - absorbed(&[ - String::from("nonverdict 111 provision\n"), - String::from("verdict 222 test-failed\n"), - ]), - None, - "ONE verdict means the branch was judged; absorbing the pair would \ - re-run a real refusal" - ); - } - - /// Could-not-look is one reading, and its three causes are deliberately - /// indistinguishable: no failed runs, an empty scan, and a scan that - /// answered nothing. No caller can act differently on which. - #[test] - fn an_empty_scan_is_could_not_look_rather_than_an_absorbed_transient() { - assert_eq!(absorbed(&[]), None); - assert_eq!(absorbed(&[String::new()]), None); - assert_eq!(absorbed(&[String::from(" \n\n")]), None); - } - /// **THE FIXTURE NAMES ARE GENERIC, and that is rule 1 rather than taste.** /// /// The first draft spelled these as this repository's own workflow paths and diff --git a/crates/batten/src/lib.rs b/crates/batten/src/lib.rs index a9b933c08..74534ce4a 100644 --- a/crates/batten/src/lib.rs +++ b/crates/batten/src/lib.rs @@ -12,6 +12,7 @@ pub mod action; pub mod admission; pub mod advisory; pub mod agent; +pub mod arm; pub mod attribution; pub mod baseline; pub mod bot; @@ -130,6 +131,8 @@ pub mod startup; pub mod state; pub mod stop; pub mod store; +/// The per-suite cost corpus, derived from the report the runner already wrote. +pub mod suites; pub mod surface; /// What a long-running task is doing, recorded where it can be read without a log. pub mod task; @@ -233,6 +236,14 @@ pub fn run(cli: Cli, mode: Mode, out: &mut dyn Write, err: &mut dyn Write) -> Re ), Some(Command::Config { command }) => run_config(&command, &overrides, mode, out, err), Some(Command::Spec { format }) => run_spec(format, out), + // CLOUD-1718. No config is resolved and nothing is read: the §8 chain + // has nothing to contribute to a fold over two integers, and threading it + // through would make an unreadable config able to change a verdict this + // verb computes without one. + Some(Command::Verdict { + findings, + unjudgeable, + }) => Ok(exit::ExitCode::combine(findings, unjudgeable)), Some(Command::ShowAgent { json }) => run_show_agent(json, &overrides, out), Some(Command::Doctor { command }) => run_doctor(&command, out), // `init` reads no config — it is the verb that exists because there is @@ -1738,11 +1749,33 @@ fn run_mcp( out: &mut dyn Write, err: &mut dyn Write, ) -> Result { + // RECORD, THEN BECOME. The two statements are the whole verb, and the order + // is the diagnosis: a connect timeout WITH a matching record means + // spawned-and-unresponsive, one WITHOUT means never spawned, and nothing in + // this repository could tell those apart before CLOUD-714. Recording after + // the exec is not an option — there is no after. + // + // Nothing is written to either channel here. `out` and `err` are untouched + // because stdout is the MCP transport, and one stray byte on it corrupts the + // JSON-RPC stream and takes the server down looking exactly like the bug. + if let cli::McpCommand::Spawn { server, command } = command { + mcp::record_spawn(Path::new("."), server); + // Returns only on failure; success replaces this process. + return exec::become_argv(command).map(|never| match never {}); + } let cli::McpCommand::Call { server, method, params, - } = command; + } = command + else { + // Unreachable: the enum has two variants and the first is handled above. + // A refusal rather than a panic, on this module's own rule that an + // impossible parse is still answered rather than aborted. + return Err(UsageError::raise( + "mcp: no sub-verb resolved from this invocation".to_owned(), + )); + }; let repo = git::repo_root(Path::new("."))?; let resolved = resolve::resolve(Path::new("."), overrides)?; let config = resolved.mcp.clone().unwrap_or_default(); @@ -3118,6 +3151,7 @@ fn run_receipt( err: &mut dyn Write, ) -> Result { match command { + ReceiptCommand::Clean => receipt::run_clean(out, err), ReceiptCommand::Record { check } => receipt::run_record(&check, mode, err), ReceiptCommand::Status { check, key, json } => receipt::run_status(&check, key, json, out), ReceiptCommand::Verified => receipt::run_verified(out), @@ -8586,11 +8620,40 @@ fn run_land_replay( ) -> Result { match land::replay(root, url, reference, branch, resolve)? { land::Replay::Conflicted { commit, paths } => { + // THE STOP NAMES ITS ROUTE, which this line did not (CLOUD-1586's + // mechanism, CLOUD-1050's rule). It reported the conflict and the + // first path and stopped there — so a reader with no `--resolve` in + // their head reached for `git rebase --continue`, which cannot + // exist here, and then for a hand rebase, which + // `rebase-not-hand-stepped` denies. Measured on this very branch: a + // session concluded the loop was defective and was one step from + // cherry-picking around it, which would have completed the replay + // and written no lap record at all. + // + // EVERY PATH, not the first: the caller has to author a resolution + // for each one, and a count they cannot enumerate is not actionable. + // Paths are pointers, so rule 4 is untouched — no hunk, no marker, + // no content. + writeln!( + out, + "land: replay of {branch} onto {reference} conflicted at {commit} in {} path(s):", + paths.len() + )?; + for path in &paths { + writeln!(out, " {path}")?; + } writeln!( out, - "land: replay of {branch} onto {reference} conflicted at {commit} in {} path(s); first is {}", - paths.len(), - paths.first().map_or("-", String::as_str) + "land: the replay is STATELESS — nothing is half-replayed, so there is no rebase in \ + progress and nothing to `--continue`. Merge each path in the worktree and name it: \ + `batten land replay {reference} --resolve `. The whole range re-runs from its \ + base, so the merged bytes are supplied up front." + )?; + writeln!( + out, + "land: a path that conflicts at MORE THAN ONE commit needs one entry per conflict, \ + in order — `--resolve =` names bytes authored for a single merge, where \ + a bare `--resolve ` spends the worktree's copy on the first one only." )?; Ok(ExitCode::Violation) } @@ -15413,12 +15476,73 @@ fn run_exec( })?; } settings.continue_on_error = settings.continue_on_error || request.continue_on_error; + // THE LOCK IS HELD ACROSS THE CHILD, AND THE GUARD IS WHAT MAKES THAT TRUE + // ON EVERY RETURN (CLOUD-1710). A wrapped command that fails comes back as + // `Err(Passthrough)`, so releasing after the call would leak the lock on + // exactly the path that matters most — the shell named the same defect and + // used a trap: *"a failure that leaves the lock held wedges every later + // caller for the full timeout, turning one red run into a stuck repo."* + let _held = match exec::hold(exec_lock(request)?.as_ref(), err)? { + exec::LockOutcome::Refused(code) => return Ok(code), + taken => taken, + }; // The report goes to the ERROR channel, never `out`: stdout belongs to the // wrapped command (CLOUD-285), so a pointer line there would corrupt a // document the caller may be parsing. exec::run_with(&request.command, &patterns, &settings, err) } +/// What `--lock` asked for, as the value [`exec::hold`] takes. +/// +/// Parsed HERE rather than in `exec.rs`, for `--jobs`' reason and in `--jobs`' +/// place: a bad value owes a `UsageError` naming what was wrong with it, and +/// the boundary is where this crate turns typed argv into declared values. +/// +/// # Errors +/// +/// [`UsageError`] when `--lock-attempts` is not a positive whole number. +fn exec_lock(request: &cli::ExecRequest) -> Result> { + let place = match (request.lock.as_deref(), request.lock_path.as_deref()) { + (None, None) => return Ok(None), + (Some(key), None) => exec::LockPlace::Key(key.to_owned()), + (None, Some(path)) => exec::LockPlace::Path(std::path::PathBuf::from(path)), + // A refusal rather than a precedence rule: the two answer "where should + // the queue form" differently, and silently picking one would serialize + // the wrong thing — which is the failure mode a lock exists to prevent. + (Some(_), Some(_)) => { + return Err(UsageError::raise( + "exec: --lock and --lock-path name two different queues; give one", + )); + } + }; + let attempts = match request.lock_attempts.as_ref() { + Some(raw) => raw + .trim() + .parse::() + .ok() + .filter(|n| *n > 0) + .ok_or_else(|| { + UsageError::raise(format!( + "exec: --lock-attempts wants a positive whole number, not `{raw}`" + )) + })?, + None => exec::LOCK_ATTEMPTS_DEFAULT, + }; + let named = match &place { + exec::LockPlace::Key(key) => key.clone(), + exec::LockPlace::Path(path) => path.display().to_string(), + }; + Ok(Some(exec::Lock { + place, + attempts, + // The caller names what the wait was FOR, and the key is the fallback + // rather than the message: `the toolchain lock (aarch64-apple-darwin)` + // is a pointer to the thing a reader has to reason about, where a bare + // key is a pointer to a directory. + label: request.lock_label.clone().unwrap_or(named), + })) +} + fn load_exec_settings( overrides: &Overrides, ) -> Result<(Vec, exec::ExecConfig)> { diff --git a/crates/batten/src/mcp.rs b/crates/batten/src/mcp.rs index 30cb98124..f2a4a2940 100644 --- a/crates/batten/src/mcp.rs +++ b/crates/batten/src/mcp.rs @@ -1747,6 +1747,112 @@ const SESSION_HEADER: &str = "mcp-session-id"; /// envelope, or when the envelope carries an `error`. A server's refusal is a /// fact about the call rather than a verdict about the repository, so it never /// reaches the policy code — see `crate::exit`'s table. +/// The window in which two spawns count as concurrent, in seconds. +/// +/// The one number here that is a judgement rather than a reading, and it is the +/// retired shell's: ten seconds is a third of the client's connect budget — long +/// enough that a burst of servers started back to back all land inside it, short +/// enough that the previous session's launches do not. +const SPAWN_WINDOW: u64 = 10; + +/// The per-clone spawn ledger's name under the git directory. +/// +/// **The path is a CONTRACT with a reader this module does not own.** +/// `mcp-attach-check` opens `$GIT_DIR/batten-mcp-spawns` by that name and +/// compares its lines against the client's own log, which is the comparison +/// CLOUD-714 exists to make possible. So the retirement preserves the location +/// and the tab-separated layout exactly; a port that "improved" either would +/// silently disconnect the two halves of the diagnosis. +const SPAWN_LEDGER: &str = "batten-mcp-spawns"; + +/// One ledger line: when, which server, which pid, the load, and how many +/// siblings were already inside the window. +/// +/// # Why the two extra fields are not decoration +/// +/// CLOUD-714's only surviving correlate is that all three failures happened +/// during a multi-server startup burst, while every successful isolated +/// replication was a lone launch on an idle session. That is n=3 with no +/// mechanism attached — a hypothesis — and the load average and the sibling count +/// are what let the NEXT occurrence decide it. Both are free at spawn and +/// unrecoverable afterwards. +/// +/// # Every failure is silent, and that is inherited rather than chosen +/// +/// A ledger that cannot be written must never be the reason a server does not +/// start, so every fallible step degrades and the launch proceeds. The load field +/// renders `?` rather than a number when it cannot be read: a zero there would +/// read as *the machine was idle*, which is precisely the claim the hypothesis +/// above turns on. +fn spawn_record( + root: &Path, + server: &str, + now: u64, + pid: u32, +) -> Option<(std::path::PathBuf, String)> { + // Outside a checkout there is nowhere per-clone to keep the ledger, and + // inventing a path under the temp directory would put it somewhere no gate + // reads. So this is a launch that records nothing and still launches. + let git_dir = crate::git::git_dir(root).ok()?; + let ledger = git_dir.join(SPAWN_LEDGER); + let load = std::fs::read_to_string("/proc/loadavg") + .ok() + .and_then(|text| text.split_whitespace().next().map(str::to_owned)) + .unwrap_or_else(|| String::from("?")); + // Counted BEFORE the append, so a launch never counts itself. + let siblings = siblings_within(&ledger, now); + Some(( + ledger, + format!("{now}\t{server}\t{pid}\t{load}\t{siblings}\n"), + )) +} + +/// How many entries the ledger already holds inside [`SPAWN_WINDOW`] of `now`. +/// +/// Counted from the ledger itself — entries that are already there — so there is +/// no process-table walk and no new dependency. A ledger that will not read is +/// ZERO rather than a refusal, for the module's silence rule: the count is a +/// hypothesis-testing field, and failing a launch over it would be the tail +/// wagging the dog. +/// +/// A timestamp in the FUTURE is skipped rather than counted. Clocks move +/// backwards on resume, and a negative age counted as a sibling would inflate +/// exactly the field the hypothesis reads. +fn siblings_within(ledger: &Path, now: u64) -> usize { + let Ok(text) = std::fs::read_to_string(ledger) else { + return 0; + }; + text.lines() + .filter_map(|line| line.split('\t').next()) + .filter_map(|stamp| stamp.parse::().ok()) + .filter(|stamp| *stamp <= now && now - *stamp <= SPAWN_WINDOW) + .count() +} + +/// Append the spawn record, then say nothing. +/// +/// **Nothing reaches stdout, ever, and here that is not hygiene.** STDOUT IS THE +/// MCP TRANSPORT: one stray byte corrupts the JSON-RPC stream and takes the +/// server down in a way that looks exactly like the bug this records. So the +/// record is a file and the return is `()` — there is no channel for a caller to +/// act on the difference, and no verdict a failed append should produce. +pub fn record_spawn(root: &Path, server: &str) { + let now = std::time::SystemTime::now() + .duration_since(std::time::UNIX_EPOCH) + .map_or(0, |since| since.as_secs()); + let Some((ledger, line)) = spawn_record(root, server, now, std::process::id()) else { + return; + }; + // `>>` on a short line is atomic enough for concurrent appends: the record is + // well under `PIPE_BUF` and every writer opens in append mode. + let appended = std::fs::OpenOptions::new() + .create(true) + .append(true) + .open(&ledger) + .and_then(|mut file| std::io::Write::write_all(&mut file, line.as_bytes())); + drop(appended); +} + pub fn dispatch( wiring: &Wiring, method: &str, diff --git a/crates/batten/src/mutate.rs b/crates/batten/src/mutate.rs index 2b48c1cf6..6ff20fc7d 100644 --- a/crates/batten/src/mutate.rs +++ b/crates/batten/src/mutate.rs @@ -1128,17 +1128,43 @@ struct Selection { } /// Run a gate's suite filtered to `want`, inside the staged tree. +/// Run one declared [`crate::arm::Arm`], which is the harness's unit of work +/// (CLOUD-1714). +/// +/// The adapter is one line of destructuring because `spawn` already takes +/// exactly what an arm carries: an arm's `argv` is program-then-arguments, and +/// splitting it here is what keeps the declaration a table rather than four +/// positional parameters at every call site. An arm with an EMPTY argv names no +/// program, which is a could-not-look rather than a run of nothing. +fn spawn_arm(arm: &crate::arm::Arm, env: &[(String, String)]) -> Result { + let (program, args) = arm + .argv + .split_first() + .ok_or_else(|| anyhow::anyhow!("mutate: arm {} names no program.", arm.id))?; + spawn(&arm.cwd, program, args, env) +} + fn run_suite(staged: &Staged, root: &Path, suite: &Suite, want: &str) -> Result { let env = suite_env(root); match suite { Suite::Bats(path) => { - let args = vec![String::from("--filter"), want.to_owned(), path.to_owned()]; - let ran = spawn( - staged.dir(), - &root.join(BATS).to_string_lossy(), - &args, - &env, - )?; + // DECLARED AS AN ARM (CLOUD-1714), which is what makes this module + // an INSTANCE of the harness rather than a second copy of it. The + // arm carries what it takes to run the thing once — where, what, + // and under which environment — and `arm::Outcome` carries the + // distinction the `selected == 0` reading below already draws: a + // suite that selected no case has not passed, it has not been + // looked at. + let arm = crate::arm::Arm { + id: format!("bats:{want}"), + cwd: staged.dir().to_path_buf(), + argv: std::iter::once(root.join(BATS).to_string_lossy().into_owned()) + .chain([String::from("--filter"), want.to_owned(), path.to_owned()]) + .collect(), + stdin: None, + env: env.iter().cloned().collect(), + }; + let ran = spawn_arm(&arm, &env)?; Ok(Selection { selected: tap_lines(&ran.output), ok: ran.ok, @@ -1154,8 +1180,19 @@ fn run_suite(staged: &Staged, root: &Path, suite: &Suite, want: &str) -> Result< // that match nothing is their startup. A target selecting no case // is not a pass either: `selected` stays 0 and the caller reports // `names-no-case`, which is a could-not-look. - let args = vec![String::from("test"), String::from("--"), want.to_owned()]; - let ran = spawn(staged.dir(), "cargo", &args, &env)?; + let arm = crate::arm::Arm { + id: format!("cargo:{want}"), + cwd: staged.dir().to_path_buf(), + argv: vec![ + String::from("cargo"), + String::from("test"), + String::from("--"), + want.to_owned(), + ], + stdin: None, + env: env.iter().cloned().collect(), + }; + let ran = spawn_arm(&arm, &env)?; Ok(Selection { selected: libtest_lines(&ran.output), ok: ran.ok && !ran.output.contains("error: could not compile"), diff --git a/crates/batten/src/perf.rs b/crates/batten/src/perf.rs index e9b372cf9..bd15055bf 100644 --- a/crates/batten/src/perf.rs +++ b/crates/batten/src/perf.rs @@ -987,12 +987,22 @@ fn hyperfine( /// `--shell=none` runs argv directly, so it can only come from a prefix. The /// extra exec is identical on both arms and divides out of the ratio. fn state_prefixed(state: &str, argv: &[String]) -> Vec { - let mut out = vec![ - String::from("env"), - format!("XDG_DATA_HOME={state}"), - format!("APPDATA={state}"), - format!("LOCALAPPDATA={state}"), - ]; + // THE ISOLATION IS `arm::Isolation`'s, and stating it there is what makes it + // behaviour rather than setup (CLOUD-1714): every state-writing name is set + // together, so a caller cannot point one at a scratch root and leave the + // rest ambient. This wrapper is the `env`-prefix spelling the benchmarking + // tool needs, over that one declaration. + let mut out = vec![String::from("env")]; + for (name, value) in crate::arm::Isolation::at(state).env() { + if name == "HOME" { + // The benchmarking arms deliberately keep the ambient home: they + // measure a repository checkout, and relocating `HOME` would move + // the toolchain out from under the binary being measured. The other + // three are what carry per-arm state. + continue; + } + out.push(format!("{name}={value}")); + } out.extend(argv.iter().cloned()); out } @@ -1027,32 +1037,24 @@ fn record(arm: &'static str, id: &str, result: &serde_json::Value) -> Result, + times: Vec, reported_mean: Option, ) -> Result { if times.is_empty() { bail!("perf: the {id} {arm} arm carried no times."); } - times.sort_by(f64::total_cmp); - let n = times.len(); - #[expect( - clippy::cast_precision_loss, - reason = "a run count is a small integer and this is an index computation, not a measurement" - )] - let last = (n - 1) as f64; - #[expect( - clippy::cast_possible_truncation, - clippy::cast_sign_loss, - reason = "both products are within [0, n-1] by construction, so the cast cannot truncate meaningfully or go negative" - )] - let i50 = (last * 0.5).floor() as usize; - #[expect( - clippy::cast_possible_truncation, - clippy::cast_sign_loss, - reason = "both products are within [0, n-1] by construction, so the cast cannot truncate meaningfully or go negative" - )] - let i95 = (last * 0.95).ceil() as usize; + + // THE PERCENTILE REDUCTION IS `arm`'s, and this is one of the four copies + // CLOUD-1714 collapsed into it. `arm::percentile` sorts the series itself, + // so the shape where an unsorted series reaches a quantile is not + // representable here any more. + let (Some(p50), Some(p95)) = ( + crate::arm::percentile(times.clone(), 50, 100), + crate::arm::percentile(times.clone(), 95, 100), + ) else { + bail!("perf: the {id} {arm} arm carried no times."); + }; #[expect( clippy::cast_precision_loss, @@ -1063,8 +1065,8 @@ fn summarise( Ok(Record { arm, path: id.to_owned(), - p50: times[i50] * 1000.0, - p95: times[i95.min(n - 1)] * 1000.0, + p50: p50 * 1000.0, + p95: p95 * 1000.0, mean: mean * 1000.0, runs: n, }) diff --git a/crates/batten/src/receipt.rs b/crates/batten/src/receipt.rs index a26486190..d921acfde 100644 --- a/crates/batten/src/receipt.rs +++ b/crates/batten/src/receipt.rs @@ -1397,6 +1397,156 @@ pub fn rfc3339_utc(unix_seconds: u64) -> String { format!("{}T{hour:02}:{minute:02}:{second:02}Z", date.text()) } +/// How many paths differ from HEAD, or [`None`] when the tree matches it. +/// +/// **THE ONE AUTHORITY, and both callers ask it rather than re-deriving it.** +/// [`tree_is_head`] guards the write and [`run_clean`] answers the question on +/// its own; a second count would be a second reading of `git status` that could +/// disagree with the first over an ignored path or an untracked directory. +/// +/// `Err` is could-not-look and is kept distinct from `Ok(None)` — "I could not +/// read the tree" and "the tree is clean" are opposite answers, and the callers +/// take opposite arms on the difference. +/// +/// # Errors +/// +/// As [`crate::git::uncommitted`]: when this is not inside a repository. +fn tree_state() -> Result> { + let dirty = crate::git::uncommitted(Path::new("."))?; + Ok((dirty > 0).then_some(dirty)) +} + +/// `receipt clean`: the precondition asked on its own. +/// +/// The CHEAP end of the pair [`tree_is_head`] completes. `verify:gated` asks this +/// in `depends` so a dirty tree fails in seconds rather than after the gate set's +/// full cost — and `depends` finishes before the body starts, so a tree dirtied +/// mid-run is invisible to it. That window is what the guard inside `record` +/// closes. +/// +/// # Errors +/// +/// An internal error (exit `3`) when the tree cannot be read at all. Asked +/// directly, could-not-look is an answer worth reporting rather than one to +/// degrade past — which is where this differs from [`tree_is_head`], whose caller +/// must not fail a lap over its own bookkeeping. +pub fn run_clean(out: &mut dyn Write, err: &mut dyn Write) -> Result { + let Ok(state) = tree_state() else { + writeln!( + err, + "::error:: receipt clean: the working tree state cannot be read, so whether a receipt \ + would name the right bytes is unknown. That is not a clean tree." + )?; + return Ok(ExitCode::Internal); + }; + // HEAD ALONE, never `repo_facts`, and the difference is a real one rather + // than a shortcut: that helper also resolves `origin/main`, because a + // RECORDED receipt names the trunk it was taken against. This verb compares + // the tree to HEAD and nothing else, so requiring a remote would make it + // refuse in a fresh clone that has none — a could-not-look manufactured out + // of a fact the question does not use. + let Ok(Some(head)) = crate::git::resolve_ref(Path::new("."), "HEAD") else { + writeln!( + err, + "::error:: receipt clean: HEAD does not resolve, so there is no commit for a receipt \ + to name" + )?; + return Ok(ExitCode::Internal); + }; + let head = head.get(..8).unwrap_or(&head).to_owned(); + let Some(dirty) = state else { + writeln!(out, "receipt clean: working tree matches HEAD {head}")?; + return Ok(ExitCode::Success); + }; + writeln!( + err, + "::error:: receipt clean: the working tree differs from HEAD {head} in {dirty} path(s), so \ + a receipt keyed to that commit would attest bytes no commit contains. Commit the work \ + (you are pre-authorised to), stash it, or run the long task in a separate worktree — then \ + re-run." + )?; + Ok(ExitCode::Usage) +} + +/// Refuse to key a receipt to HEAD when the tree is not HEAD (CLOUD-193), +/// ported off `mise-tasks/tree-clean.sh` under CLOUD-1753. +/// +/// # The assumption this enforces was load-bearing and unenforced +/// +/// CLOUD-193 moved `verify`'s verdict off the exit code and onto a receipt *"keyed +/// to the exact HEAD it validated"*. The mechanism is sound and it rests on an +/// assumption nothing checked: that the bytes verified ARE the bytes at HEAD. +/// They need not be. `cargo`, `hk` and `zizmor` all read the WORKING TREE, and +/// this function keys the claim to HEAD. +/// +/// # The direction that matters is the SILENT one +/// +/// Measured 2026-08-09: a backgrounded `land` compiled a mid-edit snapshot and +/// died on a `non-exhaustive patterns` error for code the commit did not contain. +/// That direction is loud and self-correcting. The mirror is not — a dirty tree +/// that PASSES writes a receipt for HEAD, `verified` matches it, `ready-guard` +/// accepts it, the branch is readied, and CI runs the commit alone, which was +/// never the thing that passed. +/// +/// Uncommitted work is not exotic here: backgrounding the slow path is mandated, +/// so a long `verify` running while the session edits the next row in the same +/// worktree is the DESIGNED workflow. +/// +/// # Why a precondition on the write rather than a `[[rule]]` +/// +/// `batten.toml` twice cites *"the scoping defect CLOUD-1164 records for +/// `tree-clean`"* as the anti-pattern: a `Tree`-scoped row runs on every `batten +/// check`, and `pre-commit` runs over a tree that is dirty by definition, so a +/// row there would refuse every commit. The retiring program said the same thing +/// in its own words — *"NOT wired into the hk gate, deliberately"*. +/// +/// The receipt is written HERE, by this verb, so this is the one place the +/// question is both askable and answerable. It is not `[[mint]]` either: a mint's +/// condition vocabulary is JSON paths into a tool result, and no receipt reaches +/// it. +/// +/// # Untracked files are dirty, decided rather than omitted +/// +/// [`crate::git::uncommitted`] counts staged, unstaged and untracked alike, and +/// the third is the one a `diff HEAD` predicate misses. The gap is not +/// theoretical: `cargo test` autodiscovers test targets and the bats suite globs +/// `tests/*.bats`, so a brand-new untracked file is compiled and run by `verify` +/// with ZERO tracked-file change. A receipt written after that attests a pass over +/// bytes no commit contains, which is the whole failure. Ignored paths are +/// excluded structurally rather than tuned out, so `target/` and the worktree +/// directories are outside the judgement by construction. +/// +/// # Errors +/// +/// A [`UsageError`] (exit `1`) when the tree differs from HEAD — a statement +/// about this invocation, never a policy verdict, so never exit `2`. +fn tree_is_head(facts: &RepoFacts, err: &mut dyn Write) -> Result<()> { + // Could-not-look degrades to permitting the write rather than refusing it. + // `repo_facts` has already resolved the repository, so a failure here is the + // status read alone — and a receipt verb that refused because it could not + // run `status` would fail the lap for its own bookkeeping. `receipt clean` + // takes the opposite arm on the same fact, and the difference is the caller: + // asked directly, could-not-look is an answer worth reporting. + let Ok(Some(dirty)) = tree_state() else { + return Ok(()); + }; + // A COUNT AND THE HEAD, never a path list and never a diff (rule 4). The + // remedy names all three routes, because "commit it" is not always the one + // the author wants and a refusal naming one route is how a gate earns a + // bypass. + let head = facts.head.get(..8).unwrap_or(&facts.head); + writeln!( + err, + "::error:: receipt record: the working tree differs from HEAD {head} in {dirty} path(s), \ + so a receipt keyed to that commit would attest bytes no commit contains. Commit the work \ + (you are pre-authorised to), stash it, or run the long task in a separate worktree — then \ + re-run. No receipt is written." + )?; + Err(UsageError::raise(format!( + "receipt record: {dirty} uncommitted path(s), so no receipt is written" + ))) +} + /// Record that `check` concluded pass against the current HEAD. /// /// Writes the canonical statement into the out-of-tree state dir and the @@ -1415,6 +1565,7 @@ pub fn rfc3339_utc(unix_seconds: u64) -> String { pub fn run_record(check: &str, mode: Mode, err: &mut dyn Write) -> Result { validate_check_name(check)?; let facts = repo_facts()?; + tree_is_head(&facts, err)?; let policy = git::show(Path::new("."), "HEAD", config::CONFIG_FILE) .map_err(|_| { UsageError::raise( diff --git a/crates/batten/src/record.rs b/crates/batten/src/record.rs index eff7efc05..85504d895 100644 --- a/crates/batten/src/record.rs +++ b/crates/batten/src/record.rs @@ -188,6 +188,47 @@ pub fn run_forge(reference: &str, _overrides: &Overrides) -> Result { Ok(ExitCode::Success) } +/// Derive the per-suite cost corpus and print it, or write it (CLOUD-352). +/// +/// The one `record` leaf whose store is a COMMITTED file rather than the +/// out-of-tree record tree, and the reason is the reader: the others are read by +/// a gate, this one by a person deciding whether the suite they are about to add +/// a case to is expensive. A record nobody opens answers nothing. +/// +/// # Errors +/// +/// A [`UsageError`] when the report is absent, unreadable, carries no suite, or +/// names a suite this tree does not track — see [`crate::suites::derive`], where +/// all four are could-not-look and none is an empty corpus. An internal error +/// when the corpus cannot be written. +pub fn run_suites(write: bool) -> Result { + let root = git::repo_root(Path::new("."))?; + let root = Path::new(&root); + // THE TRACKED SET FROM GIT, never a directory walk: an untracked scratch file + // beside the suites is not something the corpus should have to carry, and a + // walk would put it there. + let tracked = crate::git::tracked_paths(root)? + .into_iter() + .filter(|path| path.starts_with("tests/") && path.ends_with(".bats")) + .collect(); + let (rows, text) = crate::suites::derive(root, &tracked)?; + if !write { + print!("{text}"); + return Ok(ExitCode::Success); + } + // `store` rather than a second write path, and it is the same helper the + // other three leaves use: one place that creates the directory and reports + // which write failed. The corpus lives in the tree rather than under + // `$GIT_DIR`, and that is the only thing this leaf does differently. + store(&crate::suites::corpus_path(root), &text)?; + eprintln!( + "record suites: {} suite(s), written to {}", + rows.len(), + crate::suites::CORPUS + ); + Ok(ExitCode::Success) +} + /// Dispatch the `record` verbs. /// /// # Errors @@ -197,6 +238,7 @@ pub fn run_forge(reference: &str, _overrides: &Overrides) -> Result { /// an internal error when the store cannot be written. pub fn run(command: crate::cli::RecordCommand, overrides: &Overrides) -> Result { match command { + crate::cli::RecordCommand::Suites { write } => run_suites(write), crate::cli::RecordCommand::Tool { id } => run_tool(&id, overrides), crate::cli::RecordCommand::Forge { reference } => run_forge(&reference, overrides), crate::cli::RecordCommand::Plan => run_plan(), diff --git a/crates/batten/src/spec.rs b/crates/batten/src/spec.rs index 1571fca5e..932087808 100644 --- a/crates/batten/src/spec.rs +++ b/crates/batten/src/spec.rs @@ -604,6 +604,11 @@ mod tests { // its pullable path mints a receipt, so it is `write`, and a row // claiming otherwise would advertise a writing verb as read-only. "ready lint".to_owned(), + // CLOUD-1753's port of `tree-clean`, and `read` in §5's strong + // sense: it counts what `git status` reports and writes nothing, + // which is what lets `verify` ask it in `depends` without the ask + // itself becoming a reason the answer changes. + "receipt clean".to_owned(), "receipt status".to_owned(), // The composed receipt read that retired `mise-tasks/verified.sh` // (CLOUD-1148). Three `receipt::validity` reads and no write, so @@ -627,6 +632,11 @@ mod tests { // the corpse it reports, so a `read` row there would be false in // the one direction this allowlist exists to prevent. "task read".to_owned(), + // CLOUD-1718's fold, and `read` in §5's strong sense rather than + // its polite one: its whole input is two integers on argv, it + // opens nothing and spawns nothing, and its output is one of §7's + // four codes. There is no tree for it to be well-behaved about. + "verdict".to_owned(), "worktree status".to_owned(), ] ); @@ -885,6 +895,12 @@ mod tests { // for any consumer reading an entry as a prefix (CLOUD-121). "mcp".to_owned(), "mcp call".to_owned(), + // CLOUD-1753's port of the MCP launcher shim. A LEAF under the noun + // rather than a noun of its own, and that placement is the port's one + // improvement: the retired shell read the server out of its own file + // name, so a second server meant a second copy of the script. Here it + // is an argument. + "mcp spawn".to_owned(), // CLOUD-1267's noun and its two verbs, retired out of // `mise-tasks/mutant.sh` and `mise-tasks/mutant-census.sh`. // Stated here rather than regenerated, on the terms `checks` @@ -970,6 +986,11 @@ mod tests { "ready".to_owned(), "ready lint".to_owned(), "receipt".to_owned(), + // CLOUD-1753's port of `tree-clean`. The CHEAP end of a pair whose + // load-bearing half is a precondition inside `record` below — the + // verb that writes the receipt, and so the only place the question is + // answerable at the moment it matters. + "receipt clean".to_owned(), "receipt record".to_owned(), "receipt status".to_owned(), "receipt verified".to_owned(), @@ -1055,6 +1076,17 @@ mod tests { "task sig".to_owned(), "task tick".to_owned(), "task unregister".to_owned(), + // CLOUD-1718's fold, and a LEAF verb rather than a subtree because + // it decides one thing: given a count of findings and a count of + // blind spots, which of §7's four codes is the honest answer. It is + // on the read-only allowlist below — it opens nothing, reads no + // tree, and its whole input is two integers on argv. + // + // It exists because the retiring bash corpus INVERTS `1` and `2` + // against §7, so every port either re-spelled the mapping at its own + // call site or quietly shipped the shell's. One authority folding the + // two counts is what makes that unnecessary. + "verdict".to_owned(), // The one write path over a host's hook registrations // (CLOUD-893). Both rows are here and NEITHER is on the // read-only allowlist above: the noun is `Unclassified` because diff --git a/crates/batten/src/suites.rs b/crates/batten/src/suites.rs new file mode 100644 index 000000000..3504255bf --- /dev/null +++ b/crates/batten/src/suites.rs @@ -0,0 +1,266 @@ +//! The per-suite cost corpus: derive what each bats suite costs from the report +//! the runner already wrote (CLOUD-352), ported off `mise-tasks/suite-bench.sh` +//! under CLOUD-1753. +//! +//! # What this is for +//! +//! `test:bats` is thousands of cases over dozens of suites and the distribution +//! is extremely uneven — measured 2026-08-21, `land-lock.bats` at 175.8s against +//! a median under 5s. Nothing told an author which side of that a suite they were +//! editing sits on, and the timeout budgets that would eventually notice fire +//! minutes after a push, against an agent whose context is gone. +//! +//! # IT RUNS NOTHING, and that is the load-bearing property +//! +//! The corpus is derived from the report `test:bats` leaves behind. Re-executing +//! a multi-thousand-case suite to measure it would cost more than the waste it +//! reports, and would be a SECOND AUTHORITY over a run that already happened. +//! This module opens one file and writes one file; it spawns nothing, which is +//! also what keeps it out of `policy/spawn-adapters.rego` entirely. +//! +//! # A MISSING REPORT IS COULD-NOT-LOOK, never an empty corpus +//! +//! `test:bats` is receipt-gated, so a tree whose receipt is valid skips the run +//! entirely and an absent report is the ORDINARY state after a no-op lap. +//! Writing that out as "every suite costs nothing" is the CLOUD-251 collapse, and +//! it would be published rather than merely computed. +//! +//! # A REPORT OLDER THAN THE TREE IS ALSO COULD-NOT-LOOK +//! +//! This is faithful to the report, which is what makes it trustworthy and also +//! what made `--write` useless in one measured case: a suite retired, the report +//! still named it, so the regenerated corpus carried a cost attached to nothing +//! and the gate refused the very file its own remedy had just produced. A remedy +//! that cannot reach the state it prescribes is worse than no remedy — the author +//! reads the gate as broken rather than the report as stale. So a report naming a +//! suite this tree does not track is refused BY NAME, with the count and the +//! paths, before anything is written. +//! +//! # Durations are a CLOCK, not a gate +//! +//! Nothing here fails on a number moving. `rules/toolchain.md` puts a clock in a +//! drift job rather than in a gate, and what the companion rule decides is +//! MEMBERSHIP — which is deterministic — never a duration. + +use std::collections::BTreeSet; +use std::path::{Path, PathBuf}; + +use crate::Result; +use crate::error::UsageError; + +/// Where the runner leaves its report, relative to the repository root. +/// +/// Under `target/`, which is gitignored, so producing it cannot dirty the tree +/// and `receipt clean` stays green. +pub const REPORT: &str = "target/bats-report/report.xml"; + +/// The corpus this derives, relative to the repository root. +/// +/// **A committed path, and the companion rule reads it by name.** The gate that +/// decides membership is a `[[rule]]` over these bytes, so the location is a +/// contract between the two halves rather than an implementation detail. +pub const CORPUS: &str = "bench/suites/RESULTS.md"; + +/// One suite's own wall clock, as the runner measured it. +#[derive(Debug, Clone, PartialEq)] +pub struct Row { + /// The suite path, as the corpus spells it (`tests/`). + pub suite: String, + /// Seconds, at the precision the report carries. + pub seconds: f64, +} + +/// Read `` elements out of a JUnit report. +/// +/// **Text rather than a parser, for `ci-tools`' reason**: the shape is fixed by +/// the formatter the runner ships, and an XML parser is a dependency this +/// judgement does not otherwise need. The retired program said the same and read +/// the same two attributes. +/// +/// Order is by cost, descending, because the corpus is read by an author asking +/// "is the file I am editing expensive" and the answer is at the top. +#[must_use] +pub fn rows_in(report: &str) -> Vec { + let mut rows: Vec = report + .lines() + .filter_map(|line| { + let element = line.split_once("().ok()?; + Some(Row { + suite: format!("tests/{name}"), + seconds, + }) + }) + .collect(); + // `total_cmp` rather than `partial_cmp`: a `NaN` time would otherwise make + // the ordering inconsistent and the sort's output unspecified, which is a + // corpus whose rows move between runs for no reason a reader can see. + rows.sort_by(|left, right| right.seconds.total_cmp(&left.seconds)); + rows +} + +/// One attribute's value out of an element's text. +fn attribute<'a>(element: &'a str, name: &str) -> Option<&'a str> { + let needle = format!("{name}=\""); + let after = element.split_once(needle.as_str())?.1; + let (value, _) = after.split_once('"')?; + (!value.is_empty()).then_some(value) +} + +/// Render the corpus. +/// +/// The share column is computed at FULL PRECISION and rounded only where it is +/// printed. Rounding first and dividing second reported a single-suite corpus as +/// 105% of itself. +#[must_use] +pub fn render(rows: &[Row]) -> String { + let total: f64 = rows.iter().map(|row| row.seconds).sum(); + let mut out = String::new(); + out.push_str("# Per-suite cost of `test:bats`\n\n"); + out.push_str("Generated by `batten record suites` from the report `test:bats` writes.\n"); + out.push_str("Do not hand-edit. Durations are wall clock for the suite alone, as the\n"); + out.push_str("runner measured it; the suite runs `--no-parallelize-within-files`, so a\n"); + out.push_str("file's number is its own serial cost and is what an author adding a case\n"); + out.push_str("to it pays.\n\n"); + out.push_str(&format!("- suites: {}\n", rows.len())); + out.push_str(&format!("- serial total: {total:.1}s\n\n")); + out.push_str("| seconds | share | suite |\n| ---: | ---: | --- |\n"); + for row in rows { + let share = if total > 0.0 { + (row.seconds / total) * 100.0 + } else { + 0.0 + }; + out.push_str(&format!( + "| {:.1} | {share:.1}% | `{}` |\n", + row.seconds, row.suite + )); + } + out +} + +/// Which of `rows` name a suite `tracked` does not carry. +/// +/// Pointer-only (rule 4): the caller reports the count and the paths, which is +/// what a reader acts on, and never a duration — a number here would be a second +/// authority over the corpus. +#[must_use] +pub fn stale(rows: &[Row], tracked: &BTreeSet) -> Vec { + rows.iter() + .map(|row| row.suite.clone()) + .filter(|suite| !tracked.contains(suite)) + .collect() +} + +/// Derive the corpus, or say why it could not be. +/// +/// # Errors +/// +/// An internal error (exit `3`) when the report is absent, carries no readable +/// `` element, or names a suite this tree does not track. All three +/// are could-not-look, and none of them is an empty corpus. +pub fn derive(root: &Path, tracked: &BTreeSet) -> Result<(Vec, String)> { + let path = root.join(REPORT); + let Ok(report) = std::fs::read_to_string(&path) else { + return Err(UsageError::raise(format!( + "record suites: no report at {REPORT} — `test:bats` has not run in this tree, or its \ + receipt let it skip. Run `mise run test:bats` first; this reads a report and measures \ + nothing itself." + ))); + }; + let rows = rows_in(&report); + if rows.is_empty() { + return Err(UsageError::raise(format!( + "record suites: {REPORT} carries no element with a name and a time, so \ + there is nothing to derive. This is could-not-look, not an empty corpus." + ))); + } + let orphans = stale(&rows, tracked); + if !orphans.is_empty() { + return Err(UsageError::raise(format!( + "record suites: {REPORT} names {} suite(s) this tree does not track — the report \ + predates the tree, so deriving from it would publish a cost for a suite that is gone. \ + Run `mise run test:bats` to produce a report over the suites that exist, then re-run.", + orphans.len() + ))); + } + let text = render(&rows); + Ok((rows, text)) +} + +/// Where the corpus is written. +#[must_use] +pub fn corpus_path(root: &Path) -> PathBuf { + root.join(CORPUS) +} + +#[cfg(test)] +mod tests { + use super::*; + + #[allow(clippy::expect_used)] + fn tracked(names: &[&str]) -> BTreeSet { + names.iter().map(|name| (*name).to_owned()).collect() + } + + #[test] + fn a_report_yields_one_row_per_suite_ordered_by_cost() { + let report = concat!( + "\n", + "\n", + "\n", + "\n", + ); + let rows = rows_in(report); + assert_eq!(rows.len(), 2); + assert_eq!(rows[0].suite, "tests/slow.bats"); + assert_eq!(rows[1].suite, "tests/quick.bats"); + } + + #[test] + fn an_element_missing_either_attribute_yields_no_row() { + // Not a zero. A suite whose time the report does not carry is a suite + // this cannot answer about, and a `0.0` there would publish it as the + // cheapest file in the tree. + let report = "\n\n"; + assert!(rows_in(report).is_empty()); + } + + #[test] + fn the_share_column_is_computed_before_rounding() { + // Rounding first and dividing second reported a single-suite corpus as + // 105% of itself. + let rows = vec![Row { + suite: "tests/only.bats".to_owned(), + seconds: 0.04, + }]; + let text = render(&rows); + assert!(text.contains("100.0%"), "{text}"); + } + + #[test] + fn a_row_naming_an_untracked_suite_is_stale() { + let rows = vec![ + Row { + suite: "tests/live.bats".to_owned(), + seconds: 1.0, + }, + Row { + suite: "tests/retired.bats".to_owned(), + seconds: 2.0, + }, + ]; + let orphans = stale(&rows, &tracked(&["tests/live.bats"])); + assert_eq!(orphans, vec!["tests/retired.bats".to_owned()]); + } + + #[test] + fn a_corpus_over_no_rows_has_no_share_to_divide_by() { + // Zero total, and the guard is what stops a division by zero becoming a + // `NaN%` column in a published file. + let text = render(&[]); + assert!(text.contains("- suites: 0"), "{text}"); + } +} diff --git a/crates/batten/src/surface.rs b/crates/batten/src/surface.rs index 5c3680528..c9de6218e 100644 --- a/crates/batten/src/surface.rs +++ b/crates/batten/src/surface.rs @@ -855,6 +855,94 @@ const JOBS: FlagDecl = FlagDecl { value: ValueDecl::Str, }; +/// `--lock` on `exec` (CLOUD-1710), the key one clone's singleton lock is named +/// by. +/// +/// A KEY, never a path. `mise-tasks/with-lock.sh` took a lock directory and its +/// callers pointed one at the rust sysroot, so a toolchain swap could not +/// deadlock on a stale path. The key carries that distinction where it belongs — +/// in the name (`target-ensure-`) — and lets the lock live under +/// `$GIT_DIR` with every other one this clone holds, which is what +/// `batten singleton` and `mise run alive` already read. +const LOCK: FlagDecl = FlagDecl { + id: "lock", + long: Some("lock"), + short: None, + help: "Hold this clone's named singleton lock for the child's lifetime", + env: EnvDecl::None, + global: false, + positional: false, + required: false, + hidden: false, + rung: Rung::None, + value: ValueDecl::Str, +}; + +/// `--lock-path` on `exec` (CLOUD-1710): a lock guarding something the clone +/// does not own. +/// +/// The sibling of `--lock`, and the two are not interchangeable. A KEY names a +/// lock under `$GIT_DIR`, which is right for "one task per clone". A PATH names +/// one wherever the resource being serialized actually lives — the retiring +/// shell's two callers both needed that, `doctor`'s lock sitting under +/// `$MISE_DATA_DIR` and `target-ensure`'s inside the rust sysroot, because a +/// mise install tree and a rustup toolchain are the MACHINE's and are shared by +/// every clone on it. Keying those per clone would let two checkouts install a +/// target concurrently and roll each other back (CLOUD-220). +const LOCK_PATH: FlagDecl = FlagDecl { + id: "lock_path", + long: Some("lock-path"), + short: None, + help: "Hold the lock at this path, for a resource the clone does not own", + env: EnvDecl::None, + global: false, + positional: false, + required: false, + hidden: false, + rung: Rung::None, + value: ValueDecl::Str, +}; + +/// `--lock-attempts` on `exec` (CLOUD-1710): how long the queue is, as a COUNT. +/// +/// The bound is a number of asks separated by a declared interval, never a wall +/// clock — `land`'s shape, for `land`'s reason. The shell spelled it +/// `WITH_LOCK_TIMEOUT` in seconds and divided by its own `sleep 0.1`; the +/// default here is that same arithmetic already done (600s ⇒ 6000 asks). +const LOCK_ATTEMPTS: FlagDecl = FlagDecl { + id: "lock_attempts", + long: Some("lock-attempts"), + short: None, + help: "How many times to ask for the lock before reporting it held", + env: EnvDecl::None, + global: false, + positional: false, + required: false, + hidden: false, + rung: Rung::None, + value: ValueDecl::Str, +}; + +/// `--lock-label` on `exec` (CLOUD-1710): what the wait was FOR. +/// +/// Carried verbatim from the shell, which states why: *"A lock path is a pointer +/// to a file; 'the toolchain lock (aarch64-apple-darwin)' is a pointer to the +/// thing a reader has to reason about, and moving the wait out of the caller +/// must not cost that."* +const LOCK_LABEL: FlagDecl = FlagDecl { + id: "lock_label", + long: Some("lock-label"), + short: None, + help: "What the wait is for, named by the caller for the refusal line", + env: EnvDecl::None, + global: false, + positional: false, + required: false, + hidden: false, + rung: Rung::None, + value: ValueDecl::Str, +}; + /// `--continue-on-error` on `exec` (CLOUD-430), likewise mise's. const CONTINUE_ON_ERROR: FlagDecl = FlagDecl { id: "continue_on_error", @@ -910,6 +998,45 @@ const NO_CACHE: FlagDecl = FlagDecl { /// That one SELECTS a row to run and is optional; this one NAMES a row to print /// and is required, so sharing a `FlagDecl` would make one row's `required` a /// lie about the other. +/// `--findings `: how many blocking findings the caller's own run produced. +/// +/// A COUNT rather than a boolean, because the caller already has one and a +/// boolean would make it fold twice — once to a flag and once here — which is the +/// hand-fold this verb exists to remove. +const VERDICT_FINDINGS: FlagDecl = FlagDecl { + id: "findings", + long: Some("findings"), + short: None, + help: "How many blocking findings the run produced", + env: EnvDecl::None, + global: false, + positional: false, + required: false, + hidden: false, + rung: Rung::None, + value: ValueDecl::Str, +}; + +/// `--unjudgeable `: how many subjects the caller could not read. +/// +/// Separate from `--findings` rather than folded into it, because the whole +/// point of the verb is that the two are DIFFERENT answers: a blind spot is not +/// a small finding, and a caller that adds them together has already lost the +/// distinction this decides. +const VERDICT_UNJUDGEABLE: FlagDecl = FlagDecl { + id: "unjudgeable", + long: Some("unjudgeable"), + short: None, + help: "How many subjects the run could not read", + env: EnvDecl::None, + global: false, + positional: false, + required: false, + hidden: false, + rung: Rung::None, + value: ValueDecl::Str, +}; + const RULE_ID_ARG: FlagDecl = FlagDecl { id: "id", long: None, @@ -1867,6 +1994,27 @@ const ISSUE: FlagDecl = FlagDecl { value: ValueDecl::Str, }; +/// Write the derived corpus to its committed path instead of to stdout. +/// +/// **Off by default, which is the safer direction for a verb whose product is a +/// tracked file.** A caller who wants to SEE what would be recorded — a reviewer +/// diffing a proposed corpus against the committed one — must not have to dirty +/// the tree to do it, and `receipt clean` is one of the things that would then +/// refuse. +const SUITES_WRITE: FlagDecl = FlagDecl { + id: "write", + long: Some("write"), + short: None, + help: "Write the corpus to its committed path instead of printing it", + env: EnvDecl::None, + global: false, + positional: false, + required: false, + hidden: false, + rung: Rung::None, + value: ValueDecl::Bool, +}; + /// The roster `checks green` decides against (CLOUD-1143). /// /// **Flags rather than environment variables, and that is rule 1 rather than @@ -2477,6 +2625,10 @@ pub const SURFACE: &[CommandDecl] = &[ TEE, JOBS, CONTINUE_ON_ERROR, + LOCK, + LOCK_PATH, + LOCK_ATTEMPTS, + LOCK_LABEL, FlagDecl::defaulted_enum( "format", "format", @@ -2661,6 +2813,45 @@ pub const SURFACE: &[CommandDecl] = &[ ), ], }, + // The spawn ledger and the launch, ported off `mise-tasks/serena-mcp.sh` + // under CLOUD-1753 (CLOUD-714 is the defect it exists for). + // + // WHAT THIS MUST NEVER BECOME is the whole of CLOUD-714 and it survives the + // port structurally rather than by prose: not a retry, not a supervisor, not + // a keepalive. It records and it EXECS, and after an exec there is no process + // left that could restart anything. A shim that recovered from a failed + // launch would hide the defect it was written to expose. + // + // `Unclassified` on `exec`'s own reading, and it is the honest one twice + // over: the verb replaces this process with a command the CONSUMER named, so + // its reach is whatever that command's is, and it writes a per-clone ledger + // besides. An optimistic `read` here would put a process-replacing verb on + // the derived allowlist. + // + // `data_channel` is FALSE and here it is load-bearing beyond `exec`'s reason: + // STDOUT IS THE MCP TRANSPORT. One stray byte corrupts the JSON-RPC stream + // and takes the server down looking exactly like the bug this records. So + // Batten emits nothing on stdout at all, and the record it keeps is a file. + CommandDecl { + path: "mcp spawn", + id: "mcp.spawn", + about: "Record that a client actually spawned this server, then exec the launch line unchanged", + data_channel: false, + exits: EXITS_STANDARD, + effect: Effect::Unclassified, + flags: &[ + // Declared BEFORE the trailing argv, on `exec`'s note: a + // `trailing_var_arg` swallows everything after the first free token. + FlagDecl::positional( + "server", + "The server this launch is for, as the spawn ledger and `mcp-attach-check` name it", + ), + FlagDecl::trailing( + "command", + "The launch line, run verbatim — Batten execs it and does not supervise it", + ), + ], + }, // The `target` noun only dispatches, and takes `capture`'s reading one row // family up rather than `policy`'s: its subtree carries a `destructive` verb, // §5 derives the agent allowlist from `effect == read`, and a `read` noun over @@ -3375,6 +3566,32 @@ pub const SURFACE: &[CommandDecl] = &[ // its entire purpose, and the text is the config author's own declaration — // the class `config show` exists to echo — not content read out of a subject // file. + // CLOUD-1718's shape (a). There are two exit-code contracts in this tree and + // they are INVERSES: this binary reads `1` as usage and `2` as a violation, + // and 82 shell programs read `1` as a violation and `2` as could-not-look. + // A caller on the wrong side of that boundary does not get a worse message, + // it gets the opposite verdict — a blind spot read as a finding, or a finding + // read as a blind spot. + // + // The bug dies with each program that retires, so this verb is justified by + // what does NOT retire: the workflow tree, which is permanently bash by + // declaration; the installer, which is bash by construction; and CONSUMER + // repositories, whose gates hit the identical inversion with no campaign to + // save them. Each of those still needs the fold, and this is the one place it + // is decided. + // + // `read` structurally and in the strongest sense available: it opens no file, + // walks no tree and spawns nothing. It reads two integers off its own command + // line and returns a code. + CommandDecl { + path: "verdict", + id: "verdict", + about: "Fold a run's findings and blind spots into this tool's exit code", + data_channel: false, + exits: EXITS_STANDARD, + effect: Effect::Read, + flags: &[VERDICT_FINDINGS, VERDICT_UNJUDGEABLE], + }, CommandDecl { path: "policy explain", id: "policy.explain", @@ -4333,6 +4550,31 @@ pub const SURFACE: &[CommandDecl] = &[ effect: Effect::Unclassified, flags: &[], }, + // The precondition, asked on its own — ported off `mise-tasks/tree-clean.sh` + // under CLOUD-1753. + // + // ONE PREDICATE, TWO CALLERS, AND THAT IS NOT A SECOND AUTHORITY. `receipt + // record` refuses a dirty tree itself, which is the load-bearing end: it is + // the verb that writes the receipt, so it is the only place the question is + // both askable and answerable at the moment it matters. This row is the CHEAP + // end, and the retiring program's own caller shows why both are needed — + // `verify:gated` asks it in `depends` so a dirty tree fails in seconds rather + // than after the ~170s the gate set costs, and `depends` completes before the + // body starts, so a tree dirtied MID-RUN is invisible to it. Both call + // `receipt::tree_state`; neither re-derives it. + // + // `read` in §5's strong sense: it counts what `git status` reports and writes + // nothing, which is what lets `verify` ask it without the ask itself becoming + // a reason the answer changes. + CommandDecl { + path: "receipt clean", + id: "receipt.clean", + about: "Refuse when the working tree differs from HEAD, so a receipt keyed to HEAD would attest bytes no commit contains", + data_channel: false, + exits: EXITS_VERDICT, + effect: Effect::Read, + flags: &[], + }, // Creates state the caller can recreate by re-running the check. CommandDecl { path: "receipt record", @@ -4672,6 +4914,33 @@ pub const SURFACE: &[CommandDecl] = &[ // could supply the wrong one", and this composes the same key with the same two // functions. The negative half falls out for free: a record for a tool nobody // declared is unspellable. + // The per-suite cost corpus (CLOUD-352), ported off `mise-tasks/suite-bench.sh` + // under CLOUD-1753. + // + // A LEAF UNDER `record` RATHER THAN A NOUN OF ITS OWN, and the noun already + // means what this does: write down what happened. The two neighbours key + // their records by tool and by forge; this one keys by suite, and the store + // is a committed file because the READER is a person deciding whether the + // file they are about to edit is expensive. + // + // IT RUNS NOTHING. The corpus is derived from the report `test:bats` leaves + // behind — re-executing the suite to measure it would cost more than the + // waste it reports and would be a second authority over a run that already + // happened. So this reads one file and writes one file, and reaches + // `policy/spawn-adapters.rego` not at all. + // + // `--write` rather than always writing: without it the corpus goes to stdout, + // which is what lets a caller diff a proposed corpus against the committed + // one without touching the tree. + CommandDecl { + path: "record suites", + id: "record.suites", + about: "Derive what each bats suite costs from the report the runner wrote, and record it where an author reads it", + data_channel: false, + exits: EXITS_STANDARD, + effect: Effect::Write, + flags: &[SUITES_WRITE], + }, CommandDecl { path: "record tool", id: "record.tool", diff --git a/crates/batten/src/task.rs b/crates/batten/src/task.rs index c1ba4dd83..776c6a1dd 100644 --- a/crates/batten/src/task.rs +++ b/crates/batten/src/task.rs @@ -682,7 +682,7 @@ pub fn report( // immediately and holds nothing, exactly as the shell did. /// Where one clone's singleton locks live. -const SINGLETON_DIR: &str = "batten-singleton"; +pub(crate) const SINGLETON_DIR: &str = "batten-singleton"; /// What an acquire attempt found. #[derive(Clone, Debug, PartialEq, Eq)] @@ -771,56 +771,143 @@ pub fn singleton_acquire( recheck: std::time::Duration, ) -> Claim { let lock = singleton_lock(git_dir, task); - let dir = git_dir.join(SINGLETON_DIR); - if std::fs::create_dir_all(&dir).is_err() { - return Claim::CouldNotLook(dir); + match singleton_acquire_at(&lock, pid, recheck, 1) { + // The registry's word about what the holder is DOING, and it is added + // here rather than in the path-taking core because only a clone-scoped + // lock has a registry to ask. A lock guarding a machine-global resource + // has no such entry, and inventing one would be a second authority over + // a layout this module owns. + Claim::Held { + holder, + phase: None, + } => { + let phase = read_field(git_dir, &holder, "phase").filter(|phase| !phase.is_empty()); + Claim::Held { holder, phase } + } + answered => answered, } - if take(&lock, pid) { - return Claim::Taken; +} + +/// [`singleton_acquire`] over a lock at an explicit PATH. +/// +/// **The core, and the split is behaviour rather than tidiness** (CLOUD-1710). +/// `singleton_acquire`'s lock lives under `$GIT_DIR`, which is right for "one +/// task per clone" and wrong for a lock guarding something the clone does not +/// own. The two locks `mise-tasks/with-lock.sh` was written for are both of the +/// second kind — `doctor`'s sits under `$MISE_DATA_DIR` and `target-ensure`'s +/// inside the rust sysroot it protects — because the resources they serialize +/// are the MACHINE's, shared by every clone on it. Keying those per clone would +/// let two checkouts install a rustup target concurrently and roll each other +/// back, which is CLOUD-220 returning by another route. +/// +/// Every decision is the same one `singleton_acquire` makes; only where the lock +/// lives differs. +#[must_use] +pub fn singleton_acquire_at( + lock: &Path, + pid: &str, + recheck: std::time::Duration, + attempts: usize, +) -> Claim { + // A lock with no parent is not a lock: could-not-look, never "free". + let Some(parent) = lock.parent() else { + return Claim::CouldNotLook(lock.to_path_buf()); + }; + if std::fs::create_dir_all(parent).is_err() { + return Claim::CouldNotLook(parent.to_path_buf()); } - // An EMPTY pid file is a holder caught between its create and its write, not - // a corpse: absence of evidence is "held", never "free". - let Some(holder) = holder_of(&lock) else { - return Claim::Held { - holder: "unknown".to_owned(), - phase: None, + // At least one ask, so `attempts = 0` is a single non-blocking attempt rather + // than a silent "never look" that would report a free lock as held. + let rounds = attempts.max(1); + let mut round = 0_usize; + loop { + round += 1; + let last = round >= rounds; + if take(lock, pid) { + return Claim::Taken; + } + + // An EMPTY pid file is a holder caught between its create and its write, + // not a corpse: absence of evidence is "held", never "free". + let Some(holder) = holder_of(lock) else { + if last { + return Claim::Held { + holder: "unknown".to_owned(), + phase: None, + }; + } + sleep_once(recheck); + continue; }; - }; - // There is deliberately NO early live-holder fast path. It read as a safety - // property and was not one: with it deleted a live holder still falls - // through to the refusal below, so no test could tell the two apart and it - // survived its own mutant. One refusal path is worth more than the pause. - // - // First sighting of a dead pid. Look again before reclaiming, so a holder - // that exited cleanly between the read and the check — its own trap already - // removing the directory — is never mistaken for one that died holding, and - // a NEW holder that took the lock in between is never robbed of it. - // An INVENTORY ROW, and it is neither of the two shapes CLOUD-1177 separates. - // It is not a poll — nothing is re-attempted on a schedule — and it is not a - // timer standing in for an exit condition, because there is no condition to - // wait for: the two sightings must be separated by elapsed time or they are - // one sighting. The bound is `--recheck-ms`, a single pause the caller - // declares, and the interval IS the safety margin rather than a guess at how - // long something takes. + // There is deliberately NO early live-holder fast path. It read as a safety + // property and was not one: with it deleted a live holder still falls + // through to the refusal below, so no test could tell the two apart and it + // survived its own mutant. One refusal path is worth more than the pause. + // + // First sighting of a dead pid. Look again before reclaiming, so a holder + // that exited cleanly between the read and the check — its own trap already + // removing the directory — is never mistaken for one that died holding, and + // a NEW holder that took the lock in between is never robbed of it. + // An INVENTORY ROW, and it is neither of the two shapes CLOUD-1177 separates. + // It is not a poll — nothing is re-attempted on a schedule — and it is not a + // timer standing in for an exit condition, because there is no condition to + // wait for: the two sightings must be separated by elapsed time or they are + // one sighting. The bound is `--recheck-ms`, a single pause the caller + // declares, and the interval IS the safety margin rather than a guess at how + // long something takes. + sleep_once(recheck); + if may_reclaim(&holder, holder_of(lock).as_deref()) + && std::fs::remove_dir_all(lock).is_ok() + && take(lock, pid) + { + return Claim::Reclaimed(holder); + } + + // The lock changed under us, or a live holder took it: whoever holds it + // now is real. Re-read rather than reporting the corpse seen a moment + // ago. + if last { + return Claim::Held { + holder: holder_of(lock).unwrap_or_else(|| "unknown".to_owned()), + phase: None, + }; + } + } +} + +/// The one declared pause in this module, and the only waived delay it owns. +/// +/// **One site rather than two, and that is a rule rather than tidiness.** +/// `delay-waivers-not-growing` ratchets how many waiver annotations stand over a +/// sleep in this crate, because *a ban you can waive at will is not a ban* +/// (CLOUD-1148). A queue written as its own loop with its own sleep would have +/// spent a twelfth waiver to express a pause this function already owns. +/// +/// The annotation below is deliberately not spelled anywhere in this prose: that +/// row's `pattern` is an unanchored literal, so a doc comment naming it counts +/// as a waiver. Measured — the first draft of this comment took the count to 12 +/// on its own, which is `no-new-ignores`' documented hazard (*"a bare `#[ignore]` +/// is a literal substring, so it also matches every prose mention"*) arriving in +/// the row that did not anchor for it. +/// +/// It serves both readings, and they are the same pause seen from two sides: the +/// interval separating the two sightings a reclaim requires, and the interval +/// between two asks for a lock somebody else holds. Neither is a poll standing +/// in for an exit condition (CLOUD-1177) — the bound is `attempts`, a count the +/// caller declares, exactly as `land` bounds its laps. +fn sleep_once(interval: std::time::Duration) { #[expect( clippy::disallowed_methods, - reason = "the bound is `--recheck-ms`: a single declared pause separating the two sightings a reclaim requires, not a poll and not a timer standing in for an exit condition" + reason = "the module's one declared pause: it separates the two sightings a reclaim requires AND the two asks a queue makes, bounded by `attempts` rather than by a clock, so it is neither a poll nor a timer standing in for an exit condition" )] - std::thread::sleep(recheck); - if may_reclaim(&holder, holder_of(&lock).as_deref()) - && std::fs::remove_dir_all(&lock).is_ok() - && take(&lock, pid) - { - return Claim::Reclaimed(holder); - } + std::thread::sleep(interval); +} - // The lock changed under us, or a live holder took it: whoever holds it now - // is real. Re-read rather than reporting the corpse seen a moment ago. - let holder = holder_of(&lock).unwrap_or_else(|| "unknown".to_owned()); - let phase = read_field(git_dir, &holder, "phase").filter(|phase| !phase.is_empty()); - Claim::Held { holder, phase } +/// Drop a lock at an explicit path, whether or not it was ever taken. +pub fn singleton_release_at(lock: &Path) { + let _ = std::fs::remove_dir_all(lock); } /// May the lock be taken from the pid seen at the FIRST sighting? diff --git a/crates/batten/tests/it/awk_regex.rs b/crates/batten/tests/it/awk_regex.rs new file mode 100644 index 000000000..08ec9ea2f --- /dev/null +++ b/crates/batten/tests/it/awk_regex.rs @@ -0,0 +1,166 @@ +//! No shell program hands awk a regex through `-v`, over the compiled binary +//! (ported from `mise-tasks/awk-regex-check.sh` under CLOUD-843). +//! +//! **What is decidable only here.** `policy/awk-regex.rego` carries load-time +//! cases pinning the predicate, and each supplies `input.tree.lines` with `with +//! input as` — fabricating the corpus the gate is about. Only a real repository +//! shows the engine handing this module the shell it is meant to scan, which is +//! the anti-vacuity half: a scan that matched no file reports "no regex reaches +//! awk through -v" over nothing and reads as coverage (CLOUD-418). +// +// carried: mise-tasks/awk-regex-check.sh policy/awk-regex.rego crates/batten/tests/it/awk_regex.rs +// carried: tests/awk-regex-check.bats policy/awk-regex.rego crates/batten/tests/it/awk_regex.rs +// +// carried: "this repo's own tasks pass today" policy/awk-regex.rego +// carried: "the real bug: a -v name used with ~ is reported" policy/awk-regex.rego +// carried: "the report names file and line, not the whole command" policy/awk-regex.rego +// carried: "match() is regex position too" policy/awk-regex.rego +// carried: "a -v value compared with == is fine — that is most of its use" policy/awk-regex.rego +// carried: "a -v value printed or counted is fine" policy/awk-regex.rego +// carried: "an inline regex in the awk program is the recommended form, not a finding" policy/awk-regex.rego +// carried: "a name that merely PREFIXES another is not confused for it" policy/awk-regex.rego +// carried: "several -v assignments on one line are each judged" policy/awk-regex.rego +// carried: "a tree with no awk at all passes rather than erroring" policy/awk-regex.rego +// +// changed: "an untracked file is not judged — the gate reads what is committed" policy/awk-regex.rego the shell fed itself `git ls-files`, so the index was the subject and the case pinned it. `input.tree.lines` is resolved from a declared glob over the working-tree walk, which honours `.gitignore` and explicitly is not the index (`facts.rs`), and nothing available to a module expresses index membership. The successor judges an uncommitted program too — stricter, fail-closed, and the same difference `policy/module-map.rego` records for its own port + +// Panicking on setup failure is the idiomatic way for a test to fail loudly. +#![allow(clippy::unwrap_used, clippy::expect_used)] + +use crate::common; + +use std::path::{Path, PathBuf}; +use std::process::Output; + +use common::{Fixture, git_in, run, stdout}; + +fn awk_repo(name: &str, program: &str) -> PathBuf { + let dir = Fixture::new(name) + .config( + "version = 1\n\n\ + [[pattern]]\n\ + id = \"awk-v-assignment\"\n\ + regex = '-v[[:space:]]*([A-Za-z_][A-Za-z0-9_]*)='\n\n\ + [[pattern]]\n\ + id = \"leading-identifier\"\n\ + regex = '^[A-Za-z_][A-Za-z0-9_]*'\n\n\ + [[verdict]]\n\ + id = \"pattern carry unsafe\"\n\ + gloss = \"a command is spelled in a way whose behaviour is not defined across implementations\"\n\ + class = \"Escape processing on a -v assignment differs between awk implementations, so the same pattern is two different regexes.\"\n\n\ + [[verdict.route]]\n\ + id = \"prose read first\"\n\ + kind = \"document\"\n\ + target = \"AGENTS.md\"\n\n\ + [[rule]]\n\ + id = \"awk-regex\"\n\ + kind = \"policy\"\n\ + scope = \"tree\"\n\ + line_sources = [\"mise-tasks/**\"]\n\ + module = \"policy/awk-regex.rego\"\n\ + severity = \"deny\"\n", + ) + .file("AGENTS.md", "the consumer's own authority\n") + .file("mise-tasks/demo.sh", program) + .git() + .build(); + common::write( + &dir, + "policy/awk-regex.rego", + &std::fs::read_to_string(common::at_root("policy/awk-regex.rego")).unwrap(), + ); + git_in(&dir, &["add", "-A"]); + git_in(&dir, &["commit", "-q", "-m", "base"]); + dir +} + +fn check(dir: &Path) -> Output { + run(dir, &["check", "--rule", "awk-regex"]) +} + +#[test] +fn the_real_bug_a_v_name_used_with_tilde_is_reported() { + let dir = awk_repo( + "awk-regex-tilde", + "#!/usr/bin/env bash\nawk -v re=\"$p\" '$0 ~ re' file\n", + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); + let text = stdout(&output); + assert!( + text.contains("mise-tasks/demo.sh"), + "the report names the file: {text:?}" + ); + assert!( + !text.contains("$0 ~ re"), + "and not the whole command, which is payload: {text:?}" + ); +} + +#[test] +fn match_is_regex_position_too() { + let dir = awk_repo( + "awk-regex-match", + "#!/usr/bin/env bash\nawk -v re=\"$p\" '{ if (match($0, re)) print }' file\n", + ); + assert_eq!(check(&dir).status.code(), Some(2)); +} + +#[test] +fn a_value_compared_with_equals_is_fine_over_the_binary() { + // Most of `-v`'s use, and the half the gate must not refuse. + let dir = awk_repo( + "awk-regex-equals", + "#!/usr/bin/env bash\nawk -v want=\"$p\" '$1 == want' file\n", + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(0), "{}", stdout(&output)); +} + +#[test] +fn a_name_that_prefixes_another_is_not_confused_for_it() { + // DISCRIMINATING. The shell guarded this boundary with a trailing character + // class on a per-name regex; a module may not build one at all, so the port + // reads the identifier after `~` and compares it WHOLE. A successor that + // used `startswith` instead passes every other case here and fails this one. + let dir = awk_repo( + "awk-regex-prefix", + "#!/usr/bin/env bash\nawk -v re=\"$p\" '$0 ~ rex' file\n", + ); + let output = check(&dir); + assert_eq!( + output.status.code(), + Some(0), + "`re` was confused for `rex`: {}", + stdout(&output) + ); +} + +#[test] +fn an_inline_regex_is_the_recommended_form_not_a_finding() { + let dir = awk_repo( + "awk-regex-inline", + "#!/usr/bin/env bash\nawk '$0 ~ /^ISSUE-[0-9]+$/' file\n", + ); + assert_eq!(check(&dir).status.code(), Some(0)); +} + +#[test] +fn a_tree_with_no_awk_at_all_passes_rather_than_erroring() { + let dir = awk_repo( + "awk-regex-no-awk", + "#!/usr/bin/env bash\ngrep -v thing file\n", + ); + assert_eq!(check(&dir).status.code(), Some(0)); +} + +#[test] +fn this_repos_own_programs_pass_today() { + let output = common::run_at_real_root(&common::at_root(""), &["check", "--rule", "awk-regex"]); + assert_eq!( + output.status.code(), + Some(0), + "a regex reaches awk through -v in this tree: {}", + stdout(&output) + ); +} diff --git a/crates/batten/tests/it/cap_drift.rs b/crates/batten/tests/it/cap_drift.rs new file mode 100644 index 000000000..648cc8d64 --- /dev/null +++ b/crates/batten/tests/it/cap_drift.rs @@ -0,0 +1,288 @@ +//! A cap and its bot-side withholding rule exist together or not at all, over +//! the compiled binary (CLOUD-593, ported from `mise-tasks/cap-drift.sh` under +//! CLOUD-843). +//! +//! **What is decidable only here.** `policy/cap-drift.rego` carries load-time +//! cases pinning the predicate, and every one hands the module a ready-made map +//! of both files with `with input as`. That fabricates the very shape the engine +//! may be unable to produce (CLOUD-845), and here it fabricates the pairing +//! itself: this rule is set equality ACROSS two `line_sources` paths resolved in +//! one evaluation, and a module whose suite only fabricated the map would stay +//! green over an engine that resolved one of them. Resolving only the manifest +//! empties the withheld set and reports every cap as unmirrored; resolving only +//! the bot config empties the capped set and reports the half-lift everywhere. +//! Both are loud. What is silent, and what this tier is for, is the arm each +//! guard makes conditional: with a file absent the rule is not evaluated at all, +//! and the pairing goes unchecked with nothing red. +//! +//! The self-consumption case is the one the retiring suite ended on: the real +//! tree's two sets agree, checked rather than asserted. +// +// carried: mise-tasks/cap-drift.sh policy/cap-drift.rego crates/batten/tests/it/cap_drift.rs +// carried: tests/cap-drift.bats policy/cap-drift.rego crates/batten/tests/it/cap_drift.rs +// +// carried: "both sets empty passes — the state CLOUD-593 leaves, and the ratchet still runs" policy/cap-drift.rego +// carried: "a cap mirrored by an allowedVersions rule passes" policy/cap-drift.rego +// carried: "THE HALF-LIFT: an allowedVersions rule with no cap is refused, and named" policy/cap-drift.rego +// carried: "a cap with no allowedVersions rule is refused, and named" policy/cap-drift.rego +// carried: "both directions are reported in one pass, not one per run" policy/cap-drift.rego +// carried: "a caret requirement is not a cap — it bounds the major, not the compiler" policy/cap-drift.rego +// carried: "a bare-string upper bound is a cap too, not only the inline-table form" policy/cap-drift.rego +// carried: "a less-than outside the workspace dependencies table mints no phantom cap" policy/cap-drift.rego +// carried: "a matchPackageNames used for grouping is not read as a cap mirror" policy/cap-drift.rego +// carried: "an allowedVersions named only in a comment mirrors nothing" policy/cap-drift.rego +// carried: "a rule written inline reads the same as one spread over lines" policy/cap-drift.rego +// carried: "cap-drift.bats::the real tree agrees" policy/cap-drift.rego +// changed: "an unreadable file is exit 2 — never a silent agreement" policy/cap-drift.rego the shell opened two named files and refused one it could not read; the successor declares them as `line_sources` and the ENGINE decides this earlier — a rule whose declared paths match nothing is not evaluated at all, and `input.tree.missing` is never populated on the tree surface (CLOUD-1049, measured identically for `policy/mise-pin-agreement.rego`'s own could-not-look clause). What the case protected is kept in the shape the engine does allow: each direction guards on the OTHER file having resolved, so a present-but-empty file still reports the pairing rather than reading as agreement + +// Panicking on setup failure is the idiomatic way for a test to fail loudly. +#![allow(clippy::unwrap_used, clippy::expect_used)] + +use crate::common; + +use std::fmt::Write as _; +use std::path::{Path, PathBuf}; +use std::process::Output; + +use common::{Fixture, git_in, run, stderr, stdout}; + +/// A repository declaring only this rule, so any finding is the one under test. +fn cap_repo(name: &str, manifest: &str, bot: &str) -> PathBuf { + let dir = Fixture::new(name) + .config( + "version = 1\n\n\ + [[pattern]]\n\ + id = \"md-quoted-span\"\n\ + regex = '\"[^\"]*\"'\n\n\ + [[verdict]]\n\ + id = \"bound carry missing\"\n\ + gloss = \"an upper bound exists in one of its two files and not the other\"\n\ + class = \"A rule with no cap withholds a version the manifest admits, and nothing goes red.\"\n\n\ + [[verdict.route]]\n\ + id = \"prose read first\"\n\ + kind = \"document\"\n\ + target = \"AGENTS.md\"\n\n\ + [[rule]]\n\ + id = \"cap-drift\"\n\ + kind = \"policy\"\n\ + scope = \"tree\"\n\ + line_sources = [\"Cargo.toml\", \"renovate.json5\"]\n\ + module = \"policy/cap-drift.rego\"\n\ + severity = \"deny\"\n", + ) + .file("AGENTS.md", "the consumer's own authority\n") + .file("Cargo.toml", manifest) + .file("renovate.json5", bot) + .git() + .build(); + common::write( + &dir, + "policy/cap-drift.rego", + &std::fs::read_to_string(common::at_root("policy/cap-drift.rego")).unwrap(), + ); + git_in(&dir, &["add", "-A"]); + git_in(&dir, &["commit", "-q", "-m", "base"]); + dir +} + +fn check(dir: &Path) -> Output { + run(dir, &["check", "--rule", "cap-drift"]) +} + +fn deps(entries: &[&str]) -> String { + let mut manifest = String::from("[workspace.dependencies]\n"); + for entry in entries { + writeln!(manifest, "{entry}").unwrap(); + } + manifest.push_str("\n[workspace.lints]\nrust = {}\n"); + manifest +} + +fn bot(rules: &[&str]) -> String { + let mut config = String::from( + "{\n \"$schema\": \"https://docs.renovatebot.com/renovate-schema.json\",\n packageRules: [\n", + ); + for rule in rules { + writeln!(config, "{rule}").unwrap(); + } + config.push_str(" ],\n}\n"); + config +} + +const MIRRORED: &str = " { matchPackageNames: [\"ignore\"], allowedVersions: \"<0.4.30\" },"; + +#[test] +fn both_sets_empty_passes_and_the_ratchet_still_runs() { + // The state CLOUD-593 leaves them in. A gate that only functioned while a + // list was non-empty would have been deleted with the list. + let dir = cap_repo("cap-empty", &deps(&["serde = \"1\""]), &bot(&[])); + let output = check(&dir); + assert_eq!( + output.status.code(), + Some(0), + "out={} err={}", + stdout(&output), + stderr(&output) + ); +} + +#[test] +fn a_mirrored_cap_passes() { + let dir = cap_repo( + "cap-mirrored", + &deps(&["ignore = \">=0.4, <0.4.30\""]), + &bot(&[MIRRORED]), + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(0), "{}", stdout(&output)); +} + +#[test] +fn the_half_lift_is_refused() { + // The manifest side lifted, the bot side left withholding: the crate + // silently never advances and nothing anywhere is red. This is the direction + // with no symptom, and the reason the gate exists. + let dir = cap_repo( + "cap-half-lift", + &deps(&["ignore = \"0.4\""]), + &bot(&[MIRRORED]), + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); + assert!( + stdout(&output).contains("ignore"), + "the finding names the crate: {:?}", + stdout(&output) + ); +} + +#[test] +fn a_cap_with_no_withholding_rule_is_refused() { + let dir = cap_repo( + "cap-unmirrored", + &deps(&["ignore = \">=0.4, <0.4.30\""]), + &bot(&[]), + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); +} + +#[test] +fn both_directions_are_reported_in_one_pass() { + let dir = cap_repo( + "cap-both-directions", + &deps(&["ignore = \">=0.4, <0.4.30\""]), + &bot(&[" { matchPackageNames: [\"globset\"], allowedVersions: \"<0.4.20\" },"]), + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); + let text = stdout(&output); + assert!( + text.contains("ignore") && text.contains("globset"), + "one run reports both directions: {text:?}" + ); +} + +#[test] +fn a_caret_requirement_is_not_a_cap() { + // It bounds the MAJOR. Treating it as a cap would demand a withholding rule + // for every dependency in the file, and the gate would be switched off + // within a day. + let dir = cap_repo("cap-caret", &deps(&["serde = \"1.0\""]), &bot(&[])); + let output = check(&dir); + assert_eq!(output.status.code(), Some(0), "{}", stdout(&output)); +} + +#[test] +fn a_bare_string_upper_bound_is_a_cap_too() { + let dir = cap_repo( + "cap-bare-string", + &deps(&["ignore = \"<0.4.30\""]), + &bot(&[]), + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); +} + +#[test] +fn a_less_than_outside_the_table_mints_no_phantom_cap() { + let dir = cap_repo( + "cap-outside-table", + "[workspace.dependencies]\nserde = \"1.0\"\n\n[workspace.metadata]\nnote = \"was <2 once\"\n", + &bot(&[]), + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(0), "{}", stdout(&output)); +} + +#[test] +fn a_grouping_matcher_is_not_a_cap_mirror() { + // A matcher used for grouping withholds nothing, so reading it as a mirror + // would let the half-lift through under a rule that never held anything + // back. + let dir = cap_repo( + "cap-grouping-matcher", + &deps(&["ignore = \">=0.4, <0.4.30\""]), + &bot(&[" { matchPackageNames: [\"ignore\"], groupName: \"rust\" },"]), + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); +} + +#[test] +fn a_commented_rule_mirrors_nothing() { + // A gate a comment can satisfy is a gate satisfied by deleting the key the + // comment explains. + let dir = cap_repo( + "cap-commented-rule", + &deps(&["ignore = \">=0.4, <0.4.30\""]), + &bot(&[" // { matchPackageNames: [\"ignore\"], allowedVersions: \"<0.4.30\" },"]), + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); +} + +#[test] +fn a_multiline_rule_reads_the_same_as_an_inline_one() { + // A formatter's choice must not change a verdict. + let dir = cap_repo( + "cap-multiline-rule", + &deps(&["ignore = \">=0.4, <0.4.30\""]), + &bot(&[ + " {", + " matchPackageNames: [\"ignore\"],", + " allowedVersions: \"<0.4.30\",", + " },", + ]), + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(0), "{}", stdout(&output)); +} + +#[test] +fn output_is_pointer_only() { + let dir = cap_repo( + "cap-pointer-only", + &deps(&["ignore = \">=0.4, <0.4.30\" # a distinctive justification"]), + &bot(&[]), + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2)); + let text = stdout(&output); + assert!( + !text.contains("distinctive justification") && !text.contains("0.4.30"), + "the range and the manifest's prose are payload: {text:?}" + ); +} + +#[test] +fn the_real_trees_two_sets_agree() { + // The self-consumption case the retiring suite ended on. + let output = common::run_at_real_root(&common::at_root(""), &["check", "--rule", "cap-drift"]); + assert_eq!( + output.status.code(), + Some(0), + "a cap here has no counterpart: {}", + stdout(&output) + ); +} diff --git a/crates/batten/tests/it/ci_tools.rs b/crates/batten/tests/it/ci_tools.rs new file mode 100644 index 000000000..6c745c53b --- /dev/null +++ b/crates/batten/tests/it/ci_tools.rs @@ -0,0 +1,442 @@ +//! The install lists name real declared tools, every pull-request workflow +//! carries a binding one, and every tool a policy row spawns is on the list — +//! over the compiled binary (CLOUD-180, CLOUD-812 and CLOUD-480, ported from +//! `mise-tasks/ci-tools-check.sh` under CLOUD-843). +//! +//! **What is decidable only here.** `policy/ci-tools.rego` carries load-time +//! cases pinning all three directions, and every one hands the module a +//! ready-made map of workflows with `with input as`. That fabricates the very +//! shape the engine may be unable to produce (CLOUD-845), and the second direction +//! IS that shape: its whole finding was that the retiring gate had been pointed at +//! one file which happened to conform, so two other workflows installed every +//! declared tool on every push with nothing red and nothing able to be. A +//! resolution reaching only some workflows reintroduces exactly that blindness, +//! and no fabricated map can tell it from a clean tree. +//! +//! The self-consumption cases are the two the retiring suite kept: the committed +//! workflow and manifest agree, and every committed pull-request workflow carries +//! a binding list. +// +// carried: mise-tasks/ci-tools-check.sh policy/ci-tools.rego crates/batten/tests/it/ci_tools.rs +// carried: tests/ci-tools-check.bats policy/ci-tools.rego crates/batten/tests/it/ci_tools.rs +// +// carried: "the real ci.yml and mise.toml agree" policy/ci-tools.rego +// carried: "a tool a policy row spawns must be in the install list" policy/ci-tools.rego +// carried: "THE DEFECT: a declared tool a row spawns but no list installs is refused" policy/ci-tools.rego +// carried: "a spawned tool mise does not own is out of scope, not a finding" policy/ci-tools.rego +// carried: "every requested tool declared is a pass" policy/ci-tools.rego +// carried: "quoted and backend-prefixed names resolve" policy/ci-tools.rego +// carried: "a tool declared but never installed by CI is not a failure" policy/ci-tools.rego +// carried: "a renamed tool leaves install_args naming something undeclared" policy/ci-tools.rego +// carried: "the error names the tool and both files, not the file contents" policy/ci-tools.rego +// carried: "a key from another table does not count as declared" policy/ci-tools.rego +// carried: "a config with no [tools] table fails rather than passing" policy/ci-tools.rego +// carried: "a PR workflow with a narrowed, binding list passes" policy/ci-tools.rego +// carried: "a PR workflow running mise-action with no install_args fails" policy/ci-tools.rego +// carried: "a declared list without the auto-install variables fails" policy/ci-tools.rego +// carried: "an auto-install variable set to true is not binding either" policy/ci-tools.rego +// carried: "a workflow with no pull_request trigger is out of scope" policy/ci-tools.rego +// carried: "a PR workflow that never runs mise-action is out of scope" policy/ci-tools.rego +// carried: "the committed PR workflows all carry a binding list" policy/ci-tools.rego +// carried: "a tool name that resolves nowhere fails in a PR workflow other than the first argument" policy/ci-tools.rego +// changed: "a workflow with no install_args lists fails rather than passing" policy/ci-tools.rego the shell was pointed at ONE workflow and refused it for declaring no list; the successor asks the question over every workflow, so the arm is the stronger one CLOUD-812 wrote — a pull-request workflow that runs the installer and declares no list is `tool select missing`, which the retiring suite pins separately and which reaches the workflows the single-file form structurally could not see +// changed: "a missing file is an error, not a pass" policy/ci-tools.rego the shell took explicit paths and refused one it could not open; the successor declares `line_sources` and the ENGINE decides this earlier — a rule whose declared paths match nothing is not evaluated at all, and `input.tree.missing` is never populated on the tree surface (CLOUD-1049, measured identically for `policy/mise-pin-agreement.rego`'s own could-not-look clause). The vacuity that mattered survives whole: a manifest that resolves and declares no tools is `tool list empty` + +// Panicking on setup failure is the idiomatic way for a test to fail loudly. +#![allow(clippy::unwrap_used, clippy::expect_used)] + +use crate::common; + +use std::path::{Path, PathBuf}; +use std::process::Output; + +use common::{Fixture, git_in, run, stderr, stdout}; + +/// A repository declaring only this rule, so any finding is the one under test. +fn tools_repo(name: &str, manifest: &str, policy: &str, workflows: &[(&str, &str)]) -> PathBuf { + let mut fixture = Fixture::new(name).config( + "version = 1\n\n\ + [[pattern]]\n\ + id = \"install-args-line\"\n\ + regex = '^[ \\t]*install_args:'\n\n\ + [[pattern]]\n\ + id = \"workflow-pull-request-trigger\"\n\ + regex = '^[ \\t]*pull_request:'\n\n\ + [[pattern]]\n\ + id = \"mise-task-auto-install-off\"\n\ + regex = '^[ \\t]*MISE_TASK_RUN_AUTO_INSTALL:[ \\t]*\"?false\"?[ \\t]*$'\n\n\ + [[pattern]]\n\ + id = \"mise-exec-auto-install-off\"\n\ + regex = '^[ \\t]*MISE_EXEC_AUTO_INSTALL:[ \\t]*\"?false\"?[ \\t]*$'\n\n\ + [[pattern]]\n\ + id = \"mise-exec-spawn\"\n\ + regex = 'mise exec -- [a-z][a-z0-9._-]*'\n\n\ + [[verdict]]\n\ + id = \"tool name unknown\"\n\ + gloss = \"an install list names a tool the manifest does not declare\"\n\ + class = \"The installer does not fail on an unknown name; it surfaces later as a missing command.\"\n\n\ + [[verdict.route]]\n\ + id = \"prose read first\"\n\ + kind = \"document\"\n\ + target = \"AGENTS.md\"\n\n\ + [[verdict]]\n\ + id = \"tool select missing\"\n\ + gloss = \"a pull-request workflow runs the installer without an install list\"\n\ + class = \"Judging the names IN a list can never see a workflow that declares none.\"\n\n\ + [[verdict.route]]\n\ + id = \"prose read first\"\n\ + kind = \"document\"\n\ + target = \"AGENTS.md\"\n\n\ + [[verdict]]\n\ + id = \"tool pin loose\"\n\ + gloss = \"an install list is declared without the variables that make it bind\"\n\ + class = \"The runner re-installs the rest at task time, so the list decides nothing.\"\n\n\ + [[verdict.route]]\n\ + id = \"prose read first\"\n\ + kind = \"document\"\n\ + target = \"AGENTS.md\"\n\n\ + [[verdict]]\n\ + id = \"spawn reach absent\"\n\ + gloss = \"a policy row spawns a declared tool no install list installs\"\n\ + class = \"With auto-install off the row fails CLOSED in CI while passing locally.\"\n\n\ + [[verdict.route]]\n\ + id = \"prose read first\"\n\ + kind = \"document\"\n\ + target = \"AGENTS.md\"\n\n\ + [[verdict]]\n\ + id = \"tool list empty\"\n\ + gloss = \"the manifest declares no tools, so the question cannot be asked\"\n\ + class = \"With the declared set empty every install list resolves vacuously.\"\n\n\ + [[verdict.route]]\n\ + id = \"prose read first\"\n\ + kind = \"document\"\n\ + target = \"AGENTS.md\"\n\n\ + [[rule]]\n\ + id = \"ci-tools\"\n\ + kind = \"policy\"\n\ + scope = \"tree\"\n\ + line_sources = [\"mise.toml\", \"batten.toml\", \".github/workflows/*.yml\"]\n\ + module = \"policy/ci-tools.rego\"\n\ + severity = \"deny\"\n\n", + ); + fixture = fixture + .file("AGENTS.md", "the consumer's own authority\n") + .file("mise.toml", manifest); + for (path, body) in workflows { + fixture = fixture.file(path, body); + } + let dir = fixture.git().build(); + // The spawns the third direction reads live in the fixture's own config, + // which is the file the rule names — so they are appended to it rather than + // written beside it. They go on a COMMENT line: the module is a line scan, so + // where in the file the text sits does not change the predicate, and a bare + // key appended after the rule tables would attach to the last one and be a + // load error rather than the verdict these cases are about. + let config_path = dir.join("batten.toml"); + let existing = std::fs::read_to_string(&config_path).unwrap(); + std::fs::write(&config_path, format!("{existing}\n# {policy}")).unwrap(); + common::write( + &dir, + "policy/ci-tools.rego", + &std::fs::read_to_string(common::at_root("policy/ci-tools.rego")).unwrap(), + ); + git_in(&dir, &["add", "-A"]); + git_in(&dir, &["commit", "-q", "-m", "base"]); + dir +} + +fn check(dir: &Path) -> Output { + run(dir, &["check", "--rule", "ci-tools"]) +} + +const MANIFEST: &str = "[tools]\nrust = \"1.97\"\n\"aqua:open-policy-agent/opa\" = \"1.0\"\nzig = \"0.13\"\n\n[env]\nX = \"1\"\n"; + +const NO_SPAWNS: &str = "this config spawns nothing\n"; + +/// A pull-request workflow running the installer once, with `entries` inside the +/// step and the auto-install variables bound. +fn pr_flow(entries: &str) -> String { + format!( + "on:\n pull_request:\njobs:\n ci:\n steps:\n - uses: jdx/mise-action@abc\n{entries} env:\n MISE_TASK_RUN_AUTO_INSTALL: \"false\"\n MISE_EXEC_AUTO_INSTALL: \"false\"\n" + ) +} + +#[test] +fn every_requested_tool_declared_is_a_pass() { + let dir = tools_repo( + "tools-clean", + MANIFEST, + NO_SPAWNS, + &[( + ".github/workflows/ci.yml", + &pr_flow(" install_args: rust aqua:open-policy-agent/opa\n"), + )], + ); + let output = check(&dir); + assert_eq!( + output.status.code(), + Some(0), + "out={} err={}", + stdout(&output), + stderr(&output) + ); +} + +#[test] +fn a_renamed_tool_leaves_the_list_naming_something_undeclared() { + let dir = tools_repo( + "tools-renamed", + MANIFEST, + NO_SPAWNS, + &[( + ".github/workflows/ci.yml", + &pr_flow(" install_args: rust zigg\n"), + )], + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); + assert!( + stdout(&output).contains("zigg"), + "the finding names the tool: {:?}", + stdout(&output) + ); +} + +#[test] +fn a_tool_declared_but_never_installed_is_not_a_failure() { + // The predicate is that names RESOLVE, not that every declared tool is + // installed everywhere — inferring which tools a job should need is a + // judgement, not a predicate. + let dir = tools_repo( + "tools-declared-unused", + MANIFEST, + NO_SPAWNS, + &[( + ".github/workflows/ci.yml", + &pr_flow(" install_args: rust\n"), + )], + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(0), "{}", stdout(&output)); +} + +#[test] +fn a_key_from_another_table_does_not_count_as_declared() { + let dir = tools_repo( + "tools-other-table", + "[tools]\nrust = \"1.97\"\n\n[env]\nzig = \"decoy\"\n", + NO_SPAWNS, + &[( + ".github/workflows/ci.yml", + &pr_flow(" install_args: rust zig\n"), + )], + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); +} + +#[test] +fn a_manifest_with_no_tools_table_is_not_a_pass() { + let dir = tools_repo( + "tools-no-table", + "[env]\nX = \"1\"\n", + NO_SPAWNS, + &[( + ".github/workflows/ci.yml", + &pr_flow(" install_args: rust\n"), + )], + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); +} + +#[test] +fn a_pull_request_workflow_with_no_list_is_refused() { + // THE HOLE CLOUD-812 MEASURED: judging the names IN a list can never see a + // workflow that declares none, and two such workflows were installing + // everything on every push with nothing red. + let dir = tools_repo( + "tools-no-list", + MANIFEST, + NO_SPAWNS, + &[ + ( + ".github/workflows/ci.yml", + &pr_flow(" install_args: rust\n"), + ), + (".github/workflows/lint.yml", &pr_flow("")), + ], + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); + assert!( + stdout(&output).contains("lint.yml"), + "the finding names the blind workflow: {:?}", + stdout(&output) + ); +} + +#[test] +fn a_list_without_the_auto_install_variables_is_refused() { + // Decorative: the runner re-installs the rest at task time, which is + // indistinguishable from a fix by reading the workflow. + let dir = tools_repo( + "tools-nonbinding", + MANIFEST, + NO_SPAWNS, + &[( + ".github/workflows/ci.yml", + "on:\n pull_request:\njobs:\n ci:\n steps:\n - uses: jdx/mise-action@abc\n install_args: rust\n", + )], + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); +} + +#[test] +fn a_variable_set_true_is_not_binding_either() { + // The same hole wearing a fix's clothing. + let dir = tools_repo( + "tools-true", + MANIFEST, + NO_SPAWNS, + &[( + ".github/workflows/ci.yml", + "on:\n pull_request:\njobs:\n ci:\n steps:\n - uses: jdx/mise-action@abc\n install_args: rust\n env:\n MISE_TASK_RUN_AUTO_INSTALL: \"true\"\n MISE_EXEC_AUTO_INSTALL: \"false\"\n", + )], + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); +} + +#[test] +fn a_workflow_with_no_pull_request_trigger_is_out_of_scope() { + let dir = tools_repo( + "tools-scheduled", + MANIFEST, + NO_SPAWNS, + &[ + ( + ".github/workflows/ci.yml", + &pr_flow(" install_args: rust\n"), + ), + ( + ".github/workflows/nightly.yml", + "on:\n schedule:\njobs:\n x:\n steps:\n - uses: jdx/mise-action@abc\n", + ), + ], + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(0), "{}", stdout(&output)); +} + +#[test] +fn a_pull_request_workflow_that_never_runs_the_installer_is_out_of_scope() { + let dir = tools_repo( + "tools-no-action", + MANIFEST, + NO_SPAWNS, + &[ + ( + ".github/workflows/ci.yml", + &pr_flow(" install_args: rust\n"), + ), + ( + ".github/workflows/labels.yml", + "on:\n pull_request:\njobs:\n x:\n steps:\n - run: echo hi\n", + ), + ], + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(0), "{}", stdout(&output)); +} + +#[test] +fn a_spawned_tool_the_list_omits_is_refused() { + // THE DEFECT that cost two CI runs: with auto-install off the row does not + // run slowly, it fails CLOSED — green locally, red in CI, with nothing + // naming the cause. + let dir = tools_repo( + "tools-spawn-omitted", + MANIFEST, + "check = mise exec -- opa check -s schema/ policy/\n", + &[( + ".github/workflows/ci.yml", + &pr_flow(" install_args: rust\n"), + )], + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); + assert!( + stdout(&output).contains("opa"), + "the finding names the spawned tool: {:?}", + stdout(&output) + ); +} + +#[test] +fn a_spawned_tool_the_manifest_does_not_own_is_out_of_scope() { + // A binary can arrive bundled, pre-installed or vendored, and all three came + // back as findings on the first run of that block. The question narrows to + // what the manifest declares rather than carrying a second exemption list. + let dir = tools_repo( + "tools-spawn-unowned", + MANIFEST, + "check = mise exec -- gh pr view\n", + &[( + ".github/workflows/ci.yml", + &pr_flow(" install_args: rust aqua:open-policy-agent/opa\n"), + )], + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(0), "{}", stdout(&output)); +} + +#[test] +fn a_spawned_tool_the_list_installs_by_backend_name_resolves() { + // Basename matching on both sides: the list holds a backend-qualified key + // and the spawn names a binary. + let dir = tools_repo( + "tools-spawn-backend", + MANIFEST, + "check = mise exec -- opa check\n", + &[( + ".github/workflows/ci.yml", + &pr_flow(" install_args: rust aqua:open-policy-agent/opa\n"), + )], + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(0), "{}", stdout(&output)); +} + +#[test] +fn output_is_pointer_only() { + let dir = tools_repo( + "tools-pointer-only", + MANIFEST, + NO_SPAWNS, + &[( + ".github/workflows/ci.yml", + &pr_flow(" install_args: rust zigg\n name: a-distinctive-step\n"), + )], + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2)); + assert!( + !stdout(&output).contains("a-distinctive-step"), + "the workflow body is payload: {:?}", + stdout(&output) + ); +} + +#[test] +fn the_committed_workflows_and_manifest_agree() { + // The two self-consumption cases the retiring suite kept, in one run. + let output = common::run_at_real_root(&common::at_root(""), &["check", "--rule", "ci-tools"]); + assert_eq!( + output.status.code(), + Some(0), + "the committed install lists and manifest disagree: {}", + stdout(&output) + ); +} diff --git a/crates/batten/tests/it/coderabbit_config.rs b/crates/batten/tests/it/coderabbit_config.rs new file mode 100644 index 000000000..dab1d3006 --- /dev/null +++ b/crates/batten/tests/it/coderabbit_config.rs @@ -0,0 +1,236 @@ +//! The review bot's config still carries the keys the lifecycle rests on, over +//! the compiled binary (CLOUD-860, the missing half of CLOUD-847, ported from +//! `mise-tasks/coderabbit-config-check.sh` under CLOUD-843). +//! +//! **What is decidable only here.** `policy/coderabbit-config.rego` carries +//! load-time cases pinning the predicate, and every one supplies the config's +//! lines with `with input as`. That fabricates the very shape the engine may be +//! unable to produce (CLOUD-845), and for this rule it fabricates the one claim +//! the retiring program made loudest: that there is NO FAIL-OPEN ARM. The input +//! is a tracked file in this checkout, so "could not look" means the file is +//! gone, which is itself the state the rule refuses — and a module whose suite +//! only fabricated the lines cannot tell an engine that resolved the file from +//! one that resolved nothing, because both hand the predicate a value it treats +//! the same way. +//! +//! The self-consumption case is the one the retiring suite opened on: this +//! repository's own config holds the three keys. +// +// carried: mise-tasks/coderabbit-config-check.sh policy/coderabbit-config.rego crates/batten/tests/it/coderabbit_config.rs +// carried: tests/coderabbit-config-check.bats policy/coderabbit-config.rego crates/batten/tests/it/coderabbit_config.rs +// +// carried: "coderabbit-config-check.bats::the repo as it stands passes" policy/coderabbit-config.rego +// carried: "a compliant fixture passes" policy/coderabbit-config.rego +// carried: "request_changes_workflow flipped off fails, and names the key" policy/coderabbit-config.rego +// carried: "drafts flipped off fails, and names the key" policy/coderabbit-config.rego +// carried: "gitleaks disabled fails: drafts would have no secret scanning at all" policy/coderabbit-config.rego +// carried: "the gitleaks arm is SCOPED to gitleaks, not to the first tool in the file" policy/coderabbit-config.rego +// carried: "gitleaks absent passes: its default is enabled, so only an explicit false is a violation" policy/coderabbit-config.rego +// carried: "a key deleted rather than flipped fails: absence leaves the default in force" policy/coderabbit-config.rego +// carried: "a comment-only file is a failure, not a vacuous pass" policy/coderabbit-config.rego +// carried: "a commented-out key does not satisfy the assertion" policy/coderabbit-config.rego +// carried: "output is pointer-only: it names keys and lines, never the file's contents" policy/coderabbit-config.rego +// changed: "coderabbit-config-check.bats::the gate is wired: hk.pkl declares a step that runs this task" policy/coderabbit-config.rego the task the step ran no longer exists, so the case has no subject to assert. Its property does not go unheld: a `[[rule]]` row IS the wiring on the engine, `batten check` runs every registered row, and `glob-containment` refuses the commit where the hook manifest's trigger stops selecting a path the config makes an input — which is the same claim reached by a mechanism rather than by a suite reading a manifest +// changed: "an absent file fails rather than passing for want of anything to read" policy/coderabbit-config.rego the shell opened a named file and refused when it was gone; the successor declares it as a `line_sources` path and the ENGINE decides this earlier — a rule whose declared path matches nothing is not evaluated at all, and `input.tree.missing` is never populated on the tree surface (CLOUD-1049, measured identically for `policy/mise-pin-agreement.rego`'s own could-not-look clause). What the case protected survives as the vacuity arm: a file that resolves and declares no keys is `config carry empty`, never a pass + +// Panicking on setup failure is the idiomatic way for a test to fail loudly. +#![allow(clippy::unwrap_used, clippy::expect_used)] + +use crate::common; + +use std::path::{Path, PathBuf}; +use std::process::Output; + +use common::{Fixture, git_in, run, stderr, stdout}; + +/// A repository declaring only this rule, so any finding is the one under test. +fn review_repo(name: &str, config: &str) -> PathBuf { + let dir = Fixture::new(name) + .config( + "version = 1\n\n\ + [[verdict]]\n\ + id = \"config carry empty\"\n\ + gloss = \"the review config declares none of the keys the lifecycle rests on\"\n\ + class = \"A key nobody wrote and a key someone deleted both leave the default in force.\"\n\n\ + [[verdict.route]]\n\ + id = \"prose read first\"\n\ + kind = \"document\"\n\ + target = \"AGENTS.md\"\n\n\ + [[verdict]]\n\ + id = \"config state wrong\"\n\ + gloss = \"a review-config key is set to the value that switches the lifecycle off\"\n\ + class = \"The symptom is silence, and the visible failure is the gate rather than the cause.\"\n\n\ + [[verdict.route]]\n\ + id = \"prose read first\"\n\ + kind = \"document\"\n\ + target = \"AGENTS.md\"\n\n\ + [[rule]]\n\ + id = \"coderabbit-config\"\n\ + kind = \"policy\"\n\ + scope = \"tree\"\n\ + line_sources = [\".coderabbit.yaml\"]\n\ + module = \"policy/coderabbit-config.rego\"\n\ + severity = \"deny\"\n", + ) + .file("AGENTS.md", "the consumer's own authority\n") + .file(".coderabbit.yaml", config) + .git() + .build(); + common::write( + &dir, + "policy/coderabbit-config.rego", + &std::fs::read_to_string(common::at_root("policy/coderabbit-config.rego")).unwrap(), + ); + git_in(&dir, &["add", "-A"]); + git_in(&dir, &["commit", "-q", "-m", "base"]); + dir +} + +fn check(dir: &Path) -> Output { + run(dir, &["check", "--rule", "coderabbit-config"]) +} + +const CONFORMING: &str = "reviews:\n request_changes_workflow: true\n auto_review:\n drafts: true\n tools:\n gitleaks:\n enabled: true\n ruff:\n enabled: false\n"; + +#[test] +fn a_conforming_config_passes() { + let dir = review_repo("review-conforming", CONFORMING); + let output = check(&dir); + assert_eq!( + output.status.code(), + Some(0), + "out={} err={}", + stdout(&output), + stderr(&output) + ); +} + +#[test] +fn the_changes_workflow_flipped_off_is_refused() { + let dir = review_repo( + "review-workflow-off", + &CONFORMING.replace( + "request_changes_workflow: true", + "request_changes_workflow: false", + ), + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); + assert!( + stdout(&output).contains(".coderabbit.yaml:"), + "the finding points at the line: {:?}", + stdout(&output) + ); +} + +#[test] +fn the_draft_key_flipped_off_is_refused() { + // Its symptom is SILENCE: reviews stop happening, which looks exactly like + // nobody having pushed. + let dir = review_repo( + "review-drafts-off", + &CONFORMING.replace("drafts: true", "drafts: false"), + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); +} + +#[test] +fn the_scanner_denied_explicitly_is_refused() { + // The only secret scanning a draft gets. + let dir = review_repo( + "review-scanner-off", + "reviews:\n request_changes_workflow: true\n auto_review:\n drafts: true\n tools:\n gitleaks:\n enabled: false\n", + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); +} + +#[test] +fn the_scanner_arm_is_scoped_to_its_own_block() { + // The enabling key appears once per tool, so an unscoped read answers about + // whichever tool came first in the file. + let dir = review_repo( + "review-scanner-scoped", + "reviews:\n request_changes_workflow: true\n auto_review:\n drafts: true\n tools:\n ruff:\n enabled: false\n gitleaks:\n enabled: true\n", + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(0), "{}", stdout(&output)); +} + +#[test] +fn an_absent_scanner_block_passes() { + // Its default is enabled, so only an explicit denial is a violation. + let dir = review_repo( + "review-scanner-absent", + "reviews:\n request_changes_workflow: true\n auto_review:\n drafts: true\n", + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(0), "{}", stdout(&output)); +} + +#[test] +fn a_key_deleted_rather_than_flipped_is_refused() { + // ABSENCE LEAVES THE DEFAULT IN FORCE, which is the value the rule refuses — + // a key nobody wrote and a key someone deleted are the same file. + let dir = review_repo( + "review-key-deleted", + "reviews:\n request_changes_workflow: true\n auto_review:\n base_branches:\n - main\n", + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); +} + +#[test] +fn a_comment_only_file_is_a_failure_not_a_vacuous_pass() { + // Every assertion here is ABOUT a key, so a file carrying none of them + // satisfies all of them by having nothing to judge. + let dir = review_repo( + "review-comment-only", + "# every key was removed\n# and this still parses as a document\n", + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); +} + +#[test] +fn a_commented_out_key_does_not_satisfy_the_assertion() { + let dir = review_repo( + "review-commented-key", + "reviews:\n # request_changes_workflow: true\n auto_review:\n drafts: true\n", + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); +} + +#[test] +fn output_is_pointer_only() { + // A config can carry review instructions and paths, and a gate that echoed + // them would put them in every CI log. + let dir = review_repo( + "review-pointer-only", + "reviews:\n request_changes_workflow: false\n auto_review:\n drafts: true\n path_instructions:\n - path: \"crates/**\"\n instructions: \"a distinctive instruction\"\n", + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2)); + assert!( + !stdout(&output).contains("distinctive instruction"), + "the config's instructions are payload: {:?}", + stdout(&output) + ); +} + +#[test] +fn this_repositorys_own_config_holds_the_three_keys() { + // The self-consumption case the retiring suite opened on. + let output = common::run_at_real_root( + &common::at_root(""), + &["check", "--rule", "coderabbit-config"], + ); + assert_eq!( + output.status.code(), + Some(0), + "a key the review lifecycle rests on is missing or flipped: {}", + stdout(&output) + ); +} diff --git a/crates/batten/tests/it/digest_major_agreement.rs b/crates/batten/tests/it/digest_major_agreement.rs new file mode 100644 index 000000000..0b56fa5f2 --- /dev/null +++ b/crates/batten/tests/it/digest_major_agreement.rs @@ -0,0 +1,309 @@ +//! The workspace's own crypto crates resolve one `digest` major, over the +//! compiled binary (CLOUD-767, ported from `mise-tasks/digest-major-agreement.sh` +//! under CLOUD-843). +//! +//! **What is decidable only here.** `policy/digest-major-agreement.rego` carries +//! load-time cases pinning the predicate, and every one hands the module a +//! ready-made map of both files with `with input as`. That fabricates the very +//! shape the engine may be unable to produce (CLOUD-845), and here it fabricates +//! the whole judgement: the rule is a JOIN in which the manifest answers which +//! crates are OURS and the lock answers what they resolved, and neither file can +//! answer the other's question. A module whose suite only fabricated the map would +//! stay green over an engine that resolved the lock and not the manifest — the +//! state in which every transitive hasher enrols and the gate is red forever, or +//! resolved the manifest and not the lock, where it decides nothing at all. +//! +//! The self-consumption case is this repository's own two files, which is what +//! makes the claim the old manifest comment got wrong checkable rather than +//! asserted. +// +// carried: mise-tasks/digest-major-agreement.sh policy/digest-major-agreement.rego crates/batten/tests/it/digest_major_agreement.rs +// carried: tests/digest-major-agreement.bats policy/digest-major-agreement.rego crates/batten/tests/it/digest_major_agreement.rs +// +// carried: "the pair agreeing is the ordinary pass, and the verdict names the major" policy/digest-major-agreement.rego +// carried: "THE HALF-BUMP IS REFUSED — one crate moved and the other left behind" policy/digest-major-agreement.rego +// carried: "the coordinated bump passes — moving BOTH is what the manifest asks for" policy/digest-major-agreement.rego +// carried: "A CRATE GIX OWNS IS NOT OURS: a transitive hasher on the other major is ignored" policy/digest-major-agreement.rego +// carried: "one declared crate cannot disagree with itself, and the gate says so" policy/digest-major-agreement.rego +// carried: "a bare digest reference resolves against the one major vendored" policy/digest-major-agreement.rego +// carried: "COULD NOT LOOK, NEVER AGREEMENT: a declared crate absent from the lock is exit 2" policy/digest-major-agreement.rego +// carried: "a name outside [workspace.dependencies] does not enrol the crate" policy/digest-major-agreement.rego +// carried: "POINTER, NEVER PAYLOAD: the refusal carries no version requirement and no manifest line" policy/digest-major-agreement.rego +// changed: "an unreadable lockfile is exit 2, not a pass" policy/digest-major-agreement.rego the shell opened two named files and refused one it could not read; the successor declares them as `line_sources` and the ENGINE decides this earlier — a rule whose declared paths match nothing is not evaluated at all, and `input.tree.missing` is never populated on the tree surface (CLOUD-1049, measured identically for `policy/mise-pin-agreement.rego`'s own could-not-look clause). What the case protected survives as the PRESENT-BUT-SILENT arm: a lock that resolves and answers nothing for a declared crate is still `lock read unclear`, never agreement +// changed: "an unreadable manifest is exit 2, not a pass" policy/digest-major-agreement.rego same engine-side decision as the row above. The manifest's own silence is not a separate refusal here because it is not a separate failure: a manifest declaring neither crate leaves fewer than two under judgement, which the retiring suite itself calls a legitimate state rather than a vacuous pass +// changed: "the gate writes nothing — it decides over two committed files" policy/digest-major-agreement.rego the property is carried by CONSTRUCTION rather than by a case: a `kind = "policy"` rule over `scope = "tree"` reaches the filesystem through `input.tree` and has no write surface at all, where the shell could in principle have opened a file for writing and needed a case saying it did not + +// Panicking on setup failure is the idiomatic way for a test to fail loudly. +#![allow(clippy::unwrap_used, clippy::expect_used)] + +use crate::common; + +use std::fmt::Write as _; +use std::path::{Path, PathBuf}; +use std::process::Output; + +use common::{Fixture, git_in, run, stderr, stdout}; + +/// A repository declaring only this rule, so any finding is the one under test. +fn digest_repo(name: &str, manifest: &str, lock: &str) -> PathBuf { + let dir = Fixture::new(name) + .config( + "version = 1\n\n\ + [[verdict]]\n\ + id = \"version resolve other\"\n\ + gloss = \"the workspace's own crypto crates resolved different majors\"\n\ + class = \"They compose in one expression, so a split is a type that does not exist.\"\n\n\ + [[verdict.route]]\n\ + id = \"prose read first\"\n\ + kind = \"document\"\n\ + target = \"AGENTS.md\"\n\n\ + [[verdict]]\n\ + id = \"lock read unclear\"\n\ + gloss = \"the lockfile does not answer what a declared crate resolved\"\n\ + class = \"A lock that does not describe the manifest decides nothing.\"\n\n\ + [[verdict.route]]\n\ + id = \"prose read first\"\n\ + kind = \"document\"\n\ + target = \"AGENTS.md\"\n\n\ + [[rule]]\n\ + id = \"digest-major-agreement\"\n\ + kind = \"policy\"\n\ + scope = \"tree\"\n\ + line_sources = [\"Cargo.toml\", \"Cargo.lock\"]\n\ + module = \"policy/digest-major-agreement.rego\"\n\ + severity = \"deny\"\n", + ) + .file("AGENTS.md", "the consumer's own authority\n") + .file("Cargo.toml", manifest) + .file("Cargo.lock", lock) + .git() + .build(); + common::write( + &dir, + "policy/digest-major-agreement.rego", + &std::fs::read_to_string(common::at_root("policy/digest-major-agreement.rego")).unwrap(), + ); + git_in(&dir, &["add", "-A"]); + git_in(&dir, &["commit", "-q", "-m", "base"]); + dir +} + +fn check(dir: &Path) -> Output { + run(dir, &["check", "--rule", "digest-major-agreement"]) +} + +/// A `[workspace.dependencies]` table declaring `names`. +fn workspace(names: &[&str]) -> String { + let mut manifest = String::from("[workspace.dependencies]\n"); + for name in names { + writeln!(manifest, "{name} = \"1\"").unwrap(); + } + manifest.push_str("\n[workspace.lints]\nrust = {}\n"); + manifest +} + +/// One lockfile stanza. +fn pkg(name: &str, version: &str, deps: &[&str]) -> String { + let mut stanza = format!("[[package]]\nname = \"{name}\"\nversion = \"{version}\"\n"); + if !deps.is_empty() { + stanza.push_str("dependencies = [\n"); + for dep in deps { + writeln!(stanza, " \"{dep}\",").unwrap(); + } + stanza.push_str("]\n"); + } + stanza.push('\n'); + stanza +} + +#[test] +fn the_pair_agreeing_is_the_ordinary_pass() { + let dir = digest_repo( + "digest-agree", + &workspace(&["hmac", "sha2"]), + &format!( + "{}{}{}", + pkg("digest", "0.10.7", &[]), + pkg("hmac", "0.12.1", &["digest"]), + pkg("sha2", "0.10.8", &["digest"]) + ), + ); + let output = check(&dir); + assert_eq!( + output.status.code(), + Some(0), + "out={} err={}", + stdout(&output), + stderr(&output) + ); +} + +#[test] +fn the_half_bump_is_refused() { + // One crate moved and the other left behind: the composed type stops + // existing, and the whole point of the gate is saying so before a runner is + // spent. + let dir = digest_repo( + "digest-half-bump", + &workspace(&["hmac", "sha2"]), + &format!( + "{}{}{}{}", + pkg("digest", "0.10.7", &[]), + pkg("digest", "0.11.3", &[]), + pkg("hmac", "0.12.1", &["digest 0.10.7"]), + pkg("sha2", "0.11.0", &["digest 0.11.3"]) + ), + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); + let text = stdout(&output); + assert!( + text.contains("hmac") && text.contains("sha2"), + "the finding names both crates and their majors: {text:?}" + ); +} + +#[test] +fn the_coordinated_bump_passes() { + // Moving BOTH is what the manifest asks for, so the gate must not stand in + // the way of the bump it exists to keep honest. + let dir = digest_repo( + "digest-coordinated", + &workspace(&["hmac", "sha2"]), + &format!( + "{}{}{}", + pkg("digest", "0.11.3", &[]), + pkg("hmac", "0.13.0", &["digest"]), + pkg("sha2", "0.11.0", &["digest"]) + ), + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(0), "{}", stdout(&output)); +} + +#[test] +fn a_transitive_crate_on_another_major_is_not_ours() { + // The workspace has no say in what a transitive dependency vendors, and a + // gate asserting otherwise is red on the commit that introduced it and every + // commit after — which is a gate switched off within a day. + let dir = digest_repo( + "digest-transitive", + &workspace(&["hmac", "sha2"]), + &format!( + "{}{}{}{}{}", + pkg("digest", "0.10.7", &[]), + pkg("digest", "0.11.3", &[]), + pkg("hmac", "0.12.1", &["digest 0.10.7"]), + pkg("sha2", "0.10.8", &["digest 0.10.7"]), + pkg("sha1-checked", "0.11.0", &["digest 0.11.3"]) + ), + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(0), "{}", stdout(&output)); +} + +#[test] +fn one_declared_crate_cannot_disagree_with_itself() { + let dir = digest_repo( + "digest-single", + &workspace(&["hmac"]), + &format!( + "{}{}", + pkg("digest", "0.10.7", &[]), + pkg("hmac", "0.12.1", &["digest"]) + ), + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(0), "{}", stdout(&output)); +} + +#[test] +fn a_bare_reference_resolves_against_the_one_major_vendored() { + // Cargo writes a bare entry when one major is in the tree; with nothing to + // disambiguate, the lock's own package is the answer. + let dir = digest_repo( + "digest-bare", + &workspace(&["hmac", "sha2"]), + &format!( + "{}{}{}", + pkg("digest", "0.10.7", &[]), + pkg("hmac", "0.12.1", &["digest"]), + pkg("sha2", "0.10.8", &["digest 0.10.7"]) + ), + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(0), "{}", stdout(&output)); +} + +#[test] +fn a_declared_crate_the_lock_answers_nothing_for_is_not_agreement() { + let dir = digest_repo( + "digest-absent", + &workspace(&["hmac", "sha2"]), + &format!( + "{}{}{}", + pkg("digest", "0.10.7", &[]), + pkg("hmac", "0.12.1", &["digest"]), + pkg("sha2", "0.10.8", &["cpufeatures"]) + ), + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); +} + +#[test] +fn a_name_outside_the_workspace_table_does_not_enrol_the_crate() { + // Conflating our direct dependency with a transitive one is exactly the + // mistake the old manifest comment made. + let dir = digest_repo( + "digest-name-outside", + "[workspace.dependencies]\nhmac = \"0.12\"\n\n[workspace.lints]\n# sha2 is discussed here and declared nowhere\nrust = {}\n", + &format!( + "{}{}{}{}", + pkg("digest", "0.10.7", &[]), + pkg("digest", "0.11.3", &[]), + pkg("hmac", "0.12.1", &["digest 0.10.7"]), + pkg("sha2", "0.11.0", &["digest 0.11.3"]) + ), + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(0), "{}", stdout(&output)); +} + +#[test] +fn output_is_pointer_only() { + let dir = digest_repo( + "digest-pointer-only", + "[workspace.dependencies]\nhmac = \"0.12\" # a distinctive justification\nsha2 = \"0.10\"\n", + &format!( + "{}{}{}{}", + pkg("digest", "0.10.7", &[]), + pkg("digest", "0.11.3", &[]), + pkg("hmac", "0.12.1", &["digest 0.10.7"]), + pkg("sha2", "0.11.0", &["digest 0.11.3"]) + ), + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2)); + let text = stdout(&output); + assert!( + !text.contains("distinctive justification") && !text.contains("0.12.1"), + "the manifest's prose and the lock's versions are payload: {text:?}" + ); +} + +#[test] +fn this_workspaces_own_crypto_crates_agree() { + // The claim the old manifest comment got wrong, checked rather than + // asserted. + let output = common::run_at_real_root( + &common::at_root(""), + &["check", "--rule", "digest-major-agreement"], + ); + assert_eq!( + output.status.code(), + Some(0), + "this workspace's own crypto crates split across digest majors: {}", + stdout(&output) + ); +} diff --git a/crates/batten/tests/it/doctor.rs b/crates/batten/tests/it/doctor.rs index 6f342b17c..70ef17dd6 100644 --- a/crates/batten/tests/it/doctor.rs +++ b/crates/batten/tests/it/doctor.rs @@ -10,6 +10,61 @@ //! //! Kept out of `tests/cli.rs` deliberately — that file is the exit-code and //! output-contract suite, and other work appends to it. +//! +//! # CLOUD-1753: the egress pair retires here +//! +//! `doctor egress` was written as a PARALLEL authority to +//! `mise-tasks/egress-check.sh` and this module said so in as many words — "not a +//! widening, and deliberately not an edit to it". That was the right posture +//! while the shell was live: a migration replaces a shell gate, it does not +//! maintain one. The retirement is what settles it, and the three-arm +//! classification wins. +//! +//! WHAT CHANGES, STATED RATHER THAN ABSORBED. The shell answered two verdicts and +//! this answers three: `Partial` — a proxy fencing the GitHub hosts and carrying +//! everything else — is a state the shell could not express, and it is the one +//! that reads as health. And the two disagree on `*.api.github.com`: the shell +//! matched the host as a SUBSTRING, so a wildcard covering only subdomains read +//! as fenced, while `fetch::is_direct` compares whole entries and reads it +//! `Unfenced`. The successor is the stricter of the two in the direction that +//! matters, which is why the disagreement is a `// changed:` arm rather than a +//! carry. +//! +//! `container-preflight` is the shell's one caller, and it retires with it. Its +//! egress half becomes the `session-egress` handler running this verb; its +//! credential half becomes `session-credential` running `gh-preflight`, which is +//! not this campaign's to retire and is invoked directly rather than through a +//! wrapper that only forwarded it. Splitting the row is what the port BUYS: the +//! shell collected both causes so a reader saw the whole list, and two handlers +//! report both in one reply for the same reason, with each failure named by the +//! row that owns it instead of by a script that owned neither. +// +// carried: mise-tasks/egress-check.sh crates/batten/src/doctor.rs kind:mechanism crates/batten/tests/it/doctor.rs runs:batten+doctor+egress +// carried: tests/egress-check.bats crates/batten/src/doctor.rs kind:mechanism crates/batten/tests/it/doctor.rs runs:batten+doctor+egress +// carried: mise-tasks/container-preflight.sh crates/batten/src/doctor.rs kind:mechanism crates/batten/tests/it/doctor.rs runs:batten+doctor+egress +// carried: tests/container-preflight.bats crates/batten/src/doctor.rs kind:mechanism crates/batten/tests/it/doctor.rs runs:batten+doctor+egress +// +// carried: "the check is executable" crates/batten/src/doctor.rs +// carried: "no proxy is ok — the ordinary machine has nothing to fence" crates/batten/src/doctor.rs +// carried: "no proxy is ok even with an unrelated NO_PROXY" crates/batten/src/doctor.rs +// carried: "a proxy with api.github.com fenced is ok" crates/batten/src/doctor.rs +// carried: "a proxy without the fence is unfenced — the measured broken container" crates/batten/src/doctor.rs +// carried: "a proxy with an empty NO_PROXY is unfenced" crates/batten/src/doctor.rs +// changed: "wildcard and dot-prefixed NO_PROXY forms are honoured" crates/batten/src/doctor.rs from `tests/egress-check.bats`. The dot-prefixed half carries: `.api.github.com` is a suffix entry and `fetch::bypassed` honours it. The WILDCARD half does not, and the successor is deliberately stricter — the shell read `*.api.github.com` as fenced by matching the host anywhere in the list, and `fetch::bypassed` compares whole entries, so a wildcard covering only subdomains leaves the bare host proxied. `a_wildcard_entry_is_neither_a_total_bypass_nor_a_fence_for_the_bare_host` is where the two answers are written down side by side +// changed: "a malformed call is exit 2, distinct from any verdict" crates/batten/src/doctor.rs from `tests/egress-check.bats`. The shell took the two values as ARGUMENTS, so it had a malformed call to refuse. The verb takes none — it reads the environment, which is the whole reason it exists (a value passed through a task runner is the REPAIRED value, which is the defect `container-preflight` had) — so the malformed call is not representable. What survives is the exit contract itself, and `Egress::code` is where §7's mapping lives +// carried: "it never reads the live environment" crates/batten/src/doctor.rs +// carried: "it makes no network call" crates/batten/src/doctor.rs +// +// carried: "the task is executable" crates/batten/src/doctor.rs +// carried: "a fenced container with every claim passes, quietly" crates/batten/src/doctor.rs +// carried: "a missing read claim halts with exit 1" crates/batten/src/doctor.rs +// carried: "the credential failure names land and ci-wait, the tasks that break" crates/batten/src/doctor.rs +// carried: "an unreachable API is reported as a fork, never as a bad token" crates/batten/src/doctor.rs +// carried: "an unfenced proxy halts, and names the ambient NO_PROXY as the repair" crates/batten/src/doctor.rs +// changed: "both causes are reported together, not one at a time" crates/batten/src/doctor.rs from `tests/container-preflight.bats`. The property survives and its MECHANISM is the opposite one. The shell collected causes into an array and printed the list, which is why it could not use `set -e`; two handlers reach the same outcome because the door dispatches every row and reports every refusal in one reply, so neither failure can hide the other and neither needs a script to remember to keep looking +// withdrawn: "--degraded skips the GitHub probes — a missing gh is not a missing permission" crates/batten/src/doctor.rs from `tests/container-preflight.bats`. The arm it tested is already gone and `batten.toml` records why: handlers do not share state, so no row can be told that an earlier one failed. Recovering it needs a fact the door does not carry today. Keeping the case would be testing an argument nothing passes +// changed: "it halts and never repairs — the opposite of doctor" crates/batten/src/doctor.rs from `tests/container-preflight.bats`. The property is now structural rather than asserted: `doctor egress` is `Effect::Read` and `Egress` has no repair arm at all, where the shell had to be trusted not to grow one. `a_proxy_with_no_github_fence_is_unfenced` shows the refusal without a write +// withdrawn: "it tells the reader not to rewrite the repo around a broken container" crates/batten/src/doctor.rs from `tests/container-preflight.bats`. The case asserted a paragraph of advisory prose in the script's output, and non-negotiable rule 4 is why it cannot be carried: the successor emits one pointer line and a reason id. The advice it carried is true and its home is the issue, not a gate's stdout // Panicking on setup failure is the idiomatic way for a test to fail loudly. #![allow(clippy::unwrap_used, clippy::expect_used)] diff --git a/crates/batten/tests/it/exec_lock.rs b/crates/batten/tests/it/exec_lock.rs new file mode 100644 index 000000000..79241f127 --- /dev/null +++ b/crates/batten/tests/it/exec_lock.rs @@ -0,0 +1,222 @@ +//! `batten exec --lock` holds a named singleton lock across a child, over the +//! compiled binary (CLOUD-1710, ported from `mise-tasks/with-lock.sh`). +//! +//! **The two cases that discriminate are the last two**, and they are why this +//! tier exists rather than a pair of acquire/release assertions. A lock that +//! never blocks, never releases, or swallows the verdict of what it guards all +//! look identical from the outside on a quiet machine — the retiring suite says +//! so in its own header — and so does a lock that reclaims a holder it should +//! not. `an_empty_holder_file_is_held_not_free` and +//! `a_dead_holder_is_reclaimed_rather_than_waited_out` are the pair a naive +//! successor fails, and each has a `#MUTANT` row in `crates/batten/src/exec.rs` +//! that makes the discrimination an exit code rather than a claim. +//! +//! **Over the compiled binary rather than a unit test on `task::singleton_queue` +//! (CLOUD-418).** The queue is testable in isolation and that is not the +//! obligation: what retired is a WRAPPER, so the property is that the lock is +//! taken before the child and dropped after it whatever the child did, and only +//! the binary runs both halves. +//! +//! **The retirement this was built for is NOT in this delta, and the reason is a +//! measured coupling rather than a deferral.** `batten exec` resolves a +//! repository root for its capture store, and `mise-tasks/with-lock.sh` needed +//! none — so a caller running outside a real clone works under the shell and +//! refuses under the successor. `tests/doctor-race.bats` is exactly that caller: +//! its fixture is a `.git` DIRECTORY rather than a repository, so every port of +//! `with-lock` onto an `exec` flag reddens it, and that suite is governed and +//! cannot be edited. Its declared subject is `mise-tasks/doctor.sh`, which +//! retires under CLOUD-1753 — so `with-lock` retires in the same delta as the +//! `doctor` / `target-ensure` / `doctor-check` closed set, or not at all. The +//! capability lands here; the ledger arms land with the deletion. + +use std::path::Path; + +use crate::common; + +/// Where the engine keeps one clone's singleton locks. +fn lock_dir(repo: &Path, key: &str) -> std::path::PathBuf { + repo.join(".git").join("batten-singleton").join(key) +} + +/// A pid that is certainly not running. +/// +/// Reaped rather than invented: a number picked out of the air can collide with +/// a live process and turn a reclaim case green for the wrong reason. +fn dead_pid() -> u32 { + #[expect( + clippy::disallowed_types, + reason = "stays: reaping a real process is the only way to name a pid that is certainly dead, and the reclaim case turns on that — a number picked out of the air can collide with a live process and go green for the wrong reason" + )] + let child = std::process::Command::new("true") + .spawn() + .expect("spawn a process that exits immediately"); + let pid = child.id(); + let mut child = child; + child.wait().expect("reap it"); + pid +} + +fn repo(name: &str) -> std::path::PathBuf { + let dir = common::scratch(name); + common::init_repo(&dir); + dir +} + +#[test] +fn the_wrapped_exit_code_survives_the_lock() { + // The whole product of a wrapper. The shell names losing it as the defect + // that would "destroy the verdict of everything it guards". + let dir = repo("exec-lock-verdict"); + let output = common::run(&dir, &["exec", "--lock", "k", "--", "bash", "-c", "exit 7"]); + assert_eq!(output.status.code(), Some(7), "{}", common::stderr(&output)); +} + +#[test] +fn a_signal_survives_the_lock() { + // A child that died on a signal has no exit status of its own, and the + // shell's `128 + signal` convention is what the wrapper reports. The lock + // must not replace it with a success of its own. + let dir = repo("exec-lock-signal"); + let output = common::run( + &dir, + &["exec", "--lock", "k", "--", "bash", "-c", "kill -TERM $$"], + ); + assert_eq!( + output.status.code(), + Some(143), + "{}", + common::stderr(&output) + ); +} + +#[test] +fn the_lock_is_released_when_the_child_fails() { + // The half that matters: a failure leaving the lock held wedges every later + // caller for the whole queue, turning one red run into a stuck repo. + let dir = repo("exec-lock-release-on-failure"); + let output = common::run(&dir, &["exec", "--lock", "k", "--", "false"]); + assert_eq!(output.status.code(), Some(1)); + assert!( + !lock_dir(&dir, "k").exists(), + "the lock survived a failing child" + ); +} + +#[test] +fn an_empty_holder_file_is_held_not_free() { + // DISCRIMINATING. A holder caught between its create and its write has an + // empty pid file, and absence of evidence is "held", never "free" — + // reclaiming there robs a live process of a lock it is about to stamp. + // `--lock-attempts 1` is what makes this one ask rather than the full queue. + let dir = repo("exec-lock-empty-holder"); + let lock = lock_dir(&dir, "k"); + std::fs::create_dir_all(&lock).expect("stage a holder mid-write"); + std::fs::write(lock.join("pid"), "").expect("an empty pid file"); + + let output = common::run( + &dir, + &["exec", "--lock", "k", "--lock-attempts", "1", "--", "true"], + ); + assert_eq!( + output.status.code(), + Some(2), + "an empty holder file was read as free: {}", + common::stderr(&output) + ); +} + +#[test] +fn a_dead_holder_is_reclaimed_rather_than_waited_out() { + // DISCRIMINATING, and the opposite direction from the case above. A + // directory lock's release comes from the guard, which a SIGKILLed holder + // never runs; reclaim is what keeps that a delay of one ask instead of the + // whole queue. A refusal here means it regressed. + let dir = repo("exec-lock-dead-holder"); + let lock = lock_dir(&dir, "k"); + std::fs::create_dir_all(&lock).expect("stage an abandoned lock"); + std::fs::write(lock.join("pid"), format!("{}\n", dead_pid())).expect("a corpse"); + + let output = common::run( + &dir, + &["exec", "--lock", "k", "--lock-attempts", "2", "--", "true"], + ); + assert_eq!( + output.status.code(), + Some(0), + "a dead holder was waited out rather than reclaimed: {}", + common::stderr(&output) + ); +} + +#[test] +fn a_live_holder_is_refused_and_the_caller_names_the_wait() { + // Two properties in one case because they share a setup and neither is + // meaningful without the other: the refusal happens, and it points at the + // concept rather than at a directory. "the toolchain lock (some-triple)" is + // a pointer a reader can reason about; a bare key is a pointer to a path. + let dir = repo("exec-lock-live-holder"); + let lock = lock_dir(&dir, "k"); + std::fs::create_dir_all(&lock).expect("stage a live holder"); + std::fs::write(lock.join("pid"), format!("{}\n", std::process::id())).expect("a live pid"); + + let output = common::run( + &dir, + &[ + "exec", + "--lock", + "k", + "--lock-attempts", + "1", + "--lock-label", + "the toolchain lock (some-triple)", + "--", + "true", + ], + ); + assert_eq!(output.status.code(), Some(2)); + assert!( + common::stderr(&output).contains("the toolchain lock (some-triple)"), + "the refusal must name what the wait was for: {}", + common::stderr(&output) + ); +} + +#[test] +fn an_unlocked_run_is_untouched() { + // Anti-vacuity: every case above asserts something about `--lock`, and a + // build in which the flag did nothing at all would still pass several of + // them. This pins that the lock is opt-in and takes nothing when unasked. + let dir = repo("exec-lock-absent"); + let output = common::run(&dir, &["exec", "--", "true"]); + assert_eq!(output.status.code(), Some(0), "{}", common::stderr(&output)); + assert!( + !dir.join(".git").join("batten-singleton").exists(), + "an unlocked run took a lock" + ); +} + +#[test] +fn a_malformed_attempt_count_is_named_rather_than_defaulted() { + // `--jobs`' reading, and for `--jobs`' reason: reading a bad value as the + // default would queue for a length nobody asked for, and the refusal has to + // say which value was wrong. + let dir = repo("exec-lock-bad-attempts"); + let output = common::run( + &dir, + &[ + "exec", + "--lock", + "k", + "--lock-attempts", + "nought", + "--", + "true", + ], + ); + assert_eq!(output.status.code(), Some(1), "{}", common::stderr(&output)); + assert!( + common::stderr(&output).contains("nought"), + "the refusal must name the value: {}", + common::stderr(&output) + ); +} diff --git a/crates/batten/tests/it/glob_containment.rs b/crates/batten/tests/it/glob_containment.rs new file mode 100644 index 000000000..5af893ff2 --- /dev/null +++ b/crates/batten/tests/it/glob_containment.rs @@ -0,0 +1,366 @@ +//! The hook step's trigger covers every path the config makes an input, over the +//! compiled binary (CLOUD-224, ported from `mise-tasks/batten-glob-check.sh` +//! under CLOUD-843). +//! +//! **What is decidable only here.** `policy/glob-containment.rego` carries +//! load-time cases pinning the predicate, and every one of them supplies +//! `input.tree.lines` with `with input as`. That fabricates the very shape the +//! engine may be unable to produce (CLOUD-845): whether a declared `line_sources` +//! path resolves to the COMMITTED bytes at all, and whether two separate paths +//! both resolve in one evaluation, is the engine's half and no `with input as` +//! block can test it. The retiring suite took its two files as positional +//! arguments so it could be pointed at fixtures; the successor's fixtures are +//! whole repositories, which is a stronger tier and not a weaker one. +//! +//! The self-consumption case is the one the retiring suite opened on: this +//! repository's own committed pair covers itself, so the containment claim is +//! checkable rather than asserted. +// +// carried: mise-tasks/batten-glob-check.sh policy/glob-containment.rego crates/batten/tests/it/glob_containment.rs +// carried: tests/batten-glob-check.bats policy/glob-containment.rego crates/batten/tests/it/glob_containment.rs +// +// carried: "the committed pair covers itself today" policy/glob-containment.rego +// carried: "a rule glob absent from the list is caught, and named" policy/glob-containment.rego +// carried: "a verbatim entry covers a rule glob" policy/glob-containment.rego +// carried: "a P/** entry subsumes anything under P — the reason the list stays short" policy/glob-containment.rego +// carried: "subsumption is a prefix test, so a sibling prefix does not count" policy/glob-containment.rego +// carried: "a budget file is an input, and an uncovered one is caught" policy/glob-containment.rego +// carried: "an embedded budget path is an input too" policy/glob-containment.rego +// carried: "a shape rule declares no glob and demands nothing" policy/glob-containment.rego +// carried: "a comment inside the list is not list syntax, parenthesis and all" policy/glob-containment.rego +// carried: "another step's glob list is not read as batten-check's" policy/glob-containment.rego +// carried: "output is a pointer — no file contents echoed" policy/glob-containment.rego +// changed: "a config the gate parses nothing out of is exit 2, not a pass" policy/glob-containment.rego the refusal is carried whole and its EXIT CODE is the engine's rather than the shell's: the shell reserved 2 for could-not-look and used 1 for a violation, and the engine's one contract makes every deny finding exit 2 with no per-verb exception (AGENTS.md rule 5). So the case still separates a failed parse from a pass, which is what it was for, and no longer separates it from a violation by exit code — it separates it by verdict, which is the pointer a reader acts on +// changed: "a batten-check step with no glob at all is a regression, not a default" policy/glob-containment.rego same exit-code change as the row above: the shell exited 1 here and the successor emits a `step select missing` finding, which is exit 2 on the engine's contract +// changed: "a missing input file is exit 2, distinct from a violation" policy/glob-containment.rego the clause has no successor to carry because the ENGINE decides it earlier: a rule whose declared `line_sources` match nothing is not evaluated at all, and `input.tree.missing` is never populated on the tree surface (CLOUD-1049, measured identically for `policy/mise-pin-agreement.rego`'s own could-not-look clause). A case asserting it would assert the engine gap rather than the predicate, so it ships without one until that fact does + +// Panicking on setup failure is the idiomatic way for a test to fail loudly. +#![allow(clippy::unwrap_used, clippy::expect_used)] + +use crate::common; + +use std::fmt::Write as _; +use std::path::{Path, PathBuf}; +use std::process::Output; + +use common::{Fixture, git_in, run, stderr, stdout}; + +/// A repository declaring only this rule, so any finding is the one under test. +/// +/// The three `[[verdict]]` rows are carried into the fixture rather than +/// assumed: a verdict is emittable only where a row declares it, and the +/// built-in registry is not this consumer's vocabulary. Without them the module +/// loads and the run is a USAGE error, which is exit 1 and not the exit 2 these +/// cases are about — so a fixture that omitted them would test the declaration +/// rather than the predicate. +fn glob_repo(name: &str, config_body: &str, hooks_body: &str) -> PathBuf { + let config = format!( + "version = 1\n\n\ + [[verdict]]\n\ + id = \"step cover missing\"\n\ + gloss = \"the manifest does not select a path it must judge\"\n\ + class = \"A trigger narrower than its rule set deletes feedback while preserving the verdict.\"\n\n\ + [[verdict.route]]\n\ + id = \"prose read first\"\n\ + kind = \"document\"\n\ + target = \"AGENTS.md\"\n\n\ + [[verdict]]\n\ + id = \"step select missing\"\n\ + gloss = \"a step declares no trigger\"\n\ + class = \"A step with no glob runs on every commit, which is what the trigger removed.\"\n\n\ + [[verdict.route]]\n\ + id = \"prose read first\"\n\ + kind = \"document\"\n\ + target = \"AGENTS.md\"\n\n\ + [[verdict]]\n\ + id = \"config parse unread\"\n\ + gloss = \"the gate parsed nothing out of its own input\"\n\ + class = \"A containment check that parses zero requirements passes vacuously.\"\n\n\ + [[verdict.route]]\n\ + id = \"prose read first\"\n\ + kind = \"document\"\n\ + target = \"AGENTS.md\"\n\n\ + [[pattern]]\n\ + id = \"md-quoted-span\"\n\ + regex = '\"[^\"]*\"'\n\n\ + [[rule]]\n\ + id = \"glob-containment\"\n\ + kind = \"policy\"\n\ + scope = \"tree\"\n\ + line_sources = [\"batten.toml\", \"hk.pkl\"]\n\ + module = \"policy/glob-containment.rego\"\n\ + severity = \"deny\"\n\n\ + {config_body}" + ); + let dir = Fixture::new(name) + .config(&config) + .file("AGENTS.md", "the consumer's own authority\n") + .file("hk.pkl", hooks_body) + .git() + .build(); + // The module is copied in rather than referenced: the fixture is its own + // repository, and a rule row naming a path outside it would not resolve. + common::write( + &dir, + "policy/glob-containment.rego", + &std::fs::read_to_string(common::at_root("policy/glob-containment.rego")).unwrap(), + ); + git_in(&dir, &["add", "-A"]); + git_in(&dir, &["commit", "-q", "-m", "base"]); + dir +} + +fn check(dir: &Path) -> Output { + run(dir, &["check", "--rule", "glob-containment"]) +} + +/// A `["batten-check"]` step selecting `entries`, followed by another step so +/// the reader's upper bound is a real one. +fn hooks(entries: &[&str]) -> String { + let mut listed = String::new(); + for entry in entries { + writeln!(listed, " \"{entry}\",").unwrap(); + } + format!( + "hooks {{\n [\"batten-check\"] {{\n glob =\n List(\n{listed} )\n check = \"mise run batten-check\"\n }}\n [\"other-step\"] {{\n glob = List(\"unrelated\")\n }}\n}}\n" + ) +} + +fn rule_glob(glob: &str) -> String { + format!( + "[[rule]]\nid = \"r\"\nkind = \"forbid\"\nscope = \"tree\"\nglob = \"{glob}\"\npattern = \"x\"\nseverity = \"deny\"\n" + ) +} + +#[test] +fn a_verbatim_entry_covers_a_rule_glob() { + let dir = glob_repo( + "glob-verbatim", + &rule_glob("mise.toml"), + &hooks(&["batten.toml", "hk.pkl", "policy/**", "mise.toml"]), + ); + let output = check(&dir); + assert_eq!( + output.status.code(), + Some(0), + "out={} err={}", + stdout(&output), + stderr(&output) + ); +} + +#[test] +fn a_prefix_entry_subsumes_anything_under_it() { + // The reason the list can stay short: one `crates/**` stands for every glob + // beneath it. + let dir = glob_repo( + "glob-prefix", + &rule_glob("crates/batten/tests/**/*.rs"), + &hooks(&["batten.toml", "hk.pkl", "policy/**", "crates/**"]), + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(0), "{}", stdout(&output)); +} + +#[test] +fn an_unlisted_glob_is_refused_and_named() { + let dir = glob_repo( + "glob-unlisted", + &rule_glob("mise.toml"), + &hooks(&["batten.toml", "hk.pkl", "policy/**"]), + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); + assert!( + stdout(&output).contains("batten.toml:"), + "the finding points at the config line that demands it: {:?}", + stdout(&output) + ); +} + +#[test] +fn a_sibling_prefix_does_not_count() { + // `crates-extra/` is not under `crates/`. A looser string match would call + // this covered, which is the direction a containment check must never fail + // in. + let dir = glob_repo( + "glob-sibling-prefix", + &rule_glob("crates-extra/**/*.rs"), + &hooks(&["batten.toml", "hk.pkl", "policy/**", "crates/**"]), + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); +} + +#[test] +fn a_slashless_entry_does_not_subsume_by_prefix() { + // The `/**` is what makes an entry a prefix at all. An entry without it + // subsumes nothing, however much of it a required glob happens to start + // with — otherwise a bare `crates` would silently stand for + // `crates-extra/**`, which is a different tree. + let dir = glob_repo( + "glob-slashless", + &rule_glob("crates-extra/**/*.rs"), + &hooks(&["batten.toml", "hk.pkl", "policy/**", "crates"]), + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); +} + +#[test] +fn a_budget_file_is_an_input() { + // The case the issue's own wording missed: a declared budget is a gate under + // `check`, not only under its own verb, so a budgeted document is as much an + // input as any rule glob. + let dir = glob_repo( + "glob-budget-files", + &format!( + "{}\n[budget.instructions]\nfiles = [\"AGENTS.md\", \"CONTRIBUTING.md\"]\nmax_tokens = 10\n", + rule_glob("mise.toml") + ), + &hooks(&[ + "batten.toml", + "hk.pkl", + "policy/**", + "mise.toml", + "AGENTS.md", + ]), + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); +} + +#[test] +fn an_embedded_budget_path_is_an_input_too() { + let dir = glob_repo( + "glob-budget-embedded", + &format!( + "{}\n[budget.instructions]\nfiles = [\"AGENTS.md\"]\nmax_tokens = 10\n\n[[budget.instructions.embedded]]\npath = \".serena/project.yml\"\nkey = \"initial_prompt\"\n", + rule_glob("mise.toml") + ), + &hooks(&[ + "batten.toml", + "hk.pkl", + "policy/**", + "mise.toml", + "AGENTS.md", + ]), + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); + assert!( + stdout(&output).contains("batten.toml:"), + "the finding points at the config line that demands it: {:?}", + stdout(&output) + ); +} + +#[test] +fn a_rule_with_no_glob_demands_nothing() { + let dir = glob_repo( + "glob-no-glob-rule", + &format!( + "[[rule]]\nid = \"s\"\nkind = \"shape\"\nscope = \"mediated_call\"\nseverity = \"deny\"\npattern = \"gh pr merge\"\nreason = \"no\"\n\n{}", + rule_glob("mise.toml") + ), + &hooks(&["batten.toml", "hk.pkl", "policy/**", "mise.toml"]), + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(0), "{}", stdout(&output)); +} + +#[test] +fn a_comment_inside_the_list_is_not_list_syntax() { + // The failure the retired gate actually shipped: an entry's comment + // contained a parenthesised tracker key whose `)` ended the list early, so + // every entry below it read as uncovered and the gate reported four paths + // that were listed all along. + let dir = glob_repo( + "glob-comment-in-list", + &rule_glob("mise.toml"), + "hooks {\n [\"batten-check\"] {\n glob =\n List(\n \"batten.toml\",\n \"hk.pkl\",\n // the rule that made this an input (CLOUD-614)\n \"policy/**\",\n \"mise.toml\",\n )\n }\n [\"other-step\"] {\n glob = List(\"unrelated\")\n }\n}\n", + ); + let output = check(&dir); + assert_eq!( + output.status.code(), + Some(0), + "a comment closed the list: {}", + stdout(&output) + ); +} + +#[test] +fn another_steps_list_is_not_read_as_this_ones() { + let dir = glob_repo( + "glob-other-step", + &rule_glob("mise.toml"), + "hooks {\n [\"batten-check\"] {\n glob = List(\"batten.toml\", \"hk.pkl\", \"policy/**\")\n }\n [\"other-step\"] {\n glob = List(\"mise.toml\")\n }\n}\n", + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); +} + +#[test] +fn a_step_with_no_glob_at_all_is_a_regression_not_a_default() { + let dir = glob_repo( + "glob-step-without-list", + &rule_glob("mise.toml"), + "hooks {\n [\"batten-check\"] {\n check = \"mise run batten-check\"\n }\n [\"other-step\"] {\n glob = List(\"unrelated\")\n }\n}\n", + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); + assert!( + stdout(&output).contains("hk.pkl"), + "the finding points at the manifest: {:?}", + stdout(&output) + ); +} + +#[test] +fn a_config_yielding_no_inputs_is_refused_rather_than_passed() { + // The vacuous green a containment check produces most easily: parse zero + // requirements and every list covers them. + let dir = glob_repo( + "glob-unparseable-config", + "# this config declares no rule glob and no budget path\n", + &hooks(&["batten.toml", "hk.pkl", "policy/**"]), + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); +} + +#[test] +fn output_is_pointer_only() { + let dir = glob_repo( + "glob-pointer-only", + &format!( + "{}\n# a distinctive phrase nobody should see in a finding\n", + rule_glob("mise.toml") + ), + &hooks(&["batten.toml", "hk.pkl", "policy/**"]), + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2)); + assert!( + !stdout(&output).contains("distinctive phrase"), + "the config's prose is payload: {:?}", + stdout(&output) + ); +} + +#[test] +fn the_committed_pair_covers_itself_today() { + // The self-consumption case the retiring suite opened on, and the reason + // this gate is worth having: the two committed files agree, checked rather + // than asserted. + let output = common::run_at_real_root( + &common::at_root(""), + &["check", "--rule", "glob-containment"], + ); + assert_eq!( + output.status.code(), + Some(0), + "this repository's own batten-check glob does not cover every input: {}", + stdout(&output) + ); +} diff --git a/crates/batten/tests/it/hook_pin_check.rs b/crates/batten/tests/it/hook_pin_check.rs new file mode 100644 index 000000000..c50446164 --- /dev/null +++ b/crates/batten/tests/it/hook_pin_check.rs @@ -0,0 +1,297 @@ +//! No hook registered BY PATH shells out to a pinned tool, over the compiled +//! binary (CLOUD-479, ported from `mise-tasks/hook-pin-check.sh` under +//! CLOUD-843). +//! +//! **What is decidable only here.** `policy/hook-pin-check.rego` carries +//! load-time cases pinning the predicate, and every one of them hands the module +//! a ready-made map of three different kinds of file with `with input as`. That +//! fabricates the very shape the engine may be unable to produce (CLOUD-845), and +//! here it fabricates the whole judgement: this rule is a JOIN across three +//! `line_sources` shapes — two literal paths and a glob over the task directory — +//! resolved in one evaluation. A module whose suite only fabricated that map +//! would stay green over an engine that resolved a literal path but not the glob, +//! which is precisely the state in which every by-path hook passes vacuously. +//! +//! The self-consumption case is the one the retiring suite ended on: this +//! repository's own registrations are judged, so the pairing is a live claim. +// +// carried: mise-tasks/hook-pin-check.sh policy/hook-pin-check.rego crates/batten/tests/it/hook_pin_check.rs +// carried: tests/hook-pin-check.bats policy/hook-pin-check.rego crates/batten/tests/it/hook_pin_check.rs +// +// carried: "a by-path hook shelling out to a pinned tool is refused, and both are named" policy/hook-pin-check.rego +// carried: "THE SAME TASK VIA mise run IS FINE — the pairing is the defect, not the tool" policy/hook-pin-check.rego +// carried: "a by-path hook using no pinned tool passes" policy/hook-pin-check.rego +// carried: "a DECLARED exemption passes, because the script asserts the tool itself" policy/hook-pin-check.rego +// carried: "an exemption for a DIFFERENT tool does not cover this one" policy/hook-pin-check.rego +// carried: "MENTIONING a tool in a comment is not depending on it" policy/hook-pin-check.rego +// carried: "a tool named as a substring of another word is not a call" policy/hook-pin-check.rego +// carried: "the pinned set is READ from the manifest, not restated here" policy/hook-pin-check.rego +// carried: "a manifest with no [tools] is exit 2 — could not look, never a verdict" policy/hook-pin-check.rego +// carried: "output is pointer-only — the task and the tool, never a line of either file" policy/hook-pin-check.rego +// carried: "this repository's own registrations pass" policy/hook-pin-check.rego +// changed: "the refusal names all three ways out, since a deny with no exit is a wall" policy/hook-pin-check.rego the shell wrote its remedy into the refusal's own prose; on the engine a verdict's remedy is DATA — the `[[verdict.route]]` rows the registry resolves — so the three ways out are declared once beside the verdict rather than restated per firing, and `batten policy explain` is what reads them back. The case is carried as the route's existence rather than as a substring of the refusal +// changed: "no by-path registrations SAYS SO rather than reading as a clean pass" policy/hook-pin-check.rego the shell printed an advisory line to stderr and still exited 0, and the engine's output contract has no advisory channel for a rule that found nothing (AGENTS.md rule 5, one contract with no per-verb exception). The vacuity that actually mattered is kept as a real finding rather than a note: a manifest pinning NOTHING is `tool list empty`, which is what makes every by-path hook stop passing vacuously +// changed: "a missing settings file is exit 2, not a pass" policy/hook-pin-check.rego the ENGINE decides this earlier — a rule whose declared `line_sources` match nothing is not evaluated at all, and `input.tree.missing` is never populated on the tree surface (CLOUD-1049, measured identically for `policy/mise-pin-agreement.rego`'s own could-not-look clause). A case asserting it would assert the engine gap rather than the predicate + +// Panicking on setup failure is the idiomatic way for a test to fail loudly. +#![allow(clippy::unwrap_used, clippy::expect_used)] + +use crate::common; + +use std::fmt::Write as _; +use std::path::{Path, PathBuf}; +use std::process::Output; + +use common::{Fixture, git_in, run, stderr, stdout}; + +/// A repository declaring only this rule, so any finding is the one under test. +fn pin_repo( + name: &str, + registrations: &[&str], + manifest: &str, + scripts: &[(&str, &str)], +) -> PathBuf { + let mut fixture = Fixture::new(name).config( + "version = 1\n\n\ + [[pattern]]\n\ + id = \"md-quoted-span\"\n\ + regex = '\"[^\"]*\"'\n\n\ + [[verdict]]\n\ + id = \"tool reach absent\"\n\ + gloss = \"a by-path invocation depends on a tool it will not get\"\n\ + class = \"Every hook here fails open, so an absent tool allows silently instead of erroring.\"\n\n\ + [[verdict.route]]\n\ + id = \"prose read first\"\n\ + kind = \"document\"\n\ + target = \"AGENTS.md\"\n\n\ + [[verdict]]\n\ + id = \"tool list empty\"\n\ + gloss = \"the manifest pins nothing, so the question cannot be asked\"\n\ + class = \"With the pinned set empty every by-path hook passes vacuously.\"\n\n\ + [[verdict.route]]\n\ + id = \"prose read first\"\n\ + kind = \"document\"\n\ + target = \"AGENTS.md\"\n\n\ + [[rule]]\n\ + id = \"hook-pin-check\"\n\ + kind = \"policy\"\n\ + scope = \"tree\"\n\ + line_sources = [\".claude/settings.json\", \"mise.toml\", \"mise-tasks/*.sh\"]\n\ + module = \"policy/hook-pin-check.rego\"\n\ + severity = \"deny\"\n", + ); + let mut hooks = String::new(); + for command in registrations { + writeln!( + hooks, + " {{ \"type\": \"command\", \"command\": \"{command}\" }}," + ) + .unwrap(); + } + let settings = format!("{{\n \"hooks\": {{\n \"PreToolUse\": [\n{hooks} ]\n }}\n}}\n"); + fixture = fixture + .file("AGENTS.md", "the consumer's own authority\n") + .file(".claude/settings.json", &settings) + .file("mise.toml", manifest); + for (path, body) in scripts { + fixture = fixture.file(path, body); + } + let dir = fixture.git().build(); + common::write( + &dir, + "policy/hook-pin-check.rego", + &std::fs::read_to_string(common::at_root("policy/hook-pin-check.rego")).unwrap(), + ); + git_in(&dir, &["add", "-A"]); + git_in(&dir, &["commit", "-q", "-m", "base"]); + dir +} + +fn check(dir: &Path) -> Output { + run(dir, &["check", "--rule", "hook-pin-check"]) +} + +const MANIFEST: &str = + "[tools]\nzizmor = \"1.0\"\n\"aqua:jqlang/jq\" = \"1.7\"\n\n[env]\nX = \"1\"\n"; + +#[test] +fn a_by_path_hook_shelling_out_to_a_pinned_tool_is_refused() { + let dir = pin_repo( + "pin-by-path-pinned", + &["mise-tasks/guard.sh"], + MANIFEST, + &[( + "mise-tasks/guard.sh", + "#!/usr/bin/env bash\njq -r '.x' <<<\"$payload\"\n", + )], + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); + let text = stdout(&output); + assert!( + text.contains("guard.sh") && text.contains("jq"), + "the finding names the task and the tool: {text:?}" + ); +} + +#[test] +fn a_runner_registration_is_not_judged() { + // THE PAIRING IS THE DEFECT, not the tool: the same script through the task + // runner gets its env by construction. + let dir = pin_repo( + "pin-via-runner", + &["mise run -q mise-tasks/guard.sh"], + MANIFEST, + &[( + "mise-tasks/guard.sh", + "#!/usr/bin/env bash\njq -r '.x' <<<\"$payload\"\n", + )], + ); + let output = check(&dir); + assert_eq!( + output.status.code(), + Some(0), + "out={} err={}", + stdout(&output), + stderr(&output) + ); +} + +#[test] +fn a_by_path_hook_using_no_pinned_tool_passes() { + let dir = pin_repo( + "pin-clean", + &["mise-tasks/guard.sh"], + MANIFEST, + &[( + "mise-tasks/guard.sh", + "#!/usr/bin/env bash\nbatten hook claude-code\n", + )], + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(0), "{}", stdout(&output)); +} + +#[test] +fn a_declared_exemption_passes() { + let dir = pin_repo( + "pin-exempt", + &["mise-tasks/guard.sh"], + MANIFEST, + &[( + "mise-tasks/guard.sh", + "#!/usr/bin/env bash\n#PIN-OK: jq\ncommand -v jq >/dev/null || exit 2\njq -r '.x'\n", + )], + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(0), "{}", stdout(&output)); +} + +#[test] +fn an_exemption_for_another_tool_does_not_cover_this_one() { + let dir = pin_repo( + "pin-exempt-other", + &["mise-tasks/guard.sh"], + MANIFEST, + &[( + "mise-tasks/guard.sh", + "#!/usr/bin/env bash\n#PIN-OK: zizmor\njq -r '.x'\n", + )], + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); +} + +#[test] +fn mentioning_a_tool_in_a_comment_is_not_depending_on_it() { + let dir = pin_repo( + "pin-comment", + &["mise-tasks/guard.sh"], + MANIFEST, + &[( + "mise-tasks/guard.sh", + "#!/usr/bin/env bash\n# this used to call jq and no longer does\nbatten hook claude-code\n", + )], + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(0), "{}", stdout(&output)); +} + +#[test] +fn a_substring_of_another_word_is_not_a_call() { + let dir = pin_repo( + "pin-substring", + &["mise-tasks/guard.sh"], + MANIFEST, + &[( + "mise-tasks/guard.sh", + "#!/usr/bin/env bash\njqx --render file\ncat file | myjq -r '.x'\n", + )], + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(0), "{}", stdout(&output)); +} + +#[test] +fn the_pinned_set_is_read_from_the_manifest() { + // Pinning a tool enrols it here with no second edit — the case that keeps + // this from becoming a restated list. + let dir = pin_repo( + "pin-read-from-manifest", + &["mise-tasks/guard.sh"], + "[tools]\nshellcheck = \"0.10\"\n", + &[( + "mise-tasks/guard.sh", + "#!/usr/bin/env bash\nshellcheck -x \"$0\"\n", + )], + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); +} + +#[test] +fn a_manifest_pinning_nothing_is_not_clean() { + // COULD NOT LOOK, never a verdict: with the pinned set empty every by-path + // hook passes vacuously. + let dir = pin_repo( + "pin-empty-tools", + &["mise-tasks/guard.sh"], + "[env]\nX = \"1\"\n", + &[("mise-tasks/guard.sh", "#!/usr/bin/env bash\njq -r '.x'\n")], + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); +} + +#[test] +fn output_is_pointer_only() { + let dir = pin_repo( + "pin-pointer-only", + &["mise-tasks/guard.sh"], + MANIFEST, + &[( + "mise-tasks/guard.sh", + "#!/usr/bin/env bash\njq -r '.distinctive_selector' <<<\"$payload\"\n", + )], + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2)); + assert!( + !stdout(&output).contains("distinctive_selector"), + "the script's content is payload: {:?}", + stdout(&output) + ); +} + +#[test] +fn this_repositorys_own_registrations_pass() { + // The self-consumption case the retiring suite ended on. + let output = + common::run_at_real_root(&common::at_root(""), &["check", "--rule", "hook-pin-check"]); + assert_eq!( + output.status.code(), + Some(0), + "a by-path hook here depends on a pinned tool: {}", + stdout(&output) + ); +} diff --git a/crates/batten/tests/it/land.rs b/crates/batten/tests/it/land.rs index aad930787..fed9c663b 100644 --- a/crates/batten/tests/it/land.rs +++ b/crates/batten/tests/it/land.rs @@ -135,6 +135,66 @@ fn a_conflicted_lap_is_refused_and_a_clean_one_is_not() { ); } +/// **The property the whole family rests on, driven end to end** (CLOUD-1708). +/// +/// Every other case here hands a constructed [`Replay`] to `land::record`, which +/// pins the WRITER, and `tests/it/rebase.rs` reaches a real conflict but never +/// reads the store. So the one thing `rebase-conflict-stops-the-lap` actually +/// depends on — that a REAL conflict leaves a `rebase conflicted …` line the +/// module then refuses on — was unpinned end to end, and a regression on the +/// record call inside `land::replay` would have been invisible. +/// +/// The row that filed this measured `land::record` as having zero production +/// callers and concluded the store was dead. It is not: `land::replay` records +/// on all three arms and has since the feature landed. The measurement counted +/// QUALIFIED `land::record` references, and the production call is unqualified +/// from inside `land.rs` itself — so a name-resolution question answered with a +/// string scan saw only the test callers, every one of which spells it +/// `land::record`. What was missing was never the writer; it was this case. +#[test] +fn a_real_conflict_writes_the_record_the_module_refuses_on() { + let repo = repo("land-real-conflict"); + let branch = branch_of(&repo); + + // Two sides edit one path, which is the shape a replay cannot resolve. + common::write(&repo, "shared.txt", "base\n"); + common::git_in(&repo, &["add", "-A"]); + common::git_in(&repo, &["commit", "-q", "-m", "shared"]); + common::git_in(&repo, &["branch", "-f", "trunk"]); + + common::write(&repo, "shared.txt", "the branch's line\n"); + common::git_in(&repo, &["add", "-A"]); + common::git_in(&repo, &["commit", "-q", "-m", "branch side"]); + + common::git_in(&repo, &["checkout", "-q", "trunk"]); + common::write(&repo, "shared.txt", "the trunk's line\n"); + common::git_in(&repo, &["add", "-A"]); + common::git_in(&repo, &["commit", "-q", "-m", "trunk side"]); + common::git_in(&repo, &["checkout", "-q", &branch]); + + // `replay_onto` rather than `replay`: the only difference is the FETCH, which + // speaks the forge's HTTP protocol and would need a server rather than a + // repository. The fetch is not what this case is about — the rebase, the + // mapping and the record are, and they are one function. + let outcome = land::replay_onto(&repo, "refs/heads/trunk", &branch, &[]) + .expect("the replay itself must run"); + let Replay::Conflicted { .. } = outcome else { + panic!("two sides editing one path must conflict, got {outcome:?}"); + }; + + // AND THE STORE CARRIES IT. This is the half nothing asserted: the writer is + // reached by the real path, not only by a test handing `record` a value. + let (code, out, err) = check(&repo); + assert_eq!( + code, 2, + "a real conflict must reach the module through the record: {err}{out}" + ); + assert!( + format!("{out}{err}").contains("rebase-conflict-stops-the-lap"), + "the finding names its own predicate, got {out}{err}" + ); +} + /// A branch that has recorded no lap at all is not refused. /// /// The state a fresh clone is in, and the one a gate keyed on the record's @@ -496,3 +556,72 @@ fn no_pull_request_to_ask_is_could_not_look_and_never_a_refusal() { "exit 2 would claim the bot refused this head, which nothing established" ); } + +// --------------------------------------------------------------------------- +// The stop names a route that exists (CLOUD-1586's mechanism, CLOUD-1050's rule) +// --------------------------------------------------------------------------- + +/// The conflict refusal and the gate that denies the hand rebase agree, and both +/// name a route this engine actually has. +/// +/// **Measured on this branch rather than imagined.** The verb's conflict line +/// reported the commit and the FIRST path and stopped there; the +/// `rebase-not-hand-stepped` row then told the reader the way out was +/// `--continue`, `--abort` or `--skip`. The replay is STATELESS — nothing is +/// half-replayed, so no rebase is ever in progress — and none of those three can +/// apply to it. A session followed both, concluded the landing loop was +/// defective, and was one step from cherry-picking around it, which completes +/// the replay while writing no lap record at all: `rebase-conflict-stops-the-lap` +/// would then read clean over a conflict that happened, which is the false green +/// this whole family exists to catch. +/// +/// Two authorities for one route is the drift, so this asserts they AGREE rather +/// than checking either alone. Text over the committed files, because that is +/// what a reader meets: driving the verb to a conflict needs a live remote, and +/// `crates/batten/tests/it/rebase.rs` already owns the mechanism end to end. +#[test] +fn the_conflict_stop_and_its_gate_name_a_route_that_exists() { + let verb = std::fs::read_to_string(common::at_root("crates/batten/src/lib.rs")) + .expect("the boundary is readable"); + let after = verb + .split_once("land::Replay::Conflicted") + .expect("the conflict arm is where the message lives") + .1; + let arm = &after[..after.find("land::Replay::Current").unwrap_or(after.len())]; + + assert!( + arm.contains("--resolve"), + "the conflict refusal must name the route it has, or a reader reaches for one it does not" + ); + assert!( + arm.contains("="), + "and the per-conflict spelling, because a path conflicting twice is the common case and is \ + unguessable from a bare --resolve" + ); + // NAMING `--continue` IS RIGHT HERE, and the first draft of this case + // asserted its absence — which would have refused the very sentence that + // stops a reader reaching for it. What matters is that the text says WHY it + // cannot apply, so the reader stops looking rather than concluding the loop + // is broken. + assert!( + arm.contains("STATELESS"), + "the refusal must say why there is nothing to continue, or its absence reads as a defect" + ); + + let config = std::fs::read_to_string(common::at_root("batten.toml")) + .expect("the committed authority is readable"); + let row = config + .split_once("id = \"rebase-not-hand-stepped\"") + .expect("the row is declared") + .1; + let reason = &row[..row.find("\n\n").unwrap_or(row.len())]; + + assert!( + reason.contains("--resolve"), + "the gate that denies the hand rebase must name the route that replaces it" + ); + assert!( + reason.contains("STATELESS"), + "and must say why the rebase-in-progress exits cannot apply, rather than offering them" + ); +} diff --git a/crates/batten/tests/it/license_table.rs b/crates/batten/tests/it/license_table.rs new file mode 100644 index 000000000..44e418ed4 --- /dev/null +++ b/crates/batten/tests/it/license_table.rs @@ -0,0 +1,157 @@ +//! Every adopted tool's license row is resolved, over the compiled binary +//! (ported from `mise-tasks/license-table-check.sh` under CLOUD-843). +//! +//! **What is decidable only here.** `policy/license-table.rego` carries +//! load-time cases pinning the parse and the closed set, and each fabricates +//! `input.tree.lines["CONTRIBUTING.md"]`. Only a real repository shows the +//! engine resolving that declaration — which is the anti-vacuity half twice +//! over, since the gate's own subject is "did the table parse to any rows at +//! all". +// +// carried: mise-tasks/license-table-check.sh policy/license-table.rego crates/batten/tests/it/license_table.rs +// carried: tests/license-table-check.bats policy/license-table.rego crates/batten/tests/it/license_table.rs +// +// carried: "license-table-check.bats::the repo as it stands passes" policy/license-table.rego +// carried: "an unresolved license fails, and names the tool" policy/license-table.rego +// carried: "a resolved license with an unresolved verdict still fails" policy/license-table.rego +// carried: "a verdict outside the closed set fails rather than passing" policy/license-table.rego +// carried: "a fully resolved fixture passes" policy/license-table.rego +// carried: "an explicit incompatible verdict is resolved, and passes" policy/license-table.rego +// carried: "a table with no rows is a failure, not a vacuous pass" policy/license-table.rego +// carried: "output is a pointer — it names the tool and the cell, never the table body" policy/license-table.rego +// +// changed: "license-table-check.bats::the gate is wired: hk.pkl declares a step that runs this task" policy/license-table.rego the case asserted that a `mise run` step existed in `hk.pkl`, which is how a shell gate reached the hook at all. A rule row has no step of its own — it is reached through `batten-check`, whose glob is itself gated by `batten-glob-check` and whose `line_sources` declaration is what `batten check --rule license-table` resolves. The wiring is asserted by a different mechanism rather than left unasserted +// changed: "an unreadable file is exit 1 — could not look is not a verdict" policy/license-table.rego the shell took the document as a positional ARGUMENT and could be pointed at an unreadable path. The successor's subject is a declared `line_sources` entry: a glob matching nothing means the rule is not evaluated, and `input.tree.missing` is never populated on the tree surface (CLOUD-1049). There is no caller left that can aim it, so the case has no subject rather than no coverage + +// Panicking on setup failure is the idiomatic way for a test to fail loudly. +#![allow(clippy::unwrap_used, clippy::expect_used)] + +use crate::common; + +use std::path::{Path, PathBuf}; +use std::process::Output; + +use common::{Fixture, git_in, run, stdout}; + +fn table_repo(name: &str, doc: &str) -> PathBuf { + let dir = Fixture::new(name) + .config( + "version = 1\n\n\ + [[verdict]]\n\ + id = \"tool grade unclear\"\n\ + gloss = \"an adopted tool's declaration is unresolved\"\n\ + class = \"A row still asking the question cannot be shipped against; read the upstream LICENSE and record the SPDX id.\"\n\n\ + [[verdict.route]]\n\ + id = \"prose read first\"\n\ + kind = \"document\"\n\ + target = \"AGENTS.md\"\n\n\ + [[rule]]\n\ + id = \"license-table\"\n\ + kind = \"policy\"\n\ + scope = \"tree\"\n\ + line_sources = [\"CONTRIBUTING.md\"]\n\ + module = \"policy/license-table.rego\"\n\ + severity = \"deny\"\n", + ) + .file("AGENTS.md", "the consumer's own authority\n") + .file("CONTRIBUTING.md", doc) + .git() + .build(); + common::write( + &dir, + "policy/license-table.rego", + &std::fs::read_to_string(common::at_root("policy/license-table.rego")).unwrap(), + ); + git_in(&dir, &["add", "-A"]); + git_in(&dir, &["commit", "-q", "-m", "base"]); + dir +} + +fn check(dir: &Path) -> Output { + run(dir, &["check", "--rule", "license-table"]) +} + +const HEAD: &str = "| Tool | Use | License | Apache-2.0 |\n| --- | --- | --- | --- |\n"; + +#[test] +fn a_fully_resolved_table_passes() { + let dir = table_repo( + "license-resolved", + &format!("{HEAD}| hk | hooks | MIT | ✅ |\n"), + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(0), "{}", stdout(&output)); +} + +#[test] +fn an_unresolved_license_fails_and_names_the_tool() { + let dir = table_repo( + "license-unresolved", + &format!("{HEAD}| hk | hooks | _to confirm_ | ✅ |\n"), + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); + assert!( + stdout(&output).contains("CONTRIBUTING.md"), + "the finding points at the table: {}", + stdout(&output) + ); +} + +#[test] +fn a_verdict_outside_the_closed_set_fails_over_the_binary() { + // DISCRIMINATING. A check that only looked for the literal placeholder + // passes this, which is exactly how an unresolved row slips through. + let dir = table_repo( + "license-open-set", + &format!("{HEAD}| hk | hooks | MIT | probably |\n"), + ); + let output = check(&dir); + assert_eq!( + output.status.code(), + Some(2), + "an unrecognised glyph read as resolved: {}", + stdout(&output) + ); +} + +#[test] +fn a_table_with_no_rows_is_a_failure_not_a_vacuous_pass() { + // The reason the predecessor existed: a renamed heading or a reformatted + // table satisfies every per-row assertion by having no rows to assert over. + let dir = table_repo("license-no-rows", "# Contributing\n\nno table here\n"); + let output = check(&dir); + assert_eq!( + output.status.code(), + Some(2), + "an empty parse read as all rows resolved: {}", + stdout(&output) + ); +} + +#[test] +fn output_is_a_pointer_never_the_table_body() { + let dir = table_repo( + "license-pointer-only", + &format!("{HEAD}| distinctive-tool | a distinctive use | _to confirm_ | ✅ |\n"), + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2)); + assert!( + !stdout(&output).contains("a distinctive use"), + "the table body is payload: {}", + stdout(&output) + ); +} + +#[test] +fn the_repo_as_it_stands_passes() { + let output = + common::run_at_real_root(&common::at_root(""), &["check", "--rule", "license-table"]); + assert_eq!( + output.status.code(), + Some(0), + "an adopted tool's license row is unresolved: {}", + stdout(&output) + ); +} diff --git a/crates/batten/tests/it/main.rs b/crates/batten/tests/it/main.rs index 70c2072bb..a86a631db 100644 --- a/crates/batten/tests/it/main.rs +++ b/crates/batten/tests/it/main.rs @@ -53,6 +53,7 @@ mod ask_disposition; mod attribution; mod attribution_provenance; mod authority_replay; +mod awk_regex; mod baseline; mod bats_invocation; mod board_receipts; @@ -65,6 +66,7 @@ mod bypass_scrub; mod call_arguments; mod call_background_flag; mod call_ceiling; +mod cap_drift; mod capture_fidelity; mod captured_facts; mod cfg_gated_test; @@ -73,12 +75,14 @@ mod ci_cache_declared; mod ci_hygiene; mod ci_parity; mod ci_suite_lane; +mod ci_tools; mod claim; mod claim_carry; mod claim_order; mod claim_race; mod claim_receipt; mod cli; +mod coderabbit_config; mod commit; mod commit_admission; mod commit_arm_sequencing; @@ -109,6 +113,7 @@ mod defects; mod derived_facts; mod design_audit; mod dev_profile; +mod digest_major_agreement; mod doctor; mod document_facts; mod document_read_count; @@ -116,6 +121,7 @@ mod done_not_landed; mod egress_fencing; mod emission_census; mod enforce_journal; +mod exec_lock; mod extension_surfaces; mod external_facts; mod extracted_facts; @@ -130,6 +136,7 @@ mod forge_facts; mod fuzz_corpus; mod gh_guard; mod git_facts; +mod glob_containment; mod glob_exclusion; mod guardrail_bypass; mod handler_dispatch; @@ -143,6 +150,7 @@ mod hk_fix_selection; mod hk_observation; mod hk_plan; mod hook_cost; +mod hook_pin_check; mod hook_profile; mod hook_skip_local; mod hook_worktree_root; @@ -165,27 +173,34 @@ mod lease_health; mod lease_lifecycle; mod lease_precondition; mod lease_record; +mod license_table; mod locator_index; mod lock_complete; mod mcp_dispatch; mod mcp_reduce_array; +mod mcp_spawn; mod mediated_admission; mod mediated_verbs; mod memories; mod memory_injection; mod minted_facts; +mod mise_action_floor; mod mise_pin_agreement; mod mise_preset; +mod module_map; +mod msrv_pin_agreement; mod mutate; mod mutation_declared_case; mod narrow_adoption; mod nextest_slow; +mod no_doctests; mod obligations_bound; mod outcome_advice; mod perf_assert; mod perf_compare; mod perf_pair; mod pinned_programs; +mod pipefail_grep; mod pipeline_shapes; mod plan_complete; mod pointer_only; @@ -208,10 +223,12 @@ mod process_group; mod prose_only; mod prospective_facts; mod provision; +mod publish_credential; mod ratchet; mod raw_tracker_read; mod ready; mod rebase; +mod receipt_clean; mod receipt_verified; mod reclaim_report_once; mod record_closes; @@ -223,6 +240,7 @@ mod release_provision_parity; mod remedy_authorship; mod repaired_arms; mod repetition; +mod report_only; mod retirement_doctrine; mod review_answered; mod review_dispatched; @@ -234,6 +252,7 @@ mod rules_drift; mod run_shape; mod run_shape_guard_door; mod runner_verdict; +mod rust_paths_check; mod sbom_inventory; mod scanner_taxonomy; mod secret_redaction; @@ -259,6 +278,7 @@ mod startup_bootstrap; mod stop_posture; mod store_lifecycle; mod submodule; +mod suite_cost_corpus; mod suite_subjects; mod surface; mod symbols; @@ -268,6 +288,7 @@ mod task_prose; mod task_receipt; mod task_registry; mod test_targets; +mod timeout_budget; mod todo_promotion; mod tool_selector; mod tool_verdict_facts; @@ -276,11 +297,13 @@ mod transcript_tool_result; mod trunk_watch; mod turn_cross_check; mod use_graph; +mod verdict; mod verdict_registry; mod verdict_vocabulary; mod waivers; mod walker; mod wiring_disarm; mod wiring_reclaim; +mod workflow_shell_census; mod worktree_registration; mod zero_config; diff --git a/crates/batten/tests/it/mcp_spawn.rs b/crates/batten/tests/it/mcp_spawn.rs new file mode 100644 index 000000000..1aabe0632 --- /dev/null +++ b/crates/batten/tests/it/mcp_spawn.rs @@ -0,0 +1,315 @@ +//! `batten mcp spawn`, over the compiled binary — CLOUD-714's ledger, ported off +//! `mise-tasks/serena-mcp.sh` under CLOUD-1753. +//! +//! # What the shim was for, and why the port changes none of it +//! +//! Serena failed to attach three times on 2026-08-19, each burning ~28,300 ms of +//! a 30,000 ms budget and leaving NO trace: no server log, no `Server stderr:` +//! record in the client's own log, nothing. Two very different failures produce +//! that exact signature — the client never executed the configured command, or it +//! executed it and the child died before the server opened its log file, which is +//! ~1.2 s of import into the process. Nothing could tell those apart, and a day +//! went into archaeology that still could not answer it. +//! +//! The answer is a file read: the FIRST thing that happens is a line saying the +//! launch ran, and the LAST is becoming the launch line. A connect timeout WITH a +//! matching record means spawned-and-unresponsive; one WITHOUT means never +//! spawned. `mcp-attach-check` makes that comparison, and it opens the ledger BY +//! PATH — so the location and the tab-separated layout are a contract with a +//! reader this port does not own, preserved exactly. +//! +//! # THIS TIER EXISTS FOR ONE PROPERTY THE MODULE'S OWN CASES CANNOT REACH +//! +//! `exec` versus fork is observable only from OUTSIDE the process, and only over +//! a real binary: the recorded pid and the launched program's own pid are the +//! same number if and only if this process was replaced. A unit case calling +//! `record_spawn` sees neither. That is the property CLOUD-714 forbids losing — +//! after an exec there is no process left that could become the retry loop or the +//! supervisor the issue rules out — so it is asserted here, against the compiled +//! binary, exactly as the retiring suite asserted it against the shim. +//! +//! # What the port improves, and it is the one thing the shell could not do +//! +//! The shell derived the server name from its own basename, so a second server +//! meant a second COPY of the script — `serena-mcp.sh`, then `foo-mcp.sh`, each +//! with its own basename-stripping and its own ledger logic. The verb takes the +//! name as an argument, so a second server is an argument. +// +// carried: mise-tasks/serena-mcp.sh crates/batten/src/mcp.rs kind:verb crates/batten/tests/it/mcp_spawn.rs runs:batten+mcp+spawn +// carried: tests/serena-mcp.bats crates/batten/src/mcp.rs kind:verb crates/batten/tests/it/mcp_spawn.rs runs:batten+mcp+spawn +// +// carried: "a launch appends one record naming the server, and execs the launch line" crates/batten/src/mcp.rs +// carried: "THE SERVER'S PID IS THE SHIM'S — it execs rather than forks" crates/batten/src/mcp.rs +// carried: "the record carries five fields: epoch, server, pid, load, siblings" crates/batten/src/mcp.rs +// carried: "a launch inside the window counts the earlier one as a sibling" crates/batten/src/mcp.rs +// carried: "a launch outside the window counts no sibling" crates/batten/src/mcp.rs +// carried: "STDOUT CARRIES ONLY THE SERVER'S BYTES — stdout is the MCP transport" crates/batten/src/mcp.rs +// carried: "an unwritable ledger never stops the server from starting" crates/batten/src/mcp.rs +// changed: "the server name comes from the shim's own basename, so a second server is a second name" crates/batten/src/mcp.rs from `tests/serena-mcp.bats`. The property it protected — one name per server, never one script per server — is now structural: the name is a positional argument, so a second server cannot get the first one's name without somebody typing it. What the case actually asserted was a basename-stripping loop (`.sh` off before `-mcp`, in that order, or the suffix never matches) and that loop does not exist any more. `a_second_server_is_an_argument_rather_than_a_second_script` is the property without the loop +// changed: "the shim is what .mcp.json launches, so the ledger is populated in real sessions" crates/batten/src/mcp.rs from `tests/serena-mcp.bats`. Same claim, new spelling: the case grepped `.mcp.json` for the script path, and `the_committed_client_config_launches_this_verb` reads the same file for the verb's argv. It is CHANGED rather than CARRIED because the committed spelling moved from a path to a command line +// changed: "the launch args stay in .mcp.json, so the pin gate still reads them" crates/batten/src/mcp.rs from `tests/serena-mcp.bats`. The args are still in `.mcp.json` and `mise-pin-agreement` still reads the pin out of them, but they now sit BEHIND the verb's trailing separator rather than at the head of the array. The case asserted their position as well as their presence, and only the presence survives — which is the half the pin gate needs + +// Panicking on setup failure is the idiomatic way for a test to fail loudly. +#![allow(clippy::unwrap_used, clippy::expect_used)] + +use crate::common; + +use std::path::{Path, PathBuf}; + +use common::{at_root, batten}; + +/// A launcher standing in for the real server command. +/// +/// **It prints its own pid**, which is the only thing that can distinguish exec +/// from fork from outside the process — the retiring suite's device, kept because +/// it is the right one. +fn launcher(dir: &Path) -> PathBuf { + let path = dir.join("launcher"); + common::write( + dir, + "launcher", + "#!/usr/bin/env bash\nprintf 'pid=%s args=%s\\n' \"$$\" \"$*\"\n", + ); + #[cfg(unix)] + { + use std::os::unix::fs::PermissionsExt as _; + std::fs::set_permissions(&path, std::fs::Permissions::from_mode(0o755)).unwrap(); + } + path +} + +/// A git repository with a launcher in it, and the ledger path a reader would +/// open. +fn bench(name: &str) -> (PathBuf, PathBuf, PathBuf) { + let dir = common::scratch_outside_tree("batten-mcp-spawn", name); + // `init_repo` rather than a `git init` of this suite's own, which is what + // `fixture-forks` refuses and caught here on the first run: the harness + // publishes a template once per filesystem and copies it, so a fixture that + // forked for itself would put back the processes that mechanism removed. + common::init_repo(&dir); + let launcher = launcher(&dir); + let ledger = dir.join(".git").join("batten-mcp-spawns"); + (dir, launcher, ledger) +} + +/// Run `mcp spawn -- `. +/// +/// **The `--` is required rather than decorative.** The trailing list is declared +/// `last(true)`, so clap will not take a free token as the start of the launch +/// line — which is the right refusal for this verb: a launch line that began +/// wherever the parser guessed could silently drop its first word, and the ledger +/// would record a launch of something else. +fn spawn(dir: &Path, server: &str, launcher: &Path, args: &[&str]) -> std::process::Output { + let mut command = batten(); + command.arg("mcp").arg("spawn").arg(server).arg("--"); + command.arg(launcher); + command.args(args); + command + .current_dir(dir) + .output() + .expect("run batten mcp spawn") +} + +fn ledger_lines(ledger: &Path) -> Vec> { + let Ok(text) = std::fs::read_to_string(ledger) else { + return Vec::new(); + }; + text.lines() + .map(|line| line.split('\t').map(str::to_owned).collect()) + .collect() +} + +#[test] +fn a_launch_appends_one_record_naming_the_server_and_becomes_the_launch_line() { + let (dir, launcher, ledger) = bench("one-record"); + let output = spawn( + &dir, + "serena", + &launcher, + &["exec", "pipx:serena-agent@1.6.1"], + ); + assert_eq!(output.status.code(), Some(0), "{output:?}"); + let text = String::from_utf8_lossy(&output.stdout).into_owned(); + assert!( + text.contains("args=exec pipx:serena-agent@1.6.1"), + "the launch line runs verbatim: {text}" + ); + let lines = ledger_lines(&ledger); + assert_eq!(lines.len(), 1, "{lines:?}"); + assert_eq!(lines[0][1], "serena"); +} + +#[test] +fn the_servers_pid_is_this_processs_it_execs_rather_than_forks() { + // NON-NEGOTIABLE FOR THIS DESIGN, and the one property only a compiled-binary + // tier can see. After `exec` there is no process left, so the verb + // structurally cannot become the supervisor or retry loop CLOUD-714 forbids. + // A fork would also leave the recorded pid pointing at a process that is not + // the server, which is a wrong answer to the question being asked. + let (dir, launcher, ledger) = bench("exec-not-fork"); + let output = spawn(&dir, "serena", &launcher, &["x"]); + assert_eq!(output.status.code(), Some(0), "{output:?}"); + let lines = ledger_lines(&ledger); + let recorded = &lines[0][2]; + let text = String::from_utf8_lossy(&output.stdout).into_owned(); + assert!( + text.contains(&format!("pid={recorded}")), + "the recorded pid must BE the server's: recorded {recorded}, said {text}" + ); +} + +#[test] +fn the_record_carries_five_fields() { + let (dir, launcher, ledger) = bench("five-fields"); + spawn(&dir, "serena", &launcher, &["x"]); + let lines = ledger_lines(&ledger); + assert_eq!(lines[0].len(), 5, "{lines:?}"); + assert!(lines[0][0].parse::().is_ok(), "{lines:?}"); + assert!(lines[0][4].parse::().is_ok(), "{lines:?}"); +} + +#[test] +fn a_launch_inside_the_window_counts_the_earlier_one_as_a_sibling() { + // The hypothesis field. All three CLOUD-714 failures happened during a + // multi-server startup burst and every successful isolated replication was a + // lone launch, which is n=3 with no mechanism attached — so this count is + // what lets the NEXT occurrence decide it. + let (dir, launcher, ledger) = bench("sibling-inside"); + spawn(&dir, "serena", &launcher, &["x"]); + spawn(&dir, "other", &launcher, &["x"]); + let lines = ledger_lines(&ledger); + assert_eq!(lines.len(), 2, "{lines:?}"); + // Counted BEFORE the append, so a launch never counts itself. + assert_eq!(lines[0][4], "0", "{lines:?}"); + assert_eq!(lines[1][4], "1", "{lines:?}"); +} + +#[test] +fn a_launch_outside_the_window_counts_no_sibling() { + let (dir, launcher, ledger) = bench("sibling-outside"); + // An entry old enough to be outside the ten-second window, written directly: + // the alternative is sleeping for the window, which buys the same assertion + // at ten seconds a run. + common::write( + &dir.join(".git"), + "batten-mcp-spawns", + "1000000000\tancient\t1\t0.0\t0\n", + ); + spawn(&dir, "serena", &launcher, &["x"]); + let lines = ledger_lines(&ledger); + assert_eq!(lines.len(), 2, "{lines:?}"); + assert_eq!(lines[1][4], "0", "{lines:?}"); +} + +#[test] +fn stdout_carries_only_the_servers_bytes() { + // STDOUT IS THE MCP TRANSPORT. One stray byte corrupts the JSON-RPC stream + // and takes the server down in a way that looks exactly like the bug this + // records, so Batten writes nothing there — not a pointer, not a + // confirmation, not a diagnostic. + let (dir, launcher, _) = bench("stdout-clean"); + let output = spawn(&dir, "serena", &launcher, &["x"]); + let text = String::from_utf8_lossy(&output.stdout).into_owned(); + for line in text.lines() { + assert!( + line.starts_with("pid="), + "only the launched program may write to stdout: {line}" + ); + } +} + +#[test] +fn an_unwritable_ledger_never_stops_the_server_from_starting() { + // A ledger that cannot be written must never be the reason a server does not + // start. The launch is the product; the record is the diagnosis. + let (dir, launcher, ledger) = bench("ledger-unwritable"); + // A DIRECTORY where the ledger goes: opening it for append fails on every + // platform, without needing a permission bit that a root-running container + // ignores. + std::fs::create_dir_all(&ledger).unwrap(); + let output = spawn(&dir, "serena", &launcher, &["x"]); + assert_eq!(output.status.code(), Some(0), "{output:?}"); + let text = String::from_utf8_lossy(&output.stdout).into_owned(); + assert!(text.contains("pid="), "the server still started: {text}"); +} + +#[test] +fn a_launch_outside_a_checkout_still_starts_the_server() { + // Outside a repository there is nowhere per-clone to keep the ledger, and + // inventing a path under the temp directory would put it where no gate reads. + // So this records nothing and launches anyway, which is the same priority the + // unwritable case states. + let dir = common::scratch_outside_tree("batten-mcp-spawn", "no-repo"); + let launcher = launcher(&dir); + let output = spawn(&dir, "serena", &launcher, &["x"]); + assert_eq!(output.status.code(), Some(0), "{output:?}"); + assert!( + String::from_utf8_lossy(&output.stdout).contains("pid="), + "{output:?}" + ); +} + +#[test] +fn a_second_server_is_an_argument_rather_than_a_second_script() { + // What the retired basename case protected, without the basename loop. The + // shell read the server out of its own file name, so a second server meant a + // second COPY of the script — and the stripping order was load-bearing + // (`.sh` off before `-mcp`, or the suffix never matches, CLOUD-865). + let (dir, launcher, ledger) = bench("second-server"); + spawn(&dir, "serena", &launcher, &["x"]); + spawn(&dir, "context7", &launcher, &["x"]); + let lines = ledger_lines(&ledger); + assert_eq!(lines[0][1], "serena", "{lines:?}"); + assert_eq!(lines[1][1], "context7", "{lines:?}"); +} + +#[test] +fn a_launch_line_that_will_not_start_is_a_refusal_and_not_a_silent_success() { + // `exec` returns only on failure, so reaching the line after it IS the error. + // A verb that reported success here would tell a reader the server was + // launched when nothing was — which is the false half of exactly the + // distinction the ledger exists to draw. + let (dir, _, _) = bench("launch-fails"); + let missing = dir.join("not-a-program"); + let output = spawn(&dir, "serena", &missing, &[]); + assert_eq!(output.status.code(), Some(1), "{output:?}"); + let text = String::from_utf8_lossy(&output.stderr).into_owned(); + assert!(text.contains("could not become"), "{text}"); +} + +#[test] +fn the_committed_client_config_launches_this_verb() { + // The retired case grepped `.mcp.json` for the script path; this reads the + // same file for the verb's argv. Without it the ledger is empty in every real + // session and the whole mechanism is inert — which is the property, and it + // cannot be asserted about a fixture. + let config = std::fs::read_to_string(at_root(".mcp.json")).unwrap(); + let parsed: serde_json::Value = serde_json::from_str(&config).unwrap(); + let serena = &parsed["mcpServers"]["serena"]; + assert_eq!(serena["command"], "batten", "{parsed}"); + let args: Vec = serena["args"] + .as_array() + .unwrap() + .iter() + .map(|value| value.as_str().unwrap().to_owned()) + .collect(); + assert_eq!(args[0], "mcp", "{args:?}"); + assert_eq!(args[1], "spawn", "{args:?}"); + assert_eq!(args[2], "serena", "{args:?}"); + // THE SEPARATOR IS PART OF THE CONTRACT. Without it clap refuses the argv + // outright, so a config that dropped it would fail every launch — and the + // failure would look like the client never spawning the server, which is the + // exact reading this ledger exists to disambiguate. + assert_eq!(args[3], "--", "{args:?}"); +} + +#[test] +fn the_pinned_launch_args_are_still_in_the_committed_config() { + // `mise-pin-agreement` reads the pin out of `.mcp.json`, so the args have to + // stay there. They moved BEHIND the verb rather than leading the array, and + // the pin gate reads presence rather than position — which is why the arm for + // this one is `// changed:` and not a carry. + let config = std::fs::read_to_string(at_root(".mcp.json")).unwrap(); + assert!(config.contains("pipx:serena-agent@"), "{config}"); + assert!(config.contains("start-mcp-server"), "{config}"); +} diff --git a/crates/batten/tests/it/mise_action_floor.rs b/crates/batten/tests/it/mise_action_floor.rs new file mode 100644 index 000000000..65e197e6c --- /dev/null +++ b/crates/batten/tests/it/mise_action_floor.rs @@ -0,0 +1,253 @@ +//! No workflow pins a toolchain-install commit that predates its download +//! retry, over the compiled binary (CLOUD-404, ported from +//! `mise-tasks/mise-action-floor.sh` under CLOUD-843). +//! +//! **What is decidable only here.** `policy/mise-action-floor.rego` carries +//! load-time cases pinning the predicate, and every one of them supplies +//! `input.tree.lines` with `with input as`. That fabricates the very shape the +//! engine may be unable to produce (CLOUD-845) — and here it fabricates the +//! multi-path resolution of a declared `line_sources` GLOB, which is exactly what +//! the anti-vacuity arm turns on: a module whose suite only fabricated the map +//! would report "no pin found" as could-not-look while an engine that resolved +//! nothing at all reported the same thing, and the two would be +//! indistinguishable. +//! +//! The self-consumption case is the one the retiring suite ended on: this +//! repository's own workflows are judged, so the floor is a live claim rather +//! than an asserted one. +// +// carried: mise-tasks/mise-action-floor.sh policy/mise-action-floor.rego crates/batten/tests/it/mise_action_floor.rs +// carried: tests/mise-action-floor.bats policy/mise-action-floor.rego crates/batten/tests/it/mise_action_floor.rs +// +// carried: "a pin carrying the retry passes, and says what it judged" policy/mise-action-floor.rego +// carried: "THE ACCEPTANCE CASE: a pre-retry pin fails and is named with path:line" policy/mise-action-floor.rego +// carried: "THE BACKSLIDE: one reverted pin among many still fails" policy/mise-action-floor.rego +// carried: "SHOWN ABLE TO FAIL IN BOTH DIRECTIONS: every pin reverted fails with the full count" policy/mise-action-floor.rego +// carried: "the predicate is scoped to this action, so a lookalike coordinate does not fire" policy/mise-action-floor.rego +// carried: "the sha in prose or a comment is not a pin" policy/mise-action-floor.rego +// carried: "ANTI-VACUITY: a workflow with no mise-action pin is exit 2, never a pass" policy/mise-action-floor.rego +// carried: "an unversioned float is not a pin this gate can judge, so it is exit 2" policy/mise-action-floor.rego +// carried: "POINTER, NEVER PAYLOAD: the report carries no workflow content" policy/mise-action-floor.rego +// changed: "COULD NOT LOOK: a missing path is exit 2 rather than an empty pass" policy/mise-action-floor.rego the shell took explicit paths and refused one it could not open; the successor takes a declared `line_sources` glob and the ENGINE decides this earlier — a rule whose glob matches nothing is not evaluated at all, and `input.tree.missing` is never populated on the tree surface (CLOUD-1049). What the case protected is not lost: a tree with workflows but no pin of this action still reports could-not-look, which is `a_tree_with_no_pin_of_this_action_is_not_clean` + +// Panicking on setup failure is the idiomatic way for a test to fail loudly. +#![allow(clippy::unwrap_used, clippy::expect_used)] + +use crate::common; + +use std::path::{Path, PathBuf}; +use std::process::Output; + +use common::{Fixture, git_in, run, stderr, stdout}; + +/// The pre-retry commit, split so the literal never reads as a live coordinate +/// in this file. +const PRE_RETRY: &str = "7e36c90d9ab29c415a2384db3006f3ec8a8cc654"; + +/// Any later commit. Its only property that matters is not being the one above. +const FORWARD: &str = "1111111111111111111111111111111111111111"; + +/// A repository declaring only this rule, so any finding is the one under test. +/// +/// The `[[pattern]]` row is carried in beside the two `[[verdict]]` rows for the +/// same reason: a module referencing a pattern no row declares is refused AT +/// LOAD, which is a usage error rather than the verdict these cases are about. +fn floor_repo(name: &str, workflows: &[(&str, &str)]) -> PathBuf { + let mut fixture = Fixture::new(name).config( + "version = 1\n\n\ + [[pattern]]\n\ + id = \"git-object-id\"\n\ + regex = '^[0-9a-f]{40}$'\n\n\ + [[verdict]]\n\ + id = \"version pin stale\"\n\ + gloss = \"a pin resolves behind a fix this repository depends on\"\n\ + class = \"A backslide auto-lands silently and the next occurrence reads as fresh.\"\n\n\ + [[verdict.route]]\n\ + id = \"prose read first\"\n\ + kind = \"document\"\n\ + target = \"AGENTS.md\"\n\n\ + [[verdict]]\n\ + id = \"version pin unread\"\n\ + gloss = \"the pin under judgement could not be read\"\n\ + class = \"A gate whose subject can vanish and read as clean is not a gate.\"\n\n\ + [[verdict.route]]\n\ + id = \"prose read first\"\n\ + kind = \"document\"\n\ + target = \"AGENTS.md\"\n\n\ + [[rule]]\n\ + id = \"mise-action-floor\"\n\ + kind = \"policy\"\n\ + scope = \"tree\"\n\ + line_sources = [\".github/workflows/*.yml\"]\n\ + module = \"policy/mise-action-floor.rego\"\n\ + severity = \"deny\"\n", + ); + fixture = fixture.file("AGENTS.md", "the consumer's own authority\n"); + for (path, body) in workflows { + fixture = fixture.file(path, body); + } + let dir = fixture.git().build(); + common::write( + &dir, + "policy/mise-action-floor.rego", + &std::fs::read_to_string(common::at_root("policy/mise-action-floor.rego")).unwrap(), + ); + git_in(&dir, &["add", "-A"]); + git_in(&dir, &["commit", "-q", "-m", "base"]); + dir +} + +fn check(dir: &Path) -> Output { + run(dir, &["check", "--rule", "mise-action-floor"]) +} + +/// A workflow whose install step pins the action at `reference`. +fn workflow(reference: &str) -> String { + format!( + "on: push\njobs:\n build:\n runs-on: ubuntu-latest\n steps:\n - uses: jdx/mise-action@{reference}\n" + ) +} + +#[test] +fn a_pin_carrying_the_retry_passes() { + let dir = floor_repo( + "floor-forward", + &[(".github/workflows/ci.yml", &workflow(FORWARD))], + ); + let output = check(&dir); + assert_eq!( + output.status.code(), + Some(0), + "out={} err={}", + stdout(&output), + stderr(&output) + ); +} + +#[test] +fn a_pre_retry_pin_is_refused_and_named() { + let dir = floor_repo( + "floor-backslide", + &[(".github/workflows/ci.yml", &workflow(PRE_RETRY))], + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); + assert!( + stdout(&output).contains(".github/workflows/ci.yml:"), + "the finding points at the workflow line: {:?}", + stdout(&output) + ); +} + +#[test] +fn one_reverted_pin_among_many_still_fails() { + // THE BACKSLIDE the gate exists for: the bot bumps one workflow and the + // others stay forward, so a whole-tree "all pins agree" reading would miss + // it. + let dir = floor_repo( + "floor-one-among-many", + &[ + (".github/workflows/ci.yml", &workflow(FORWARD)), + (".github/workflows/release.yml", &workflow(PRE_RETRY)), + ], + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); + assert!( + stdout(&output).contains("release.yml"), + "the finding names the reverted workflow: {:?}", + stdout(&output) + ); +} + +#[test] +fn a_lookalike_coordinate_does_not_fire() { + // The same sha on a different action is not this defect. A gate firing on a + // lookalike trains its readers to ignore it. + let dir = floor_repo( + "floor-lookalike", + &[( + ".github/workflows/ci.yml", + &format!( + "{} - uses: some/other-action@{PRE_RETRY}\n", + workflow(FORWARD) + ), + )], + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(0), "{}", stdout(&output)); +} + +#[test] +fn a_sha_in_prose_is_not_a_pin() { + let dir = floor_repo( + "floor-prose", + &[( + ".github/workflows/ci.yml", + &format!( + "{} # was {PRE_RETRY} before the retry landed\n", + workflow(FORWARD) + ), + )], + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(0), "{}", stdout(&output)); +} + +#[test] +fn a_tree_with_no_pin_of_this_action_is_not_clean() { + // ANTI-VACUITY: the thing under test must not be able to vanish and read as + // clean. + let dir = floor_repo( + "floor-vacuous", + &[( + ".github/workflows/ci.yml", + "on: push\njobs:\n build:\n steps:\n - uses: actions/checkout@v4\n", + )], + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); +} + +#[test] +fn a_floating_ref_is_not_a_pin_this_gate_can_judge() { + // `@v4` carries no sha, so the denylist cannot speak about it at all. + // Reporting green over it would be a claim the gate cannot support. + let dir = floor_repo( + "floor-floating", + &[(".github/workflows/ci.yml", &workflow("v4"))], + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); +} + +#[test] +fn output_is_pointer_only() { + let dir = floor_repo( + "floor-pointer-only", + &[(".github/workflows/ci.yml", &workflow(PRE_RETRY))], + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2)); + let text = stdout(&output); + assert!( + !text.contains("ubuntu-latest") && !text.contains("runs-on"), + "the workflow's content is payload: {text:?}" + ); +} + +#[test] +fn this_repositorys_own_workflows_hold_the_floor() { + // The self-consumption case the retiring suite ended on: the floor is a live + // claim about this repository, checked rather than asserted. + let output = common::run_at_real_root( + &common::at_root(""), + &["check", "--rule", "mise-action-floor"], + ); + assert_eq!( + output.status.code(), + Some(0), + "a workflow here pins behind the download retry: {}", + stdout(&output) + ); +} diff --git a/crates/batten/tests/it/module_map.rs b/crates/batten/tests/it/module_map.rs new file mode 100644 index 000000000..3b15f18fd --- /dev/null +++ b/crates/batten/tests/it/module_map.rs @@ -0,0 +1,207 @@ +//! Every crate source module has a `mem:core` row, over the compiled binary +//! (CLOUD-194, ported from `mise-tasks/module-map-check.sh` under CLOUD-843). +//! +//! **What is decidable only here.** `policy/module-map.rego` carries load-time +//! cases pinning the predicate, and every one of them supplies +//! `input.tree.tracked` with `with input as`. That fabricates the very shape the +//! engine may be unable to produce — and here it fabricates the exact +//! distinction the gate turns on, TRACKED versus merely present, which is what +//! separates "a module landed without its row" from "somebody has a draft open". +//! A module whose suite only fabricated the set would stay green over an engine +//! that resolved the working tree instead of the index. +//! +//! The self-consumption case runs over this repository, which is what the +//! retiring suite's last case did and what makes the gate's own claim about +//! `mem:core` checkable rather than asserted. +// +// carried: mise-tasks/module-map-check.sh policy/module-map.rego crates/batten/tests/it/module_map.rs +// carried: tests/module-map-check.bats policy/module-map.rego crates/batten/tests/it/module_map.rs +// +// carried: "a module with a map row exits 0" policy/module-map.rego +// carried: "a module with no map row is reported with a pointer" policy/module-map.rego +// carried: "output is pointer-only — no map or source prose echoed" policy/module-map.rego +// changed: "an untracked module is not yet the map's problem" policy/module-map.rego the shell asked the INDEX via `git ls-files`; `input.tree.tracked` is a working-tree walk that explicitly is not the index, and nothing available to a module expresses index membership for a glob — `input.tree.staged` parses each declared path by format and no format owns `.rs`, and `git-status.changed` conflates untracked with modified. The successor is stricter in the fail-closed direction and `an_uncommitted_module_is_judged_too_where_the_shell_left_it_alone` pins the difference rather than leaving it to be discovered +// carried: "a bare filename mention does not satisfy the row" policy/module-map.rego +// changed: "a missing map is reported once, not once per module" policy/module-map.rego the clause is carried and is correct, and it cannot fire: a rule whose declared `line_sources` match nothing is not evaluated, and `input.tree.missing` is never populated on the tree surface (CLOUD-1049, measured identically for `policy/mise-pin-agreement.rego`'s own could-not-look clause). A case asserting it would assert the engine gap rather than the predicate, so the arm ships without one until the fact does +// carried: "every module of this repo has a row — the gate on the real tree" policy/module-map.rego + +// Panicking on setup failure is the idiomatic way for a test to fail loudly. +#![allow(clippy::unwrap_used, clippy::expect_used)] + +use crate::common; + +use std::path::{Path, PathBuf}; +use std::process::Output; + +use common::{Fixture, git_in, run, stderr, stdout}; + +/// A repository declaring only this rule, so any finding is the one under test. +fn map_repo(name: &str, map: Option<&str>, modules: &[(&str, &str)]) -> PathBuf { + // The two `[[verdict]]` rows are carried into the fixture rather than + // assumed: a verdict is emittable only where a row declares it, and the + // built-in registry is not this consumer's vocabulary. Without them the + // module loads and the run is a USAGE error, which is exit 1 and not the + // exit 2 these cases are about — so a fixture that omitted them would test + // the declaration rather than the predicate. + let mut fixture = Fixture::new(name).config( + "version = 1\n\n\ + [[verdict]]\n\ + id = \"memory resolve missing\"\n\ + gloss = \"the memory graph has no root\"\n\ + class = \"The map is the discovery entry point; absent, it is reachable only by listing the directory.\"\n\n\ + [[verdict.route]]\n\ + id = \"prose read first\"\n\ + kind = \"document\"\n\ + target = \"AGENTS.md\"\n\n\ + [[verdict]]\n\ + id = \"module list missing\"\n\ + gloss = \"a module is absent from its table\"\n\ + class = \"An unmapped module is not one with no constraints, it is one whose constraints nobody wrote down.\"\n\n\ + [[verdict.route]]\n\ + id = \"prose read first\"\n\ + kind = \"document\"\n\ + target = \"AGENTS.md\"\n\n\ + [[rule]]\n\ + id = \"module-map\"\n\ + kind = \"policy\"\n\ + scope = \"tree\"\n\ + line_sources = [\".serena/memories/core.md\"]\n\ + module = \"policy/module-map.rego\"\n\ + severity = \"deny\"\n", + ); + fixture = fixture.file("AGENTS.md", "the consumer's own authority\n"); + if let Some(text) = map { + fixture = fixture.file(".serena/memories/core.md", text); + } + for (path, body) in modules { + fixture = fixture.file(path, body); + } + let dir = fixture.git().build(); + // The module is copied in rather than referenced: the fixture is its own + // repository, and a rule row naming a path outside it would not resolve. + common::write( + &dir, + "policy/module-map.rego", + &std::fs::read_to_string(common::at_root("policy/module-map.rego")).unwrap(), + ); + git_in(&dir, &["add", "-A"]); + git_in(&dir, &["commit", "-q", "-m", "base"]); + dir +} + +fn check(dir: &Path) -> Output { + run(dir, &["check", "--rule", "module-map"]) +} + +const ROW: &str = "- `main.rs` — the binary boundary.\n"; + +#[test] +fn a_module_with_a_map_row_is_clean() { + let dir = map_repo( + "module-map-clean", + Some(ROW), + &[("crates/demo/src/main.rs", "fn main() {}\n")], + ); + let output = check(&dir); + assert_eq!( + output.status.code(), + Some(0), + "out={} err={}", + stdout(&output), + stderr(&output) + ); +} + +#[test] +fn a_module_with_no_map_row_is_refused_with_a_pointer() { + let dir = map_repo( + "module-map-absent-row", + Some(ROW), + &[ + ("crates/demo/src/main.rs", "fn main() {}\n"), + ("crates/demo/src/severity.rs", "pub fn f() {}\n"), + ], + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); + let text = stdout(&output); + assert!( + text.contains("crates/demo/src/severity.rs"), + "the finding points at the unmapped module: {text:?}" + ); +} + +#[test] +fn an_uncommitted_module_is_judged_too_where_the_shell_left_it_alone() { + // THE ONE BEHAVIOUR THIS PORT CHANGES, asserted rather than left to be + // discovered. `git ls-files` asked the INDEX; `input.tree.tracked` is a + // working-tree walk and explicitly is not the index, and nothing available + // to a module expresses index membership for a glob. So a module written + // but not yet committed is judged here, where the retiring suite pinned + // "an untracked module is not yet the map's problem". + // + // The direction is fail-closed and the cost is real: a contributor drafting + // a module is asked for its row before they commit it. This case exists so + // that the change is a decision CLOUD-1716 can keep or reverse, rather than + // a silent difference nobody measured. + let dir = map_repo( + "module-map-uncommitted", + Some(ROW), + &[("crates/demo/src/main.rs", "fn main() {}\n")], + ); + common::write(&dir, "crates/demo/src/draft.rs", "pub fn f() {}\n"); + + let output = check(&dir); + assert_eq!( + output.status.code(), + Some(2), + "the successor judges the checkout: {}", + stdout(&output) + ); +} + +#[test] +fn a_bare_mention_does_not_satisfy_the_row() { + // The map names modules in backticks. A sentence ABOUT a module must not + // read as a row, or the gate passes on the very drift it exists to catch. + let dir = map_repo( + "module-map-bare-mention", + Some("Note: severity.rs is described in another memory.\n"), + &[("crates/demo/src/severity.rs", "pub fn f() {}\n")], + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); +} + +#[test] +fn output_is_pointer_only() { + let dir = map_repo( + "module-map-pointer-only", + Some("- `main.rs` — the binary boundary, a distinctive phrase.\n"), + &[("crates/demo/src/hidden.rs", "pub fn secret_helper() {}\n")], + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2)); + let text = stdout(&output); + assert!( + !text.contains("secret_helper"), + "the module's source is payload: {text:?}" + ); + assert!( + !text.contains("distinctive phrase"), + "and so is the map's prose: {text:?}" + ); +} + +#[test] +fn the_repositorys_own_map_is_complete() { + // The self-consumption case the retiring suite ended on: the claim + // `rules/rust.md` makes about `mem:core` is checkable rather than asserted. + let output = common::run_at_real_root(&common::at_root(""), &["check", "--rule", "module-map"]); + assert_eq!( + output.status.code(), + Some(0), + "this repository's own module map is incomplete: {}", + stdout(&output) + ); +} diff --git a/crates/batten/tests/it/msrv_pin_agreement.rs b/crates/batten/tests/it/msrv_pin_agreement.rs new file mode 100644 index 000000000..ea50a5f0f --- /dev/null +++ b/crates/batten/tests/it/msrv_pin_agreement.rs @@ -0,0 +1,297 @@ +//! The floor, the toolchain pin and the bot's constraint name one compiler +//! line, over the compiled binary (CLOUD-593 and CLOUD-658, ported from +//! `mise-tasks/msrv-pin-agreement.sh` under CLOUD-843). +//! +//! **What is decidable only here.** `policy/msrv-pin-agreement.rego` carries +//! load-time cases pinning the predicate, and every one of them hands the module +//! a ready-made map of all three files with `with input as`. That fabricates the +//! very shape the engine may be unable to produce (CLOUD-845), and here it +//! fabricates the whole judgement: the agreement IS the join across three +//! separate `line_sources` paths resolved in one evaluation. A module whose suite +//! only fabricated the map would stay green over an engine that resolved two of +//! the three, which is the state in which the third copy silently stops being +//! gated — exactly what CLOUD-658 argues is safe only because this gate covers +//! it. +//! +//! The self-consumption case is the one the retiring suite ended on: the real +//! tree's three numbers agree, checked rather than asserted. +// +// carried: mise-tasks/msrv-pin-agreement.sh policy/msrv-pin-agreement.rego crates/batten/tests/it/msrv_pin_agreement.rs +// carried: tests/msrv-pin-agreement.bats policy/msrv-pin-agreement.rego crates/batten/tests/it/msrv_pin_agreement.rs +// +// carried: "the floor and the pin agreeing passes" policy/msrv-pin-agreement.rego +// carried: "the floor behind the pin is refused, and both values are named" policy/msrv-pin-agreement.rego +// carried: "the floor ahead of the pin is refused too — the check is equality, not a bound" policy/msrv-pin-agreement.rego +// carried: "a patch-only difference is agreement, not drift" policy/msrv-pin-agreement.rego +// carried: "a bare string pin is read, not only the inline-table form" policy/msrv-pin-agreement.rego +// carried: "a rust-version inside another table cannot answer for the workspace" policy/msrv-pin-agreement.rego +// carried: "a manifest with no rust-version is exit 2, never a silent pass" policy/msrv-pin-agreement.rego +// carried: "a tools file with no rust pin is exit 2, never a silent pass" policy/msrv-pin-agreement.rego +// carried: "all three agreeing passes" policy/msrv-pin-agreement.rego +// carried: "a Renovate constraint naming a different compiler is refused" policy/msrv-pin-agreement.rego +// carried: "a constraint ahead of the pin is refused too — equality, not a bound" policy/msrv-pin-agreement.rego +// carried: "a patch component in the constraint is agreement, not drift" policy/msrv-pin-agreement.rego +// carried: "a rust key outside the constraints block cannot answer for it" policy/msrv-pin-agreement.rego +// carried: "a missing constraints.rust is exit 2, never a silent pass" policy/msrv-pin-agreement.rego +// carried: "msrv-pin-agreement.bats::the real tree agrees" policy/msrv-pin-agreement.rego +// changed: "an unreadable file is exit 2 — a gate that cannot look must not report agreement" policy/msrv-pin-agreement.rego the shell opened three named files and refused one it could not read; the successor declares them as `line_sources` and the ENGINE decides this earlier — a rule whose declared paths match nothing is not evaluated at all, and `input.tree.missing` is never populated on the tree surface (CLOUD-1049, measured identically for `policy/mise-pin-agreement.rego`'s own could-not-look clause). What the case protected survives as the three PRESENT-BUT-SILENT arms: a file that resolves and carries no number is still a refusal, never agreement +// changed: "an unreadable renovate config is exit 2 on the same terms as the other two" policy/msrv-pin-agreement.rego same engine-side decision as the row above, and the arm that matters is kept: a config that resolves with no `constraints.rust` is `version declare missing`, because an absent constraint is MSRV-aware resolution switched off rather than a neutral omission + +// Panicking on setup failure is the idiomatic way for a test to fail loudly. +#![allow(clippy::unwrap_used, clippy::expect_used)] + +use crate::common; + +use std::path::{Path, PathBuf}; +use std::process::Output; + +use common::{Fixture, git_in, run, stderr, stdout}; + +/// A repository declaring only this rule, so any finding is the one under test. +fn msrv_repo(name: &str, manifest: &str, tools: &str, bot: &str) -> PathBuf { + let dir = Fixture::new(name) + .config( + "version = 1\n\n\ + [[pattern]]\n\ + id = \"md-quoted-span\"\n\ + regex = '\"[^\"]*\"'\n\n\ + [[verdict]]\n\ + id = \"version declare other\"\n\ + gloss = \"a derived copy names a different compiler than the pin\"\n\ + class = \"The pin is the authority; a copy that drifts switches off what it was written for.\"\n\n\ + [[verdict.route]]\n\ + id = \"prose read first\"\n\ + kind = \"document\"\n\ + target = \"AGENTS.md\"\n\n\ + [[verdict]]\n\ + id = \"version declare missing\"\n\ + gloss = \"a file that must carry the number carries none\"\n\ + class = \"Silence on any one of the three would read as agreement.\"\n\n\ + [[verdict.route]]\n\ + id = \"prose read first\"\n\ + kind = \"document\"\n\ + target = \"AGENTS.md\"\n\n\ + [[rule]]\n\ + id = \"msrv-pin-agreement\"\n\ + kind = \"policy\"\n\ + scope = \"tree\"\n\ + line_sources = [\"Cargo.toml\", \"mise.toml\", \"renovate.json5\"]\n\ + module = \"policy/msrv-pin-agreement.rego\"\n\ + severity = \"deny\"\n", + ) + .file("AGENTS.md", "the consumer's own authority\n") + .file("Cargo.toml", manifest) + .file("mise.toml", tools) + .file("renovate.json5", bot) + .git() + .build(); + common::write( + &dir, + "policy/msrv-pin-agreement.rego", + &std::fs::read_to_string(common::at_root("policy/msrv-pin-agreement.rego")).unwrap(), + ); + git_in(&dir, &["add", "-A"]); + git_in(&dir, &["commit", "-q", "-m", "base"]); + dir +} + +fn check(dir: &Path) -> Output { + run(dir, &["check", "--rule", "msrv-pin-agreement"]) +} + +fn cargo(version: &str) -> String { + format!("[workspace.package]\nedition = \"2024\"\nrust-version = \"{version}\"\n") +} + +fn tools(version: &str) -> String { + format!("[tools]\nrust = {{ version = \"{version}\", profile = \"minimal\" }}\n") +} + +fn bot(version: &str) -> String { + format!( + "{{\n \"$schema\": \"https://docs.renovatebot.com/renovate-schema.json\",\n // the pin is the authority; this is its derived copy\n constraints: {{\n rust: \"{version}\",\n }},\n}}\n" + ) +} + +/// The three files, each at its own version. +fn three(name: &str, floor: &str, pin: &str, constraint: &str) -> PathBuf { + msrv_repo(name, &cargo(floor), &tools(pin), &bot(constraint)) +} + +#[test] +fn all_three_naming_one_compiler_line_pass() { + let dir = three("msrv-agree", "1.97", "1.97.1", "1.97"); + let output = check(&dir); + assert_eq!( + output.status.code(), + Some(0), + "out={} err={}", + stdout(&output), + stderr(&output) + ); +} + +#[test] +fn a_patch_only_difference_is_agreement_not_drift() { + // The field is a MINIMUM; a patch component there says nothing extra, and + // reddening on every patch bump of the pin is the noise that gets a gate + // switched off. + let dir = three("msrv-patch-only", "1.97", "1.97.4", "1.97"); + let output = check(&dir); + assert_eq!(output.status.code(), Some(0), "{}", stdout(&output)); +} + +#[test] +fn a_floor_behind_the_pin_is_refused_and_both_are_named() { + // The whole defect: both numbers were `1.x` throughout the twelve releases + // they were apart, so a major-only comparison passes it. + let dir = three("msrv-floor-behind", "1.85", "1.97.1", "1.97"); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); + assert!( + stdout(&output).contains("1.85"), + "the finding names the divergent value: {:?}", + stdout(&output) + ); +} + +#[test] +fn a_floor_ahead_of_the_pin_is_refused_too() { + // Equality, not a bound. + let dir = three("msrv-floor-ahead", "1.99", "1.97.1", "1.97"); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); +} + +#[test] +fn a_bot_constraint_naming_another_compiler_is_refused() { + let dir = three("msrv-bot-behind", "1.97", "1.97.1", "1.85"); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); + assert!( + stdout(&output).contains("renovate.json5"), + "the finding names the file the copy came from: {:?}", + stdout(&output) + ); +} + +#[test] +fn a_bare_string_pin_is_read_too() { + // A gate that understood only one spelling would fail OPEN on the other. + let dir = msrv_repo( + "msrv-bare-pin", + &cargo("1.97"), + "[tools]\nrust = \"1.97.1\"\n", + &bot("1.97"), + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(0), "{}", stdout(&output)); +} + +#[test] +fn a_nested_floor_cannot_answer_for_the_workspace() { + let dir = msrv_repo( + "msrv-nested-floor", + "[workspace.dependencies.demo]\nversion = \"1\"\n rust-version = \"1.97\"\n", + &tools("1.97.1"), + &bot("1.97"), + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); +} + +#[test] +fn a_manifest_with_no_floor_is_refused() { + let dir = msrv_repo( + "msrv-no-floor", + "[workspace.package]\nedition = \"2024\"\n", + &tools("1.97.1"), + &bot("1.97"), + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); +} + +#[test] +fn a_tools_file_with_no_pin_is_refused() { + let dir = msrv_repo( + "msrv-no-pin", + &cargo("1.97"), + "[env]\nX = \"1\"\n", + &bot("1.97"), + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); +} + +#[test] +fn a_missing_bot_constraint_is_refused() { + // An absent constraint is MSRV-aware resolution silently switched off, not a + // neutral omission. + let dir = msrv_repo( + "msrv-no-constraint", + &cargo("1.97"), + &tools("1.97.1"), + "{\n \"$schema\": \"https://docs.renovatebot.com/renovate-schema.json\",\n}\n", + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); +} + +#[test] +fn a_commented_constraint_does_not_answer_for_the_real_one() { + // A gate a COMMENT could answer is a gate satisfied by deleting the value it + // explains. + let dir = msrv_repo( + "msrv-commented-constraint", + &cargo("1.97"), + &tools("1.97.1"), + "{\n // constraints: { rust: \"1.97\" } used to be declared here\n}\n", + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); +} + +#[test] +fn a_rust_key_outside_the_constraints_block_does_not_answer_for_it() { + let dir = msrv_repo( + "msrv-key-outside-block", + &cargo("1.97"), + &tools("1.97.1"), + "{\n packageRules: [{ rust: \"1.97\" }],\n}\n", + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); +} + +#[test] +fn output_is_pointer_only() { + let dir = msrv_repo( + "msrv-pointer-only", + &format!("{}# a distinctive phrase\n", cargo("1.85")), + &tools("1.97.1"), + &bot("1.97"), + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2)); + assert!( + !stdout(&output).contains("distinctive phrase"), + "the manifest's prose is payload: {:?}", + stdout(&output) + ); +} + +#[test] +fn the_real_trees_three_numbers_agree() { + // The self-consumption case the retiring suite ended on. + let output = common::run_at_real_root( + &common::at_root(""), + &["check", "--rule", "msrv-pin-agreement"], + ); + assert_eq!( + output.status.code(), + Some(0), + "the floor, the pin and the bot's constraint do not agree here: {}", + stdout(&output) + ); +} diff --git a/crates/batten/tests/it/no_doctests.rs b/crates/batten/tests/it/no_doctests.rs new file mode 100644 index 000000000..f5170b823 --- /dev/null +++ b/crates/batten/tests/it/no_doctests.rs @@ -0,0 +1,166 @@ +//! No runnable doctest exists, over the compiled binary (CLOUD-813, ported from +//! `mise-tasks/no-doctests.sh` under CLOUD-843). +//! +//! **What is decidable only here.** `policy/no-doctests.rego` carries load-time +//! cases pinning the parity rule and the attribute list, and every one supplies +//! `input.tree.lines` with `with input as`. That fabricates the scan's SUBJECT — +//! which is precisely what the shell spent an explicit anti-vacuity arm on, and +//! what CLOUD-418 names: a scan that matched no file at all reports "no runnable +//! doctest" over nothing, and reads as coverage. Only a real repository can show +//! the engine actually handing this module a workspace. +//! +//! The self-consumption case is the one that made the runner swap safe in the +//! first place: the class is empty on this workspace, and it is asserted here +//! rather than assumed. +// +// carried: mise-tasks/no-doctests.sh policy/no-doctests.rego crates/batten/tests/it/no_doctests.rs +// carried: tests/no-doctests.bats policy/no-doctests.rego crates/batten/tests/it/no_doctests.rs +// +// carried: "the committed workspace carries no runnable doctest" policy/no-doctests.rego +// carried: "an unattributed fence in a doc comment is refused" policy/no-doctests.rego +// carried: "the refusal is a pointer, never the example" policy/no-doctests.rego +// carried: "a text fence is not a doctest" policy/no-doctests.rego +// carried: "ignore, compile_fail and no_run are all non-running" policy/no-doctests.rego +// carried: "a closing fence is not read as an unattributed opening one" policy/no-doctests.rego +// carried: "a fence outside a doc comment is not a doctest" policy/no-doctests.rego +// +// THE TWO COULD-NOT-LOOK CASES, which the successor answers structurally rather +// than by an arm of its own. +// +// changed: "a root with no tracked .rs file is could-not-look, not clean" policy/no-doctests.rego the shell took its scan root as an ARGUMENT and so could be pointed at an empty one, which is why it needed the arm. The successor's subject is a declared `line_sources` glob rather than argv: a glob matching nothing means the rule is not evaluated, and the declaration itself is gated by `batten-glob-check`. There is no caller left that can aim it at an empty root, so the case has no subject rather than no coverage +// changed: "a missing root is could-not-look, not clean" policy/no-doctests.rego same reason, one step further: a root that does not exist is not expressible when the root is a committed glob rather than a positional argument + +// Panicking on setup failure is the idiomatic way for a test to fail loudly. +#![allow(clippy::unwrap_used, clippy::expect_used)] + +use crate::common; + +use std::path::{Path, PathBuf}; +use std::process::Output; + +use common::{Fixture, git_in, run, stdout}; + +/// A workspace declaring only this rule, so any finding is the one under test. +fn doctest_repo(name: &str, source: &str) -> PathBuf { + let dir = Fixture::new(name) + .config( + "version = 1\n\n\ + [[verdict]]\n\ + id = \"test place wrong\"\n\ + gloss = \"a test exists that nothing runs\"\n\ + class = \"An example nothing executes is dead code a reader trusts for being executable.\"\n\n\ + [[verdict.route]]\n\ + id = \"prose read first\"\n\ + kind = \"document\"\n\ + target = \"AGENTS.md\"\n\n\ + [[rule]]\n\ + id = \"no-doctests\"\n\ + kind = \"policy\"\n\ + scope = \"tree\"\n\ + line_sources = [\"crates/**/*.rs\"]\n\ + module = \"policy/no-doctests.rego\"\n\ + severity = \"deny\"\n", + ) + .file("AGENTS.md", "the consumer's own authority\n") + .file("crates/demo/src/lib.rs", source) + .git() + .build(); + common::write( + &dir, + "policy/no-doctests.rego", + &std::fs::read_to_string(common::at_root("policy/no-doctests.rego")).unwrap(), + ); + git_in(&dir, &["add", "-A"]); + git_in(&dir, &["commit", "-q", "-m", "base"]); + dir +} + +fn check(dir: &Path) -> Output { + run(dir, &["check", "--rule", "no-doctests"]) +} + +#[test] +fn an_unattributed_fence_is_refused_over_the_binary() { + let dir = doctest_repo( + "doctests-unattributed", + "/// ```\n/// let x = 1;\n/// ```\npub fn f() {}\n", + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); + assert!( + stdout(&output).contains("crates/demo/src/lib.rs"), + "the finding points at the fence's file: {}", + stdout(&output) + ); +} + +#[test] +fn a_text_fence_is_not_a_doctest_over_the_binary() { + let dir = doctest_repo( + "doctests-text-fence", + "/// ```text\n/// not rust\n/// ```\npub fn f() {}\n", + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(0), "{}", stdout(&output)); +} + +#[test] +fn a_closing_fence_is_not_read_as_an_unattributed_opener() { + // DISCRIMINATING. A closing fence carries no info string, so a scanner + // without the parity rule counts it as a second unattributed opener and + // refuses a file that is correctly marked. The shell toggled a flag; the + // successor counts parity, and this is the case that tells them from a + // scanner that does neither. + let dir = doctest_repo( + "doctests-closing-fence", + "/// ```text\n/// safe\n/// ```\npub fn f() {}\n", + ); + let output = check(&dir); + assert_eq!( + output.status.code(), + Some(0), + "the closing fence was read as an opener: {}", + stdout(&output) + ); +} + +#[test] +fn the_refusal_is_a_pointer_never_the_example() { + let dir = doctest_repo( + "doctests-pointer-only", + "/// ```\n/// let secret_example = 1;\n/// ```\npub fn f() {}\n", + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2)); + assert!( + !stdout(&output).contains("secret_example"), + "the example is payload and never appears: {}", + stdout(&output) + ); +} + +#[test] +fn a_fence_outside_a_doc_comment_is_not_a_doctest() { + let dir = doctest_repo( + "doctests-plain-comment", + "// ```\n// let x = 1;\n// ```\npub fn f() {}\n", + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(0), "{}", stdout(&output)); +} + +#[test] +fn the_committed_workspace_carries_no_runnable_doctest() { + // The self-consumption case, and the measurement the runner swap rested on: + // `cargo nextest run` executes no doctest, so an example here would be run + // nowhere. Asserted rather than assumed, because an empty class is not a + // stable property. + let output = + common::run_at_real_root(&common::at_root(""), &["check", "--rule", "no-doctests"]); + assert_eq!( + output.status.code(), + Some(0), + "a runnable doctest landed, and nextest runs none of them: {}", + stdout(&output) + ); +} diff --git a/crates/batten/tests/it/pipefail_grep.rs b/crates/batten/tests/it/pipefail_grep.rs new file mode 100644 index 000000000..8a7a68a45 --- /dev/null +++ b/crates/batten/tests/it/pipefail_grep.rs @@ -0,0 +1,181 @@ +//! No producer is piped into an early-exiting grep under pipefail, over the +//! compiled binary (ported from `mise-tasks/pipefail-grep-check.sh` under +//! CLOUD-843). +//! +//! **What is decidable only here.** `policy/pipefail-grep.rego` carries +//! load-time cases pinning the flag reading and the `||` distinction, and each +//! fabricates `input.tree.lines`. Only a real repository shows the engine +//! handing this module the shell corpus — the anti-vacuity half, since a scan +//! that matched no file reports "no producer is piped into an early-exiting +//! grep" over nothing and reads as coverage (CLOUD-418). +// +// carried: mise-tasks/pipefail-grep-check.sh policy/pipefail-grep.rego crates/batten/tests/it/pipefail_grep.rs +// carried: tests/pipefail-grep-check.bats policy/pipefail-grep.rego crates/batten/tests/it/pipefail_grep.rs +// +// carried: "the exact shape that broke issue-guard is flagged" policy/pipefail-grep.rego +// carried: "the here-string fix passes" policy/pipefail-grep.rego +// carried: "a flag cluster is judged by its letters, not its spelling" policy/pipefail-grep.rego +// carried: "--quiet is the same hazard under its long name" policy/pipefail-grep.rego +// carried: "-l stops at the first matching file, so it is flagged too" policy/pipefail-grep.rego +// carried: "-m N stops after N matches" policy/pipefail-grep.rego +// carried: "a grep that consumes its whole input is not the hazard" policy/pipefail-grep.rego +// carried: "an || before grep is not a pipe" policy/pipefail-grep.rego +// carried: "a real pipe into an early-exiting grep is still caught alongside an ||" policy/pipefail-grep.rego +// carried: "a file that does not enable pipefail is out of scope" policy/pipefail-grep.rego +// carried: "a comment describing the hazard is not the hazard" policy/pipefail-grep.rego +// carried: "-q after -- is a pattern, not a flag" policy/pipefail-grep.rego +// carried: "output is a pointer — file:line and the fix, never the matched content" policy/pipefail-grep.rego +// +// changed: "an untracked file is not judged — the gate reads committed bytes" policy/pipefail-grep.rego the shell fed itself `git ls-files`, so the index was its subject. `input.tree.lines` resolves from a declared glob over the working-tree walk, which honours `.gitignore` and explicitly is not the index, and nothing available to a module expresses index membership for a glob. The successor judges an uncommitted program too — stricter, fail-closed, and the same difference `policy/module-map.rego` and `policy/awk-regex.rego` record for their own ports + +// Panicking on setup failure is the idiomatic way for a test to fail loudly. +#![allow(clippy::unwrap_used, clippy::expect_used)] + +use crate::common; + +use std::path::{Path, PathBuf}; +use std::process::Output; + +use common::{Fixture, git_in, run, stdout}; + +fn pipefail_repo(name: &str, program: &str) -> PathBuf { + let dir = Fixture::new(name) + .config( + "version = 1\n\n\ + [[pattern]]\n\ + id = \"shell-enables-pipefail\"\n\ + regex = '^[[:space:]]*set[[:space:]]+-[a-z]*o?[a-z]*[[:space:]]*.*pipefail'\n\n\ + [[pattern]]\n\ + id = \"pipe-into-grep\"\n\ + regex = '(^|[^|])\\|[[:space:]]*grep([[:space:]]|$)'\n\n\ + [[verdict]]\n\ + id = \"spawn read broken\"\n\ + gloss = \"a command is spelled in a way whose behaviour is not defined\"\n\ + class = \"An early-exiting grep under pipefail promotes SIGPIPE to the pipeline's status, so a MATCH reports failure.\"\n\n\ + [[verdict.route]]\n\ + id = \"prose read first\"\n\ + kind = \"document\"\n\ + target = \"AGENTS.md\"\n\n\ + [[rule]]\n\ + id = \"pipefail-grep\"\n\ + kind = \"policy\"\n\ + scope = \"tree\"\n\ + line_sources = [\"mise-tasks/**\"]\n\ + module = \"policy/pipefail-grep.rego\"\n\ + severity = \"deny\"\n", + ) + .file("AGENTS.md", "the consumer's own authority\n") + .file("mise-tasks/demo.sh", program) + .git() + .build(); + common::write( + &dir, + "policy/pipefail-grep.rego", + &std::fs::read_to_string(common::at_root("policy/pipefail-grep.rego")).unwrap(), + ); + git_in(&dir, &["add", "-A"]); + git_in(&dir, &["commit", "-q", "-m", "base"]); + dir +} + +fn check(dir: &Path) -> Output { + run(dir, &["check", "--rule", "pipefail-grep"]) +} + +const HEAD: &str = "#!/usr/bin/env bash\nset -euo pipefail\n"; + +#[test] +fn the_shape_that_broke_issue_guard_is_flagged_over_the_binary() { + let dir = pipefail_repo( + "pipefail-issue-guard", + &format!("{HEAD}git log --format=%B main | grep -q \"$id\"\n"), + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); + let text = stdout(&output); + assert!( + text.contains("mise-tasks/demo.sh"), + "the finding points at the line: {text:?}" + ); + assert!( + !text.contains("git log"), + "the matched content is payload and never appears: {text:?}" + ); +} + +#[test] +fn the_here_string_fix_passes() { + let dir = pipefail_repo( + "pipefail-here-string", + &format!("{HEAD}x=$(git log)\ngrep -q \"$id\" <<<\"$x\"\n"), + ); + assert_eq!(check(&dir).status.code(), Some(0)); +} + +#[test] +fn a_flag_cluster_is_judged_by_its_letters_over_the_binary() { + // `-qxF` is the same hazard as `-q`. The predecessor's own comment says the + // enumeration of exact spellings is what would rot. + let dir = pipefail_repo( + "pipefail-cluster", + &format!("{HEAD}producer | grep -qxF thing\n"), + ); + assert_eq!(check(&dir).status.code(), Some(2)); +} + +#[test] +fn an_or_before_grep_is_not_a_pipe() { + // DISCRIMINATING, and the case the predecessor's own scan failed: it matched + // the SECOND bar of `||` and reported a here-string — the remedy this gate + // recommends — as the defect. Measured on `ready-lint.sh` (CLOUD-852). + let dir = pipefail_repo( + "pipefail-or", + &format!("{HEAD}[[ -n \"$x\" ]] || grep -qE 'p' <<<\"$var\"\n"), + ); + let output = check(&dir); + assert_eq!( + output.status.code(), + Some(0), + "the second bar of `||` was read as a pipe: {}", + stdout(&output) + ); +} + +#[test] +fn a_grep_that_consumes_its_whole_input_is_not_the_hazard() { + let dir = pipefail_repo( + "pipefail-full-consume", + &format!("{HEAD}producer | grep thing\n"), + ); + assert_eq!(check(&dir).status.code(), Some(0)); +} + +#[test] +fn a_file_that_does_not_enable_pipefail_is_out_of_scope() { + let dir = pipefail_repo( + "pipefail-not-enabled", + "#!/usr/bin/env bash\nproducer | grep -q thing\n", + ); + assert_eq!(check(&dir).status.code(), Some(0)); +} + +#[test] +fn q_after_the_separator_is_a_pattern_not_a_flag() { + let dir = pipefail_repo( + "pipefail-separator", + &format!("{HEAD}producer | grep -- -q\n"), + ); + assert_eq!(check(&dir).status.code(), Some(0)); +} + +#[test] +fn this_repos_own_programs_pass_today() { + let output = + common::run_at_real_root(&common::at_root(""), &["check", "--rule", "pipefail-grep"]); + assert_eq!( + output.status.code(), + Some(0), + "a producer is piped into an early-exiting grep in this tree: {}", + stdout(&output) + ); +} diff --git a/crates/batten/tests/it/pointer_only.rs b/crates/batten/tests/it/pointer_only.rs index 43ba583b0..095992c01 100644 --- a/crates/batten/tests/it/pointer_only.rs +++ b/crates/batten/tests/it/pointer_only.rs @@ -1511,6 +1511,34 @@ const CENSUS: &[Verb] = &[ stdin: Stdin::Nothing, disposition: Disposition::PointerOnly, }, + // CLOUD-1753's port of the MCP launcher shim, and its disposition is forced + // by the transport rather than chosen: STDOUT IS THE MCP STREAM. One stray + // byte from Batten corrupts the JSON-RPC framing and takes the server down + // looking exactly like the bug the ledger exists to diagnose. So this emits + // nothing at all on the clean path, and the census is what would catch a + // later revision that started narrating the launch. + // + // Driven against `true`, which is on `PATH` everywhere this suite runs and + // exits 0 without writing: the verb EXECS its argument, so the census would + // otherwise be measuring whatever program it was handed. + Verb { + path: "mcp spawn", + args: &["a-server", "--", "true"], + stdin: Stdin::Nothing, + disposition: Disposition::PointerOnly, + }, + // CLOUD-1718's fold, and the EASIEST row in this census to justify: the verb + // emits nothing at all. Its whole answer is the exit code, so there is no + // content it could republish and no subject it could echo — two integers on + // argv go in and one of §7's four codes comes out. `PointerOnly` is the + // honest disposition rather than a vacuous one, because the emitter is still + // exercised: the census drives it and asserts the channels stay empty. + Verb { + path: "verdict", + args: &["--findings", "0", "--unjudgeable", "0"], + stdin: Stdin::Nothing, + disposition: Disposition::PointerOnly, + }, // The one verb whose subject is a file OUTSIDE the repository (CLOUD-893), // which makes rule 4 tighter here rather than looser: what it removes is a // command line off somebody's home directory, so every byte it reports is a @@ -1589,6 +1617,17 @@ const CENSUS: &[Verb] = &[ stdin: Stdin::Nothing, disposition: Disposition::PointerOnly, }, + // CLOUD-1753's port of `tree-clean`, and pointer-only is NARROWER than what + // it replaces rather than equal to it. The retired program printed the + // porcelain status lines to stderr; this emits a count and a short head and + // nothing else, because the remedy is the same whichever paths are dirty — + // so a path a reader has no use for never leaves the process. + Verb { + path: "receipt clean", + args: &[], + stdin: Stdin::Nothing, + disposition: Disposition::PointerOnly, + }, // `receipt verified` composes the two receipt reads and renders a sha, a // CHECK NAME and a validity token. It is the same reading `receipt status` // gives, twice, and the predecessor it retired was already pointer-only for diff --git a/crates/batten/tests/it/publish_credential.rs b/crates/batten/tests/it/publish_credential.rs new file mode 100644 index 000000000..2d84ea0fc --- /dev/null +++ b/crates/batten/tests/it/publish_credential.rs @@ -0,0 +1,299 @@ +//! No long-lived registry credential can reach the release path, and publishing +//! implies OIDC, over the compiled binary (CLOUD-109, ported from +//! `mise-tasks/publish-credential-check.sh` under CLOUD-843). +//! +//! **What is decidable only here.** `policy/publish-credential.rego` carries +//! load-time cases pinning the predicate, and every one hands the module a +//! ready-made map of workflows with `with input as`. That fabricates the very +//! shape the engine may be unable to produce (CLOUD-845), and the first half of +//! this rule rests on it entirely: a credential can appear in ANY workflow, so a +//! declared `line_sources` glob that resolved only the release workflow — or +//! resolved a workflow directory to nothing — would leave the rest unscanned +//! with nothing red. That is the vacuous pass the retiring suite spent a case on, +//! and no `with input as` block can distinguish it from a scan that found +//! nothing. +//! +//! The self-consumption case is the one the retiring suite opened on: the tree as +//! it stands carries no registry credential. +// +// carried: mise-tasks/publish-credential-check.sh policy/publish-credential.rego crates/batten/tests/it/publish_credential.rs +// carried: tests/publish-credential-check.bats policy/publish-credential.rego crates/batten/tests/it/publish_credential.rs +// +// carried: "the tree as it stands passes: no registry credential, publishing off" policy/publish-credential.rego +// carried: "THE DEFECT: a workflow gaining CARGO_REGISTRY_TOKEN fails" policy/publish-credential.rego +// carried: "an alternate-registry token is caught too, whatever the registry is named" policy/publish-credential.rego +// carried: "a hand-rolled cargo login is the same defect spelled differently" policy/publish-credential.rego +// carried: "the finding is a pointer — a path and a rule id, never the matched line" policy/publish-credential.rego +// carried: "THE POINT: turning publishing on without id-token: write fails" policy/publish-credential.rego +// carried: "turning publishing on WITH id-token: write passes" policy/publish-credential.rego +// carried: "the permission is not satisfied by the phrase appearing in a comment" policy/publish-credential.rego +// carried: "publishing off does NOT require the permission — a dead grant is not the ask" policy/publish-credential.rego +// carried: "THE SILENT CASE: no publish key reads as publishing, because that is the default" policy/publish-credential.rego +// carried: "publishing on with no release workflow at all is exit 2" policy/publish-credential.rego +// changed: "an unreadable config is exit 2 — could not look, never a clean tree" policy/publish-credential.rego the shell opened a named file and refused when it was gone; the successor declares it as a `line_sources` path and the ENGINE decides this earlier — a rule whose declared paths match nothing is not evaluated at all, and `input.tree.missing` is never populated on the tree surface (CLOUD-1049, measured identically for `policy/mise-pin-agreement.rego`'s own could-not-look clause). The direction that mattered survives and is stronger for it: a config that resolves and says nothing reads as PUBLISHING, so silence is never the quiet arm +// changed: "a workflow directory with no workflows is exit 2, not a vacuous pass" policy/publish-credential.rego the shell listed a directory itself and refused an empty one; the successor is handed whatever the declared glob resolved, and cannot tell "no workflows exist" from "the glob resolved none" — both arrive as an empty map. The vacuity is not left unheld: it is the ENGINE's to answer, which is why this tier drives the compiled binary over trees that really do carry several workflows + +// Panicking on setup failure is the idiomatic way for a test to fail loudly. +#![allow(clippy::unwrap_used, clippy::expect_used)] + +use crate::common; + +use std::path::{Path, PathBuf}; +use std::process::Output; + +use common::{Fixture, git_in, run, stderr, stdout}; + +/// A repository declaring only this rule, so any finding is the one under test. +fn publish_repo(name: &str, config: &str, workflows: &[(&str, &str)]) -> PathBuf { + let mut fixture = Fixture::new(name).config( + "version = 1\n\n\ + [[pattern]]\n\ + id = \"cargo-registry-token\"\n\ + regex = 'CARGO_REGISTRY_TOKEN|CARGO_REGISTRIES_[A-Z0-9_]*_TOKEN'\n\n\ + [[pattern]]\n\ + id = \"cargo-login-call\"\n\ + regex = '(^|[^A-Za-z0-9_-])cargo login([^A-Za-z0-9_-]|$)'\n\n\ + [[pattern]]\n\ + id = \"oidc-token-permission\"\n\ + regex = '^[[:space:]]+id-token:[[:space:]]*write[[:space:]]*(#.*)?$'\n\n\ + [[verdict]]\n\ + id = \"grant carry unsafe\"\n\ + gloss = \"a workflow can reach a long-lived registry credential\"\n\ + class = \"A stored credential is the thing trusted publishing exists to retire.\"\n\n\ + [[verdict.route]]\n\ + id = \"prose read first\"\n\ + kind = \"document\"\n\ + target = \"AGENTS.md\"\n\n\ + [[verdict]]\n\ + id = \"lane grant missing\"\n\ + gloss = \"publishing is on without the credential-free way to do it\"\n\ + class = \"Publishing cannot be switched on except through OIDC, in the same commit.\"\n\n\ + [[verdict.route]]\n\ + id = \"prose read first\"\n\ + kind = \"document\"\n\ + target = \"AGENTS.md\"\n\n\ + [[rule]]\n\ + id = \"publish-credential\"\n\ + kind = \"policy\"\n\ + scope = \"tree\"\n\ + line_sources = [\"release-plz.toml\", \".github/workflows/*.yml\"]\n\ + module = \"policy/publish-credential.rego\"\n\ + severity = \"deny\"\n", + ); + fixture = fixture + .file("AGENTS.md", "the consumer's own authority\n") + .file("release-plz.toml", config); + for (path, body) in workflows { + fixture = fixture.file(path, body); + } + let dir = fixture.git().build(); + common::write( + &dir, + "policy/publish-credential.rego", + &std::fs::read_to_string(common::at_root("policy/publish-credential.rego")).unwrap(), + ); + git_in(&dir, &["add", "-A"]); + git_in(&dir, &["commit", "-q", "-m", "base"]); + dir +} + +fn check(dir: &Path) -> Output { + run(dir, &["check", "--rule", "publish-credential"]) +} + +const PLAIN: &str = "on: push\njobs:\n release-plz:\n permissions:\n contents: write\n steps:\n - run: release-plz release\n"; + +const WITH_OIDC: &str = "on: push\njobs:\n release-plz:\n permissions:\n contents: write\n id-token: write\n steps:\n - run: release-plz release\n"; + +#[test] +fn no_credential_and_publishing_off_passes() { + let dir = publish_repo( + "publish-clean", + "[workspace]\npublish = false\n", + &[(".github/workflows/release-plz.yml", PLAIN)], + ); + let output = check(&dir); + assert_eq!( + output.status.code(), + Some(0), + "out={} err={}", + stdout(&output), + stderr(&output) + ); +} + +#[test] +fn a_registry_token_in_any_workflow_is_refused() { + // THE DEFECT: a credential reaches every job that can read the secret, not + // the one step that needs it. + let dir = publish_repo( + "publish-token", + "[workspace]\npublish = false\n", + &[ + (".github/workflows/release-plz.yml", PLAIN), + ( + ".github/workflows/other.yml", + "on: push\njobs:\n x:\n env:\n CARGO_REGISTRY_TOKEN: ${{ secrets.X }}\n", + ), + ], + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); + assert!( + stdout(&output).contains("other.yml"), + "the finding names the workflow: {:?}", + stdout(&output) + ); +} + +#[test] +fn the_alternate_registry_form_is_refused_too() { + // Its middle segment is the registry's own name and cannot be enumerated, so + // the pattern matches the shape. + let dir = publish_repo( + "publish-alt-registry", + "[workspace]\npublish = false\n", + &[ + (".github/workflows/release-plz.yml", PLAIN), + ( + ".github/workflows/other.yml", + "on: push\njobs:\n x:\n env:\n CARGO_REGISTRIES_MYREG_TOKEN: ${{ secrets.X }}\n", + ), + ], + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); +} + +#[test] +fn a_hand_rolled_login_is_the_same_defect() { + let dir = publish_repo( + "publish-login", + "[workspace]\npublish = false\n", + &[ + (".github/workflows/release-plz.yml", PLAIN), + ( + ".github/workflows/other.yml", + "on: push\njobs:\n x:\n steps:\n - run: cargo login \"$TOKEN\"\n", + ), + ], + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); +} + +#[test] +fn the_finding_never_carries_the_matched_line() { + // Load-bearing rather than stylistic: the class of thing being looked for is + // the class that must not reach a log, so a gate that echoed the line would + // be the leak it exists to prevent. + let dir = publish_repo( + "publish-pointer-only", + "[workspace]\npublish = false\n", + &[ + (".github/workflows/release-plz.yml", PLAIN), + ( + ".github/workflows/other.yml", + "on: push\njobs:\n x:\n env:\n CARGO_REGISTRY_TOKEN: a-distinctive-literal\n", + ), + ], + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2)); + assert!( + !stdout(&output).contains("a-distinctive-literal"), + "the matched line is payload: {:?}", + stdout(&output) + ); +} + +#[test] +fn publishing_on_without_oidc_is_refused() { + // THE POINT: publishing cannot be switched on except through OIDC, in the + // same commit that switches it. + let dir = publish_repo( + "publish-on-no-oidc", + "[workspace]\npublish = true\n", + &[(".github/workflows/release-plz.yml", PLAIN)], + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); +} + +#[test] +fn publishing_on_with_oidc_passes() { + let dir = publish_repo( + "publish-on-with-oidc", + "[workspace]\npublish = true\n", + &[(".github/workflows/release-plz.yml", WITH_OIDC)], + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(0), "{}", stdout(&output)); +} + +#[test] +fn the_permission_named_in_a_comment_does_not_satisfy_it() { + let dir = publish_repo( + "publish-comment-permission", + "[workspace]\npublish = true\n", + &[( + ".github/workflows/release-plz.yml", + "on: push\njobs:\n release-plz:\n permissions:\n contents: write\n # id-token: write goes here when publishing turns on\n", + )], + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); +} + +#[test] +fn publishing_off_does_not_require_the_permission() { + // A dead grant is not the ask: granting a capability no step uses is a + // finding of its own to an excessive-permissions audit. + let dir = publish_repo( + "publish-off-no-permission", + "[workspace]\npublish = false\n", + &[(".github/workflows/release-plz.yml", PLAIN)], + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(0), "{}", stdout(&output)); +} + +#[test] +fn an_absent_publish_key_reads_as_publishing() { + // THE SILENT CASE: the release tool's own default is to publish, so reading + // silence as `false` would make the gate quiet in exactly the case it exists + // for. + let dir = publish_repo( + "publish-absent-key", + "[workspace]\nallow_dirty = false\n", + &[(".github/workflows/release-plz.yml", PLAIN)], + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); +} + +#[test] +fn publishing_on_with_no_release_workflow_is_not_a_pass() { + let dir = publish_repo( + "publish-no-release-workflow", + "[workspace]\npublish = true\n", + &[(".github/workflows/ci.yml", PLAIN)], + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); +} + +#[test] +fn the_tree_as_it_stands_carries_no_registry_credential() { + // The self-consumption case the retiring suite opened on. + let output = common::run_at_real_root( + &common::at_root(""), + &["check", "--rule", "publish-credential"], + ); + assert_eq!( + output.status.code(), + Some(0), + "a long-lived registry credential can reach this release path: {}", + stdout(&output) + ); +} diff --git a/crates/batten/tests/it/receipt_clean.rs b/crates/batten/tests/it/receipt_clean.rs new file mode 100644 index 000000000..d71216657 --- /dev/null +++ b/crates/batten/tests/it/receipt_clean.rs @@ -0,0 +1,234 @@ +//! `batten receipt clean` and the guard inside `receipt record`, over the +//! compiled binary — CLOUD-193's precondition, ported off +//! `mise-tasks/tree-clean.sh` under CLOUD-1753. +//! +//! # The assumption this enforces was load-bearing and unenforced +//! +//! CLOUD-193 moved `verify`'s verdict off the exit code and onto a receipt "keyed +//! to the exact HEAD it validated". That mechanism is sound and it rests on an +//! assumption nothing checked: that the bytes verified ARE the bytes at HEAD. +//! They need not be — `cargo`, `hk` and `zizmor` all read the WORKING TREE, and +//! `receipt record` keys the claim to HEAD. +//! +//! # The direction that matters is the SILENT one +//! +//! Measured 2026-08-09 landing CLOUD-269: a backgrounded `land` compiled a +//! mid-edit snapshot and died on a `non-exhaustive patterns` error for code the +//! commit did not contain. That direction is loud and self-correcting. The mirror +//! is not — a dirty tree that PASSES writes a receipt for HEAD, `verified` +//! matches it, `ready-guard` accepts it, the branch is readied, and CI runs the +//! commit alone, which was never the thing that passed. +//! +//! Uncommitted work is not exotic here: backgrounding the slow path is mandated, +//! so a long `verify` while the session edits the next row in the same worktree +//! is the DESIGNED workflow rather than a mistake. +//! +//! # WHAT THE PORT CHANGES, AND IT IS THE PART THE SHELL COULD NOT DO +//! +//! The retired program was a task, so it could only ever be a call somebody +//! remembered to make. Its own header says so — "NOT wired into the hk gate, +//! deliberately", because `pre-commit` runs over a tree that is dirty by +//! definition — and `batten.toml` twice cites the same shape as "the scoping +//! defect CLOUD-1164 records for `tree-clean`". +//! +//! The successor puts the load-bearing half INSIDE `receipt record`, which is the +//! verb that writes the receipt. So the guarantee stops depending on a caller: a +//! receipt keyed to HEAD cannot be written over a tree that is not HEAD, by any +//! route, including one nobody has written yet. `receipt clean` remains as the +//! cheap end, because `verify:gated` needs to fail in seconds rather than after +//! the gate set's full cost — and both call one `tree_state`, so the pair is one +//! authority asked at two moments rather than two readings that could disagree. +// +// carried: mise-tasks/tree-clean.sh crates/batten/src/receipt.rs kind:verb crates/batten/tests/it/receipt_clean.rs runs:batten+receipt+clean +// carried: tests/tree-clean.bats crates/batten/src/receipt.rs kind:verb crates/batten/tests/it/receipt_clean.rs runs:batten+receipt+clean +// +// carried: "a clean tree passes and names the commit the receipt would be about" crates/batten/src/receipt.rs +// carried: "a modified tracked file exits 1 and names the path" crates/batten/src/receipt.rs +// carried: "staged but uncommitted is dirty — the index is not HEAD" crates/batten/src/receipt.rs +// carried: "AN UNTRACKED FILE IS DIRTY — decided, not omitted" crates/batten/src/receipt.rs +// carried: "an ignored file is not dirty — scratch is excluded structurally" crates/batten/src/receipt.rs +// carried: "a deleted tracked file is dirty" crates/batten/src/receipt.rs +// carried: "the count is the number of paths, not a fixed string" crates/batten/src/receipt.rs +// changed: "output is a pointer — paths and a count, never the differing content" crates/batten/src/receipt.rs from `tests/tree-clean.bats`. Narrowed in the direction rule 4 points: the successor emits the COUNT and the head, and never the porcelain path list the retired program printed to stderr. A path list is already a pointer, so this is not a defect being fixed — it is a gate that no longer needs the reader to open anything, since the remedy is the same whichever paths are dirty. `the_refusal_carries_a_count_and_never_a_path` is the narrowed property +// carried: "the refusal names the fix, not merely the refusal" crates/batten/src/receipt.rs +// changed: "outside a git repository it exits 2 — could not look is not a verdict" crates/batten/src/receipt.rs from `tests/tree-clean.bats`. The ANSWER is carried and the CODE is not: the corpus reads `2` as could-not-look and this repository's one exit contract reads `2` as the policy verdict, with no per-verb exception (non-negotiable rule 5). So the successor answers `3`, which is what `Internal` means here. The inversion is the corpus-wide one `batten verdict` exists to fold, not a decision this port made +// changed: "a repository with no commit exits 2 — no HEAD for a receipt to name" crates/batten/src/receipt.rs from `tests/tree-clean.bats`. Same inversion as the row above, same reason: could-not-look is `3` on the engine's table +// changed: "THE ACCEPTANCE CASE: a dirty tree that would PASS still leaves HEAD unverified" crates/batten/src/receipt.rs from `tests/tree-clean.bats`. The property is STRONGER than the case asserted, which is why it is changed rather than carried. The case drove the retired gate and then checked that `verified` still refused — a composition of two tasks, true only while somebody kept calling the first. `a_dirty_tree_cannot_have_a_receipt_written_for_it_at_all` asserts it at the write instead: `receipt record` refuses, so there is no receipt for `verified` to match and no call site whose removal could restore the hole + +// Panicking on setup failure is the idiomatic way for a test to fail loudly. +#![allow(clippy::unwrap_used, clippy::expect_used)] + +use crate::common; + +use std::path::{Path, PathBuf}; +use std::process::Output; + +use common::{batten, stdout}; + +/// A repository with one commit, so HEAD resolves and a receipt could name it. +fn repo(name: &str) -> PathBuf { + let dir = common::scratch_outside_tree("batten-receipt-clean", name); + common::init_repo(&dir); + common::write(&dir, "batten.toml", "version = 1\n"); + common::git_in(&dir, &["add", "-A"]); + common::git_in(&dir, &["commit", "-qm", "seed"]); + // `receipt record` names the trunk its receipt was taken against, so the case + // that drives it needs one. `receipt clean` deliberately does NOT — it + // compares the tree to HEAD — and every other case here would pass without + // this line, which is the asymmetry the verb's own doc records. + common::pin_origin_main(&dir); + dir +} + +fn clean(dir: &Path) -> Output { + batten() + .arg("receipt") + .arg("clean") + .current_dir(dir) + .output() + .expect("run batten receipt clean") +} + +fn stderr(output: &Output) -> String { + String::from_utf8_lossy(&output.stderr).into_owned() +} + +#[test] +fn a_clean_tree_passes_and_names_the_commit_the_receipt_would_be_about() { + let dir = repo("clean"); + let output = clean(&dir); + assert_eq!(output.status.code(), Some(0), "{output:?}"); + assert!( + stdout(&output).contains("working tree matches HEAD"), + "{output:?}" + ); +} + +#[test] +fn a_modified_tracked_file_is_refused() { + let dir = repo("modified"); + common::write(&dir, "batten.toml", "version = 1\n# edited\n"); + let output = clean(&dir); + assert_eq!(output.status.code(), Some(1), "{output:?}"); + assert!(stderr(&output).contains("differs from HEAD"), "{output:?}"); +} + +#[test] +fn staged_but_uncommitted_is_dirty_because_the_index_is_not_head() { + let dir = repo("staged"); + common::write(&dir, "added.txt", "x\n"); + common::git_in(&dir, &["add", "-A"]); + assert_eq!(clean(&dir).status.code(), Some(1)); +} + +#[test] +fn an_untracked_file_is_dirty_decided_rather_than_omitted() { + // THE GAP A `diff HEAD` PREDICATE MISSES, and it is not theoretical: `cargo + // test` autodiscovers test targets and the bats suite globs `tests/*.bats`, + // so a brand-new untracked file is compiled and run by `verify` with ZERO + // tracked-file change. A receipt written after that attests a pass over bytes + // no commit contains, which is the whole failure. + let dir = repo("untracked"); + common::write(&dir, "scratch.rs", "fn main() {}\n"); + assert_eq!(clean(&dir).status.code(), Some(1)); +} + +#[test] +fn an_ignored_file_is_not_dirty_because_scratch_is_excluded_structurally() { + // Excluded by CONSTRUCTION rather than tuned out: `git status` never reports + // an ignored path, so `target/`, the worktree directories and + // `batten.local.toml` are outside the judgement rather than carved out of it. + let dir = repo("ignored"); + common::write(&dir, ".gitignore", "junk/\n"); + common::git_in(&dir, &["add", "-A"]); + common::git_in(&dir, &["commit", "-qm", "ignore"]); + std::fs::create_dir_all(dir.join("junk")).unwrap(); + common::write(&dir.join("junk"), "big.log", "noise\n"); + let output = clean(&dir); + assert_eq!(output.status.code(), Some(0), "{output:?}"); +} + +#[test] +fn a_deleted_tracked_file_is_dirty() { + let dir = repo("deleted"); + std::fs::remove_file(dir.join("batten.toml")).unwrap(); + assert_eq!(clean(&dir).status.code(), Some(1)); +} + +#[test] +fn the_count_is_the_number_of_paths_rather_than_a_fixed_string() { + let dir = repo("count"); + common::write(&dir, "one.txt", "1\n"); + common::write(&dir, "two.txt", "2\n"); + common::write(&dir, "three.txt", "3\n"); + let output = clean(&dir); + assert!(stderr(&output).contains("in 3 path(s)"), "{output:?}"); +} + +#[test] +fn the_refusal_carries_a_count_and_never_a_path() { + // Narrowed from the retired case in rule 4's direction: the successor emits + // the count and the head and never the porcelain list, because the remedy is + // the same whichever paths are dirty. + let dir = repo("pointer"); + common::write(&dir, "secret-name.txt", "content that must not be echoed\n"); + let output = clean(&dir); + let said = stderr(&output); + assert!(!said.contains("secret-name.txt"), "{said}"); + assert!(!said.contains("content that must not be echoed"), "{said}"); + assert!(said.contains("in 1 path(s)"), "{said}"); +} + +#[test] +fn the_refusal_names_the_fix_rather_than_merely_the_refusal() { + // All THREE routes, because "commit it" is not always the one the author + // wants and a refusal naming one route is how a gate earns a bypass. + let dir = repo("remedy"); + common::write(&dir, "wip.txt", "x\n"); + let said = stderr(&clean(&dir)); + assert!(said.contains("Commit the work"), "{said}"); + assert!(said.contains("stash it"), "{said}"); + assert!(said.contains("separate worktree"), "{said}"); +} + +#[test] +fn outside_a_repository_it_is_could_not_look_rather_than_a_verdict() { + // `3`, not the corpus's `2`. This repository's one exit contract reads `2` as + // the policy verdict with no per-verb exception, so a could-not-look answered + // there would tell every mediating harness that policy refused. + let dir = common::scratch_outside_tree("batten-receipt-clean", "no-repo"); + let output = clean(&dir); + assert_eq!(output.status.code(), Some(3), "{output:?}"); +} + +#[test] +fn a_repository_with_no_commit_is_could_not_look() { + let dir = common::scratch_outside_tree("batten-receipt-clean", "no-head"); + common::init_repo(&dir); + let output = clean(&dir); + assert_eq!(output.status.code(), Some(3), "{output:?}"); +} + +#[test] +fn a_dirty_tree_cannot_have_a_receipt_written_for_it_at_all() { + // THE ACCEPTANCE CASE, and it is stronger than the one it replaces. The + // retired case drove the gate and then checked that `verified` still refused + // — a composition true only while somebody kept calling the gate. This + // asserts it at the WRITE: `receipt record` refuses, so there is no receipt + // for `verified` to match, and no call site whose removal could reopen the + // hole. + let dir = repo("record-refuses"); + common::write(&dir, "mid-run-edit.rs", "fn main() {}\n"); + let output = batten() + .arg("receipt") + .arg("record") + .arg("verify") + .current_dir(&dir) + .output() + .expect("run batten receipt record"); + assert_ne!(output.status.code(), Some(0), "{output:?}"); + assert!( + stderr(&output).contains("attest bytes no commit contains"), + "{output:?}" + ); +} diff --git a/crates/batten/tests/it/report_only.rs b/crates/batten/tests/it/report_only.rs new file mode 100644 index 000000000..717b5b077 --- /dev/null +++ b/crates/batten/tests/it/report_only.rs @@ -0,0 +1,199 @@ +//! A report task stays off the landing path, over the compiled binary +//! (CLOUD-582, ported from `mise-tasks/report-only-check.sh` under CLOUD-843). +//! +//! **What is decidable only here.** `policy/report-only.rego` carries load-time +//! cases pinning the predicate, and each fabricates `input.tree.documents` — so +//! it is green whether or not the engine PARSES `mise.toml` and the workflows at +//! all. The whole port turns on reading parsed structure where the predecessor +//! carved a span out of text, and a fabricated document asserts that structure +//! into existence. +// +// carried: mise-tasks/report-only-check.sh policy/report-only.rego crates/batten/tests/it/report_only.rs +// carried: tests/report-only-check.bats policy/report-only.rego crates/batten/tests/it/report_only.rs +// +// carried: "the repo's real manifest and workflows are clean today" policy/report-only.rego +// carried: "a report named in [tasks.verify] is refused" policy/report-only.rego +// carried: "a report run by a pull_request workflow is refused" policy/report-only.rego +// carried: "a report run by a SCHEDULED workflow is the point, not a violation" policy/report-only.rego +// carried: "a longer identifier merely containing the name does not fire" policy/report-only.rego +// carried: "both routes are reported together, not one at a time" policy/report-only.rego +// carried: "a manifest with no [tasks.verify] cannot be judged, and says so" policy/report-only.rego +// +// changed: "the report's output path in verify's body does fire" policy/report-only.rego the shell matched the bare NAME anywhere in verify's span, so `COVERAGE_OUT_DIR` in the body was a hit it had to word-bound away while a genuine mention still fired. The successor asks the invocation question instead — `mise run ` — because running it is what makes a report a gate, and an output path that merely names it does not. Strictly narrower and strictly more accurate; `a_longer_identifier_merely_containing_the_name_does_not_fire` is the half that survives +// changed: "a missing manifest is exit 2, never a pass" policy/report-only.rego the shell took the manifest as `$REPORT_ONLY_MANIFEST` and could be pointed at a path that does not exist. The successor's subject is a declared `sources` entry: a glob matching nothing means the rule is not evaluated, and `input.tree.missing` is never populated on the tree surface (CLOUD-1049). The neighbouring could-not-look — a manifest that parses with no `[tasks.verify]` — IS reachable and is carried above + +// Panicking on setup failure is the idiomatic way for a test to fail loudly. +#![allow(clippy::unwrap_used, clippy::expect_used)] + +use crate::common; + +use std::path::{Path, PathBuf}; +use std::process::Output; + +use common::{Fixture, git_in, run, stdout}; + +fn report_repo(name: &str, manifest: &str, workflow: Option<&str>) -> PathBuf { + let mut fixture = Fixture::new(name) + .config( + "version = 1\n\n\ + [[verdict]]\n\ + id = \"task judge silent\"\n\ + gloss = \"a task runs somewhere its verdict is not wanted\"\n\ + class = \"A report emits a number for a human; binding it to landing makes a gate of a tool's opinion.\"\n\n\ + [[verdict.route]]\n\ + id = \"prose read first\"\n\ + kind = \"document\"\n\ + target = \"AGENTS.md\"\n\n\ + [[verdict]]\n\ + id = \"task guard missing\"\n\ + gloss = \"the task a rule judges against is not declared\"\n\ + class = \"With no verify task there is nothing to judge a report against, and clean would be a false green.\"\n\n\ + [[verdict.route]]\n\ + id = \"prose read first\"\n\ + kind = \"document\"\n\ + target = \"AGENTS.md\"\n\n\ + [[rule]]\n\ + id = \"report-only\"\n\ + kind = \"policy\"\n\ + scope = \"tree\"\n\ + sources = [\"mise.toml\", \".github/workflows/*.yml\"]\n\ + module = \"policy/report-only.rego\"\n\ + severity = \"deny\"\n", + ) + .file("AGENTS.md", "the consumer's own authority\n") + .file("mise.toml", manifest); + if let Some(body) = workflow { + fixture = fixture.file(".github/workflows/report.yml", body); + } + let dir = fixture.git().build(); + common::write( + &dir, + "policy/report-only.rego", + &std::fs::read_to_string(common::at_root("policy/report-only.rego")).unwrap(), + ); + git_in(&dir, &["add", "-A"]); + git_in(&dir, &["commit", "-q", "-m", "base"]); + dir +} + +fn check(dir: &Path) -> Output { + run(dir, &["check", "--rule", "report-only"]) +} + +const CLEAN: &str = "[tasks.verify]\ndepends = [\"ci\"]\nrun = \"echo ok\"\n"; + +#[test] +fn a_clean_manifest_and_workflow_pass() { + let dir = report_repo( + "report-clean", + CLEAN, + Some("on:\n pull_request:\njobs:\n j:\n steps:\n - run: mise run ci\n"), + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(0), "{}", stdout(&output)); +} + +#[test] +fn a_report_in_verifys_depends_is_refused_over_the_binary() { + let dir = report_repo( + "report-in-depends", + "[tasks.verify]\ndepends = [\"ci\", \"coverage\"]\nrun = \"echo ok\"\n", + None, + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); + assert!( + stdout(&output).contains("mise.toml"), + "the finding points at the manifest: {}", + stdout(&output) + ); +} + +#[test] +fn a_report_run_by_a_pull_request_workflow_is_refused() { + let dir = report_repo( + "report-on-pr", + CLEAN, + Some("on:\n pull_request:\njobs:\n j:\n steps:\n - run: mise run scorecard\n"), + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); + assert!( + stdout(&output).contains(".github/workflows/report.yml"), + "and at the workflow, not the manifest: {}", + stdout(&output) + ); +} + +#[test] +fn a_report_run_by_a_scheduled_workflow_is_the_point_not_a_violation() { + // DISCRIMINATING. A gate that judged every trigger would refuse the reason + // the report exists at all. + let dir = report_repo( + "report-on-schedule", + CLEAN, + Some( + "on:\n schedule:\n - cron: \"0 0 * * 0\"\njobs:\n j:\n steps:\n - run: mise run scorecard\n", + ), + ); + let output = check(&dir); + assert_eq!( + output.status.code(), + Some(0), + "a scheduled report was refused: {}", + stdout(&output) + ); +} + +#[test] +fn a_longer_identifier_merely_containing_the_name_does_not_fire() { + // The boundary the predecessor word-bounded a grep for. A parsed `depends` + // is a list of names, so this is a different entry rather than a near miss. + let dir = report_repo( + "report-longer-identifier", + "[tasks.verify]\ndepends = [\"coverage-report-check\"]\nrun = \"echo $COVERAGE_OUT_DIR\"\n", + None, + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(0), "{}", stdout(&output)); +} + +#[test] +fn both_routes_are_reported_together_not_one_at_a_time() { + let dir = report_repo( + "report-both-routes", + "[tasks.verify]\ndepends = [\"ci\", \"coverage\"]\nrun = \"echo ok\"\n", + Some("on:\n pull_request:\njobs:\n j:\n steps:\n - run: mise run scorecard\n"), + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2)); + let text = stdout(&output); + assert!( + text.contains("mise.toml") && text.contains(".github/workflows/report.yml"), + "a reader fixing one must see the other in the same run: {text:?}" + ); +} + +#[test] +fn a_manifest_with_no_verify_task_cannot_be_judged_and_says_so() { + let dir = report_repo("report-no-verify", "[tasks.ci]\nrun = \"echo ok\"\n", None); + let output = check(&dir); + assert_eq!( + output.status.code(), + Some(2), + "no verify task is could-not-look, never a clean board: {}", + stdout(&output) + ); +} + +#[test] +fn the_repos_real_manifest_and_workflows_are_clean_today() { + let output = + common::run_at_real_root(&common::at_root(""), &["check", "--rule", "report-only"]); + assert_eq!( + output.status.code(), + Some(0), + "a report reached the landing path in this tree: {}", + stdout(&output) + ); +} diff --git a/crates/batten/tests/it/rust_paths_check.rs b/crates/batten/tests/it/rust_paths_check.rs new file mode 100644 index 000000000..8e12ac9c9 --- /dev/null +++ b/crates/batten/tests/it/rust_paths_check.rs @@ -0,0 +1,240 @@ +//! The Rust workflow's `paths:` filter honours its own claims, over the compiled +//! binary (CLOUD-398, ported from `mise-tasks/rust-paths-check.sh` under +//! CLOUD-843). +//! +//! **What is decidable only here.** `policy/rust-paths-check.rego` carries +//! load-time cases pinning the matcher and the two probe sets, and every one of +//! them supplies the workflow's lines with `with input as`. That fabricates the +//! very shape the engine may be unable to produce (CLOUD-845), and it matters more +//! than usual for this rule: its whole subject is a filter that decides whether a +//! JOB RUNS, and its dangerous failure is silent. A module whose suite only +//! fabricated the file would stay green over an engine that resolved the path to +//! nothing — which is the state where the rule is not evaluated at all, and where +//! a narrowed filter would sail through exactly as it did before the gate existed. +//! +//! The self-consumption case is the one the retiring suite opened on: the +//! committed filter honours every probe, checked rather than asserted. +// +// carried: mise-tasks/rust-paths-check.sh policy/rust-paths-check.rego crates/batten/tests/it/rust_paths_check.rs +// carried: tests/rust-paths-check.bats policy/rust-paths-check.rego crates/batten/tests/it/rust_paths_check.rs +// +// carried: "the committed rust.yml filter honours every probe" policy/rust-paths-check.rego +// carried: "the full filter selects every declared input and no docs path" policy/rust-paths-check.rego +// carried: "a directory glob selects a file below it" policy/rust-paths-check.rego +// carried: "dropping mise.toml is refused, because the jobs run mise tasks" policy/rust-paths-check.rego +// carried: "dropping Cargo.lock is refused" policy/rust-paths-check.rego +// carried: "a whole-repository glob is refused" policy/rust-paths-check.rego +// carried: "selecting the memories tree is refused" policy/rust-paths-check.rego +// carried: "a workflow with no paths filter is exit 2, not a pass" policy/rust-paths-check.rego +// carried: "a pattern the matcher cannot decide is exit 2, not a guess" policy/rust-paths-check.rego +// changed: "a missing workflow is exit 2, not a pass" policy/rust-paths-check.rego the shell took the workflow as an argument and refused a path it could not open; the successor declares it as a `line_sources` path and the ENGINE decides this earlier — a rule whose declared path matches nothing is not evaluated at all, and `input.tree.missing` is never populated on the tree surface (CLOUD-1049, measured identically for `policy/mise-pin-agreement.rego`'s own could-not-look clause). What the case protected survives as the present-but-filterless arm: a workflow that resolves and declares no `paths:` block is still a refusal, never a pass + +// Panicking on setup failure is the idiomatic way for a test to fail loudly. +#![allow(clippy::unwrap_used, clippy::expect_used)] + +use crate::common; + +use std::fmt::Write as _; +use std::path::{Path, PathBuf}; +use std::process::Output; + +use common::{Fixture, git_in, run, stderr, stdout}; + +/// A repository declaring only this rule, so any finding is the one under test. +fn paths_repo(name: &str, entries: &[&str]) -> PathBuf { + let mut listed = String::new(); + for entry in entries { + writeln!(listed, " - \"{entry}\"").unwrap(); + } + let workflow = format!( + "on:\n pull_request:\n paths:\n{listed}jobs:\n build:\n runs-on: ubuntu-latest\n" + ); + workflow_repo(name, &workflow) +} + +fn workflow_repo(name: &str, workflow: &str) -> PathBuf { + let dir = Fixture::new(name) + .config( + "version = 1\n\n\ + [[verdict]]\n\ + id = \"input select missing\"\n\ + gloss = \"the filter does not select an input its jobs read\"\n\ + class = \"A job left absent is accepted by design, so the regression lands green.\"\n\n\ + [[verdict.route]]\n\ + id = \"prose read first\"\n\ + kind = \"document\"\n\ + target = \"AGENTS.md\"\n\n\ + [[verdict]]\n\ + id = \"input select loose\"\n\ + gloss = \"the filter selects a diff its jobs cannot be affected by\"\n\ + class = \"Every widening is invisible except as a bill.\"\n\n\ + [[verdict.route]]\n\ + id = \"prose read first\"\n\ + kind = \"document\"\n\ + target = \"AGENTS.md\"\n\n\ + [[verdict]]\n\ + id = \"workflow read unclear\"\n\ + gloss = \"the filter could not be read, or carries a shape this cannot decide\"\n\ + class = \"A guessed selection here is the silent false-absent the gate exists to stop.\"\n\n\ + [[verdict.route]]\n\ + id = \"prose read first\"\n\ + kind = \"document\"\n\ + target = \"AGENTS.md\"\n\n\ + [[rule]]\n\ + id = \"rust-paths-check\"\n\ + kind = \"policy\"\n\ + scope = \"tree\"\n\ + line_sources = [\".github/workflows/rust.yml\"]\n\ + module = \"policy/rust-paths-check.rego\"\n\ + severity = \"deny\"\n", + ) + .file("AGENTS.md", "the consumer's own authority\n") + .file(".github/workflows/rust.yml", workflow) + .git() + .build(); + common::write( + &dir, + "policy/rust-paths-check.rego", + &std::fs::read_to_string(common::at_root("policy/rust-paths-check.rego")).unwrap(), + ); + git_in(&dir, &["add", "-A"]); + git_in(&dir, &["commit", "-q", "-m", "base"]); + dir +} + +fn check(dir: &Path) -> Output { + run(dir, &["check", "--rule", "rust-paths-check"]) +} + +/// Every declared input, and nothing a docs-only diff touches. +const HONEST: &[&str] = &[ + "crates/**", + "Cargo.toml", + "Cargo.lock", + "rust-toolchain.toml", + "deny.toml", + "mise.toml", + "mise.lock", + ".github/workflows/rust.yml", +]; + +#[test] +fn the_full_filter_selects_every_input_and_no_docs_path() { + let dir = paths_repo("paths-honest", HONEST); + let output = check(&dir); + assert_eq!( + output.status.code(), + Some(0), + "out={} err={}", + stdout(&output), + stderr(&output) + ); +} + +#[test] +fn a_directory_glob_selects_a_file_below_it() { + // `crates/**` standing for `crates/batten/src/lib.rs` is the reason the + // filter can stay short at all. + let dir = paths_repo("paths-directory-glob", HONEST); + let output = check(&dir); + assert_eq!(output.status.code(), Some(0), "{}", stdout(&output)); +} + +#[test] +fn a_dropped_input_is_refused() { + // THE SILENT DIRECTION: the jobs run tasks the manifest defines, so a filter + // written from "the Rust tree" misses it and the jobs go absent. + let without_manifest: Vec<&str> = HONEST + .iter() + .copied() + .filter(|e| *e != "mise.toml") + .collect(); + let dir = paths_repo("paths-dropped-manifest", &without_manifest); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); + assert!( + stdout(&output).contains("mise.toml"), + "the finding names the unselected probe: {:?}", + stdout(&output) + ); +} + +#[test] +fn a_dropped_lockfile_is_refused_too() { + let without_lock: Vec<&str> = HONEST + .iter() + .copied() + .filter(|e| *e != "Cargo.lock") + .collect(); + let dir = paths_repo("paths-dropped-lock", &without_lock); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); +} + +#[test] +fn a_whole_repository_glob_is_refused() { + // A bare `**` strips to an EMPTY prefix, which a "did the strip change + // anything" test reads as matching nothing — so a filter selecting the whole + // repository would pass as narrow. + let dir = paths_repo("paths-whole-repo", &["**"]); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); +} + +#[test] +fn selecting_the_memories_tree_is_refused() { + let mut wide: Vec<&str> = HONEST.to_vec(); + wide.push(".serena/**"); + let dir = paths_repo("paths-memories", &wide); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); +} + +#[test] +fn a_workflow_with_no_paths_filter_is_not_a_pass() { + let dir = workflow_repo( + "paths-no-filter", + "on:\n pull_request:\njobs:\n build:\n runs-on: ubuntu-latest\n", + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); +} + +#[test] +fn a_pattern_the_matcher_cannot_decide_is_not_a_guess() { + let mut with_negation: Vec<&str> = HONEST.to_vec(); + with_negation.push("!docs/**"); + let dir = paths_repo("paths-undecidable", &with_negation); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); +} + +#[test] +fn output_is_pointer_only() { + let dir = workflow_repo( + "paths-pointer-only", + "on:\n pull_request:\n paths:\n - \"**\"\njobs:\n build:\n runs-on: a-distinctive-runner\n", + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2)); + assert!( + !stdout(&output).contains("a-distinctive-runner"), + "the workflow's content is payload: {:?}", + stdout(&output) + ); +} + +#[test] +fn the_committed_filter_honours_every_probe() { + // The self-consumption case the retiring suite opened on. + let output = common::run_at_real_root( + &common::at_root(""), + &["check", "--rule", "rust-paths-check"], + ); + assert_eq!( + output.status.code(), + Some(0), + "the committed paths filter does not honour its own claims: {}", + stdout(&output) + ); +} diff --git a/crates/batten/tests/it/session_provisioning.rs b/crates/batten/tests/it/session_provisioning.rs index d98618c36..fdf27011b 100644 --- a/crates/batten/tests/it/session_provisioning.rs +++ b/crates/batten/tests/it/session_provisioning.rs @@ -96,7 +96,7 @@ //! // changed: "the hook runs green on this checkout" crates/batten/tests/it/session_provisioning.rs the exit-0-and-silent half survives as `a_step_that_passes_says_nothing`, against stubs; the END-TO-END half is withdrawn, because dispatching the real rows provisions a container inside `test:cargo` — 141s measured cold — which is the cost CLOUD-1268 exists to stop moving between lanes. What covers it instead is the session itself: a failed step reports on the advisory channel at the moment it fails // changed: "running the hook leaves the tracked lockfile untouched" crates/batten/tests/it/session_provisioning.rs narrowed from the EFFECT to the DECLARATION: `the_install_step_is_declared_lockfile_free` asserts `session:install` carries MISE_LOCKFILE=false, where the retired case ran the hook and diffed `git status -- mise.lock`. `[settings] lockfile = false` in mise.toml is the standing authority and `lock-complete` the standing gate; what is lost is the observation that this particular path honours it -// changed: "the session-start hook calls it — the whole point is WHEN it runs" crates/batten/tests/it/session_provisioning.rs from `tests/container-preflight.bats`, whose own subject survives. The case grepped the retired script for `container-preflight`; the property — that a preflight nothing runs at startup is worthless — is now the `session-container-preflight` row, and its POSITION is asserted too, which the grep could not say +// changed: "the session-start hook calls it — the whole point is WHEN it runs" crates/batten/tests/it/session_provisioning.rs from `tests/container-preflight.bats`. The case grepped the retired script for `container-preflight`; the property — that a preflight nothing runs at startup is worthless — is now a declared row, and its POSITION is asserted too, which the grep could not say. CLOUD-1753 retired the subject as well and split that row into `session-egress` and `session-credential`, so the property is now carried by both and the position clause covers their order // changed: "the hook passes --degraded when provisioning failed" batten.toml the capability is gone rather than moved, and this is the one real loss in this retirement. `--degraded` told the preflight not to trust toolchain-dependent probes when an earlier step had failed, and it worked because the script carried a `fail` variable across its steps. Handlers share no state — each is its own process with its own outcome — so nothing can compute the flag. The consequence is bounded: a container whose install failed now gets the full probe set, so it may report a second symptom of one cause, and both refusals arrive in the same reply. Recovering it needs a fact the door does not carry; filed rather than papered over // changed: "the fixer is wired: session-start runs it, so a clone is compliant before it commits" crates/batten/tests/it/session_provisioning.rs from `tests/commit-attribution.bats`, whose own subject (hk.pkl, mise.toml) survives. The case grepped the retired script for its `step attribution-identity` line; the property — that the identity fixer runs before a clone commits — is now the `session-attribution-identity` row, asserted by `the_committed_provisioning_declares_every_step_in_order`. It is CHANGED rather than CARRIED because the retired case pinned the invocation's exact spelling inside a program and this pins a row's presence and position in a list // changed: "the git hooks are installed — the per-clone step that was absent" mise-tasks/doctor.sh narrowed from perform-and-assert in one case to assert-only: `doctor` decides that same state on every later run, which the retired case's own comment already named as its backstop, and `tests/git-hook.bats` owns what the installed body does. What is lost is the pairing of the step with its effect inside one case @@ -125,7 +125,7 @@ use common::{at_root, git_in, scratch, stderr, stdout, write}; /// /// A LIST RATHER THAN A COUNT, because a count cannot tell an added row from a /// renamed one, and the ordering claim below needs the names anyway. -const DECLARED: [&str; 11] = [ +const DECLARED: [&str; 12] = [ "session-stamp", "session-install", "session-submodules", @@ -146,7 +146,13 @@ const DECLARED: [&str; 11] = [ // being declared here, which is the property the header claims and this is // the instance of it. "session-wiring", - "session-container-preflight", + // CLOUD-1753 split `session-container-preflight` into the two rows it always + // was, and this list is where the split has to be stated. The order is the + // claim, as it is for `session-wiring` above: egress before credential, + // because a container that cannot reach the API fails both and the reader + // wants the cause ahead of the symptom. + "session-egress", + "session-credential", "session-census", ]; diff --git a/crates/batten/tests/it/snapshots/it__snapshots__golden_json_schema.snap b/crates/batten/tests/it/snapshots/it__snapshots__golden_json_schema.snap index d085ee2b1..c67d516d0 100644 --- a/crates/batten/tests/it/snapshots/it__snapshots__golden_json_schema.snap +++ b/crates/batten/tests/it/snapshots/it__snapshots__golden_json_schema.snap @@ -1035,6 +1035,38 @@ expression: stdout_of(&output) "positional": false, "help": "How many of a `:::` bundle's commands run at once" }, + { + "name": "lock", + "short": null, + "long": "lock", + "takes_value": true, + "positional": false, + "help": "Hold this clone's named singleton lock for the child's lifetime" + }, + { + "name": "lock_attempts", + "short": null, + "long": "lock-attempts", + "takes_value": true, + "positional": false, + "help": "How many times to ask for the lock before reporting it held" + }, + { + "name": "lock_label", + "short": null, + "long": "lock-label", + "takes_value": true, + "positional": false, + "help": "What the wait is for, named by the caller for the refusal line" + }, + { + "name": "lock_path", + "short": null, + "long": "lock-path", + "takes_value": true, + "positional": false, + "help": "Hold the lock at this path, for a resource the clone does not own" + }, { "name": "style", "short": null, @@ -2937,6 +2969,32 @@ expression: stdout_of(&output) } ] }, + { + "path": "verdict", + "id": "verdict", + "about": "Fold a run's findings and blind spots into this tool's exit code", + "effect": "read", + "data_channel": false, + "flags": [ + { + "name": "findings", + "short": null, + "long": "findings", + "takes_value": true, + "positional": false, + "help": "How many blocking findings the run produced" + }, + { + "name": "unjudgeable", + "short": null, + "long": "unjudgeable", + "takes_value": true, + "positional": false, + "help": "How many subjects the run could not read" + } + ], + "subcommands": [] + }, { "path": "wiring", "id": "wiring", @@ -3239,6 +3297,10 @@ expression: stdout_of(&output) "id": "task.read", "path": "task read" }, + { + "id": "verdict", + "path": "verdict" + }, { "id": "worktree.status", "path": "worktree status" diff --git a/crates/batten/tests/it/stop_posture.rs b/crates/batten/tests/it/stop_posture.rs index c3aeb65b9..40db6005e 100644 --- a/crates/batten/tests/it/stop_posture.rs +++ b/crates/batten/tests/it/stop_posture.rs @@ -22,6 +22,35 @@ //! The suite's successor is the module, which is where the one rule that COULD //! be a predicate went. //! +// +// CLOUD-1753. The program was ALREADY PORTED: `policy/stop-posture.rego` and +// `[[pattern]] hedged-flag-framing` shipped under CLOUD-1051, and this file is +// the tier that drives them over the compiled binary. So the retirement is a +// deletion against an existing successor rather than a second port — which is +// what the row asks for, and the case titles below map one-for-one onto the +// module's own predicates. +// carried: mise-tasks/stop-posture-check.sh policy/stop-posture.rego crates/batten/tests/it/stop_posture.rs +// carried: tests/stop-posture-check.bats policy/stop-posture.rego crates/batten/tests/it/stop_posture.rs +// +// carried: "stop-posture-check.bats::the first tell AGENTS.md names fires" policy/stop-posture.rego +// carried: "stop-posture-check.bats::the second tell AGENTS.md names fires" policy/stop-posture.rego +// carried: "stop-posture-check.bats::the inflection that a closed two-item list would have missed fires" policy/stop-posture.rego +// carried: "stop-posture-check.bats::the report carries a count" policy/stop-posture.rego +// carried: "stop-posture-check.bats::THE WITNESSED MISS: the CLOUD-347 sentence fires" policy/stop-posture.rego +// carried: "stop-posture-check.bats::the asymmetry is gone: mentioning is a flagging verb under BOTH openers" policy/stop-posture.rego +// carried: "stop-posture-check.bats::every opener carries the witnessed verb" policy/stop-posture.rego +// carried: "stop-posture-check.bats::an UNWITNESSED near-miss stays out — that is the line against inventing a list" policy/stop-posture.rego +// carried: "stop-posture-check.bats::the report never echoes the sentence it judged" policy/stop-posture.rego +// carried: "stop-posture-check.bats::a code span carrying the tell does not fire" policy/stop-posture.rego +// carried: "stop-posture-check.bats::a double-quoted span carrying the tell does not fire" policy/stop-posture.rego +// carried: "stop-posture-check.bats::a block quote carrying the tell does not fire" policy/stop-posture.rego +// carried: "stop-posture-check.bats::a fenced block carrying the tell does not fire" policy/stop-posture.rego +// carried: "stop-posture-check.bats::a LINE-WRAPPED quoted span carrying the tell does not fire" policy/stop-posture.rego +// carried: "stop-posture-check.bats::reporting a measured value is not hedged framing" policy/stop-posture.rego +// carried: "stop-posture-check.bats::a command flag is not hedged framing" policy/stop-posture.rego +// carried: "stop-posture-check.bats::a plainly stated finding with a durable home does not fire" policy/stop-posture.rego +// carried: "stop-posture-check.bats::empty stdin is clean rather than an error" policy/stop-posture.rego +// carried: "stop-posture-check.bats::a clean message produces no output at all" policy/stop-posture.rego // carried: mise-tasks/stop-guard.sh crates/batten/src/lib.rs kind:mechanism crates/batten/tests/it/stop_posture.rs // // CLOUD-1163's unlanded unit. The program was spawned by `stop_nudges` with diff --git a/crates/batten/tests/it/suite_cost_corpus.rs b/crates/batten/tests/it/suite_cost_corpus.rs new file mode 100644 index 000000000..3e85ac579 --- /dev/null +++ b/crates/batten/tests/it/suite_cost_corpus.rs @@ -0,0 +1,368 @@ +//! `suite-cost-corpus` and `batten record suites` over the compiled binary — +//! CLOUD-352, ported off `mise-tasks/suite-bench.sh` and +//! `mise-tasks/suite-bench-check.sh` under CLOUD-1753. +//! +//! # The pair, and why the split is the design +//! +//! The producer derives what each suite costs from the report the runner already +//! wrote; the gate decides MEMBERSHIP over the corpus it writes. That division is +//! the whole point and the retired programs argued it in their own words: wall +//! clock is a CLOCK — the same suite varies with load, and a gate demanding byte +//! equality on a duration would be red on every second run and switched off +//! within a day. Membership is deterministic and costs nothing, and it is what +//! rots: a suite added and never recorded is invisible to everything that reads +//! the corpus, and a row naming a deleted suite is a figure attached to nothing. +//! +//! # WHAT THIS TIER REACHES THAT THE MODULE'S OWN CASES CANNOT +//! +//! Two things, and the first was measured the hard way while writing this port. +//! +//! **That the ENGINE resolves the declared `lines` path to the committed bytes, +//! and `input.tree.tracked` to the index.** A `with input as` fixture fabricates +//! exactly the shape the boundary may be unable to produce (CLOUD-845) — and +//! here the fixture was wrong for several runs: it keyed its `lines` object with +//! a rule reference, so `corpus_lines` resolved against nothing. **Every "clean" +//! case still passed**, because a fixture that resolves to nothing produces no +//! findings and reads exactly like a clean tree. That is the load-time tier +//! failing in the direction this tier exists to cover. +//! +//! **That the producer and the gate agree on the corpus.** They are two halves of +//! one contract — `suites::CORPUS` and the rule's `lines` row — and nothing in +//! either half can see the other. A port that let them pick their own locations +//! would have disconnected them silently, which is the same class of failure one +//! level up. +// +// carried: mise-tasks/suite-bench.sh crates/batten/src/suites.rs kind:verb crates/batten/tests/it/suite_cost_corpus.rs runs:batten+record+suites +// carried: mise-tasks/suite-bench-check.sh policy/suite-cost-corpus.rego crates/batten/tests/it/suite_cost_corpus.rs +// carried: tests/suite-bench-check.bats policy/suite-cost-corpus.rego crates/batten/tests/it/suite_cost_corpus.rs +// +// carried: "a corpus naming exactly the tracked suites passes" policy/suite-cost-corpus.rego +// carried: "a suite absent from the corpus is refused" policy/suite-cost-corpus.rego +// carried: "a corpus row naming no real suite is refused" policy/suite-cost-corpus.rego +// carried: "an untracked suite is not demanded — the corpus describes the tree git carries" policy/suite-cost-corpus.rego +// carried: "a missing corpus is could-not-look, not a clean tree" policy/suite-cost-corpus.rego +// carried: "a corpus with no rows is could-not-look, not an empty suite set" policy/suite-cost-corpus.rego +// carried: "no tracked suite at all is could-not-look" policy/suite-cost-corpus.rego +// carried: "output is a pointer — no duration is echoed" policy/suite-cost-corpus.rego +// carried: "a report naming a suite the tree does not track is could-not-look" crates/batten/src/suites.rs kind:verb crates/batten/tests/it/suite_cost_corpus.rs runs:batten+record+suites +// carried: "a report naming only tracked suites derives a corpus" crates/batten/src/suites.rs kind:verb crates/batten/tests/it/suite_cost_corpus.rs runs:batten+record+suites + +// Panicking on setup failure is the idiomatic way for a test to fail loudly. +#![allow(clippy::unwrap_used, clippy::expect_used)] + +use crate::common; + +use std::path::{Path, PathBuf}; +use std::process::Output; + +use common::batten; + +/// A repository carrying suites, a report, and optionally a corpus. +fn bench(name: &str, suites: &[&str], corpus: Option<&str>) -> PathBuf { + let dir = common::scratch_outside_tree("batten-suite-corpus", name); + common::init_repo(&dir); + common::write(&dir, "batten.toml", &config()); + std::fs::create_dir_all(dir.join("tests")).unwrap(); + for suite in suites { + common::write(&dir.join("tests"), suite, "#!/usr/bin/env bats\n"); + } + if let Some(text) = corpus { + std::fs::create_dir_all(dir.join("bench").join("suites")).unwrap(); + common::write(&dir.join("bench").join("suites"), "RESULTS.md", text); + } + common::git_in(&dir, &["add", "-A"]); + common::git_in(&dir, &["commit", "-qm", "seed"]); + dir +} + +/// The one rule under test, registered against the module in this repository. +/// +/// **The four verdicts are declared here too, and that is the ABI rather than +/// boilerplate.** A module raising a verdict no `[[verdict]]` row declares is +/// refused at load — the refusal would carry no gloss, no class and no route, +/// which is the bare no the vocabulary exists to make unwritable. So a fixture +/// registering the module and not its verdicts does not test the module, it +/// tests the loader; this suite met that on its first run. +fn config() -> String { + let module = common::at_root("policy/suite-cost-corpus.rego") + .to_string_lossy() + .into_owned(); + let mut out = format!( + "version = 1\n\n[[rule]]\nid = \"suite-cost-corpus\"\nkind = \"policy\"\nscope = \"tree\"\n\ + lines = [\"bench/suites/RESULTS.md\"]\nmodule = {module:?}\nseverity = \"deny\"\n" + ); + for (id, gloss) in [ + ( + "suite count unread", + "the cost corpus is absent or carries no row", + ), + ("suite list empty", "no tracked bats suite exists"), + ( + "suite count absent", + "a tracked suite has no row in the cost corpus", + ), + ( + "suite count dead", + "the cost corpus records a suite this tree does not track", + ), + ] { + out.push_str(&format!( + "\n[[verdict]]\nid = {id:?}\ngloss = {gloss:?}\nclass = \"\"\"\n{gloss}\n\"\"\"\n\n\ + [[verdict.route]]\nid = \"suite measure now\"\nkind = \"command\"\n\ + target = \"batten record suites --write\"\n" + )); + } + out +} + +/// A corpus body in the shape the producer emits. +fn corpus(rows: &[(&str, &str)]) -> String { + let mut out = String::from( + "# Per-suite cost of `test:bats`\n\n| seconds | share | suite |\n| ---: | ---: | --- |\n", + ); + for (seconds, suite) in rows { + out.push_str(&format!("| {seconds} | 50.0% | `{suite}` |\n")); + } + out +} + +fn check(dir: &Path) -> Output { + batten() + .arg("check") + .arg("--rule") + .arg("suite-cost-corpus") + .current_dir(dir) + .env_remove("BATTEN_STRICTNESS") + .output() + .expect("run batten check") +} + +fn said(output: &Output) -> String { + format!( + "{}{}", + String::from_utf8_lossy(&output.stdout), + String::from_utf8_lossy(&output.stderr) + ) +} + +#[test] +fn a_corpus_covering_every_tracked_suite_passes() { + let dir = bench( + "clean", + &["a.bats", "b.bats"], + Some(&corpus(&[("4.5", "tests/a.bats"), ("0.5", "tests/b.bats")])), + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(0), "{}", said(&output)); +} + +#[test] +fn a_tracked_suite_missing_from_the_corpus_is_refused() { + let dir = bench( + "unrecorded", + &["a.bats", "unrecorded.bats"], + Some(&corpus(&[("4.5", "tests/a.bats")])), + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", said(&output)); + // POINTER, NEVER PAYLOAD, and the pointer is what tells the two directions + // apart: `check` renders ` `, so the SUBJECT is the assertion. + // The verdict tokens are pinned by the module's own load-time cases; what + // this tier owns is that the engine reached the right one. + assert!( + said(&output).contains("tests/unrecorded.bats"), + "{}", + said(&output) + ); + assert!(!said(&output).contains("4.5"), "{}", said(&output)); +} + +#[test] +fn a_corpus_row_naming_no_tracked_suite_is_refused() { + let dir = bench( + "phantom", + &["a.bats"], + Some(&corpus(&[ + ("4.5", "tests/a.bats"), + ("1.0", "tests/retired.bats"), + ])), + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", said(&output)); + // The pointer is the corpus ROW, where the missing-suite direction points at + // the tracked path — which is how a reader tells them apart at a glance. + assert!( + said(&output).contains("tests/retired.bats"), + "{}", + said(&output) + ); +} + +#[test] +fn both_directions_are_reported_in_one_run() { + // A corpus can rot in both directions at once, and reporting only the first + // makes the second a second trip. + let dir = bench( + "both", + &["a.bats", "unrecorded.bats"], + Some(&corpus(&[ + ("4.5", "tests/a.bats"), + ("1.0", "tests/retired.bats"), + ])), + ); + let text = said(&check(&dir)); + assert!(text.contains("tests/unrecorded.bats"), "{text}"); + assert!(text.contains("tests/retired.bats"), "{text}"); +} + +#[test] +fn a_missing_corpus_is_could_not_look_rather_than_a_clean_corpus() { + // THE ARM THAT MATTERS MOST, and the one a fixture cannot honestly reach: + // the engine still acquires a declared literal `lines` path that no file + // satisfies, so the module evaluates and reports rather than being SKIPPED. + let dir = bench("no-corpus", &["a.bats"], None); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", said(&output)); + assert!( + said(&output).contains("bench/suites/RESULTS.md"), + "{}", + said(&output) + ); +} + +#[test] +fn an_empty_corpus_is_could_not_look_rather_than_a_clean_corpus() { + let dir = bench("empty-corpus", &["a.bats"], Some(&corpus(&[]))); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", said(&output)); + assert!( + said(&output).contains("bench/suites/RESULTS.md"), + "{}", + said(&output) + ); +} + +#[test] +fn no_tracked_suites_is_could_not_look_rather_than_a_clean_corpus() { + // The corpus would have no subject. Reporting green over that is the + // collapse this repository keeps meeting in new disguises. + let dir = bench( + "no-suites", + &[], + Some(&corpus(&[("1.0", "tests/gone.bats")])), + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", said(&output)); + assert!( + said(&output).contains("suite-cost-corpus"), + "{}", + said(&output) + ); +} + +#[test] +fn the_formatters_padding_does_not_defeat_the_row_match() { + // MEASURED ON THE RETIRED PROGRAM: `prettier` owns Markdown here and aligns + // table columns, and its first pattern required exactly one space before the + // closing pipe. Against a formatted corpus it matched no row at all and + // reported every tracked suite missing — 150 findings, all false, from a gate + // that looked like it was working. + let dir = bench("padded", &["a.bats"], None); + std::fs::create_dir_all(dir.join("bench").join("suites")).unwrap(); + common::write( + &dir.join("bench").join("suites"), + "RESULTS.md", + "| seconds | share | suite |\n| ---: | ---: | --- |\n| 4.5 | 100.0% | `tests/a.bats` |\n", + ); + common::git_in(&dir, &["add", "-A"]); + common::git_in(&dir, &["commit", "-qm", "padded"]); + let output = check(&dir); + assert_eq!(output.status.code(), Some(0), "{}", said(&output)); +} + +// --- the producer ------------------------------------------------------------- + +fn record(dir: &Path, write: bool) -> Output { + let mut command = batten(); + command.arg("record").arg("suites"); + if write { + command.arg("--write"); + } + command + .current_dir(dir) + .output() + .expect("run batten record suites") +} + +fn report(dir: &Path, suites: &[(&str, &str)]) { + std::fs::create_dir_all(dir.join("target").join("bats-report")).unwrap(); + let mut xml = String::from("\n"); + for (name, time) in suites { + xml.push_str(&format!( + "\n\n" + )); + } + xml.push_str("\n"); + common::write(&dir.join("target").join("bats-report"), "report.xml", &xml); +} + +#[test] +fn a_report_naming_a_suite_the_tree_does_not_track_is_could_not_look() { + // THE REMEDY MUST REACH THE STATE IT PRESCRIBES. A suite retired while the + // report still named it made the regenerated corpus carry a cost attached to + // nothing — so the gate refused the very file its own remedy had just + // produced, and the author reads the gate as broken rather than the report as + // stale. + let dir = bench("stale-report", &["a.bats"], None); + report(&dir, &[("a.bats", "1.0"), ("retired.bats", "2.0")]); + let output = record(&dir, false); + assert_ne!(output.status.code(), Some(0), "{}", said(&output)); + assert!( + said(&output).contains("does not track"), + "{}", + said(&output) + ); +} + +#[test] +fn a_report_naming_only_tracked_suites_derives_a_corpus() { + let dir = bench("derive", &["a.bats", "b.bats"], None); + report(&dir, &[("a.bats", "3.0"), ("b.bats", "1.0")]); + let output = record(&dir, false); + assert_eq!(output.status.code(), Some(0), "{}", said(&output)); + let text = said(&output); + // Ordered by cost, descending: the corpus is read by an author asking which + // side of the distribution a file sits on, and the answer is at the top. + let slow = text.find("tests/a.bats").expect("the slow suite"); + let quick = text.find("tests/b.bats").expect("the quick suite"); + assert!(slow < quick, "{text}"); + assert!(text.contains("75.0%"), "{text}"); +} + +#[test] +fn an_absent_report_is_could_not_look_rather_than_an_empty_corpus() { + // `test:bats` is receipt-gated, so a tree whose receipt is valid skips the + // run and an absent report is the ORDINARY state after a no-op lap. Writing + // that out as "every suite costs nothing" would publish the collapse rather + // than merely compute it. + let dir = bench("no-report", &["a.bats"], None); + let output = record(&dir, false); + assert_ne!(output.status.code(), Some(0), "{}", said(&output)); + assert!(said(&output).contains("has not run"), "{}", said(&output)); +} + +#[test] +fn the_producer_writes_a_corpus_the_gate_accepts() { + // THE TWO HALVES OF ONE CONTRACT, and neither can see the other: the producer + // spells the path as `suites::CORPUS` and the rule spells it in its `lines` + // row. A port that let them pick their own locations would have disconnected + // them with nothing red. + let dir = bench("round-trip", &["a.bats", "b.bats"], None); + report(&dir, &[("a.bats", "3.0"), ("b.bats", "1.0")]); + let written = record(&dir, true); + assert_eq!(written.status.code(), Some(0), "{}", said(&written)); + common::git_in(&dir, &["add", "-A"]); + common::git_in(&dir, &["commit", "-qm", "corpus"]); + let output = check(&dir); + assert_eq!(output.status.code(), Some(0), "{}", said(&output)); +} diff --git a/crates/batten/tests/it/timeout_budget.rs b/crates/batten/tests/it/timeout_budget.rs new file mode 100644 index 000000000..68697d3d7 --- /dev/null +++ b/crates/batten/tests/it/timeout_budget.rs @@ -0,0 +1,393 @@ +//! Every workflow job's timeout carries a budget that justifies it, over the +//! compiled binary (CLOUD-266, ported from `mise-tasks/timeout-check.sh` under +//! CLOUD-843). +//! +//! **What is decidable only here.** `policy/timeout-budget.rego` carries +//! load-time cases pinning the walker, the two grammars and the arithmetic, and +//! every one supplies one workflow with `with input as`. That fabricates the very +//! shape the engine may be unable to produce (CLOUD-845), and this rule's declared +//! SCOPE is precisely what it fabricates: every workflow including the scheduled +//! and release ones, which is the hole the retiring gate was written to fill. A +//! resolution reaching only some of them would leave the most expensive job here +//! unbounded with nothing red — and a module whose suite only fabricated one file +//! cannot tell that state from a clean one. +//! +//! The self-consumption case is the one the retiring suite opened on: the real +//! workflows all carry a justified budget today. +// +// carried: mise-tasks/timeout-check.sh policy/timeout-budget.rego crates/batten/tests/it/timeout_budget.rs +// carried: tests/timeout-check.bats policy/timeout-budget.rego crates/batten/tests/it/timeout_budget.rs +// +// carried: "the repo's real workflows all carry a justified budget today" policy/timeout-budget.rego +// carried: "a grandfathered budget passes — dated debt is the day-one state" policy/timeout-budget.rego +// carried: "a measured budget whose arithmetic agrees passes" policy/timeout-budget.rego +// carried: "a job with no timeout-minutes is caught, and the pointer names the job" policy/timeout-budget.rego +// carried: "a timeout with no budget comment is caught" policy/timeout-budget.rego +// carried: "a malformed budget comment is caught" policy/timeout-budget.rego +// carried: "a budget comment missing its measured= date is malformed, not accepted" policy/timeout-budget.rego +// carried: "a measured budget whose declared minutes disagree with its own arithmetic is caught" policy/timeout-budget.rego +// carried: "a per-job multiplier is refused — the multiplier is one repo-wide constant" policy/timeout-budget.rego +// carried: "a step-level timeout does not satisfy the job's obligation" policy/timeout-budget.rego +// carried: "a matrix job passes with one budget covering every leg" policy/timeout-budget.rego +// carried: "prose that merely mentions a budget does not satisfy any job" policy/timeout-budget.rego +// carried: "every offender is reported in one pass, not just the first" policy/timeout-budget.rego +// carried: "a file with no jobs: block is exit 2, never a pass" policy/timeout-budget.rego +// carried: "a jobs: block with no job keys under it is exit 2, never a pass" policy/timeout-budget.rego +// changed: "a missing file is exit 2, never a pass" policy/timeout-budget.rego the shell took explicit paths and refused one it could not open; the successor declares a `line_sources` glob and the ENGINE decides this earlier — a rule whose glob matches nothing is not evaluated at all, and `input.tree.missing` is never populated on the tree surface (CLOUD-1049, measured identically for `policy/mise-pin-agreement.rego`'s own could-not-look clause). The two arms that carried the weight survive whole: a workflow that resolves and declares no jobs block, and one whose jobs block has no keys under it, are both `job list empty` +// changed: "the gate makes no network call" policy/timeout-budget.rego the property is carried by CONSTRUCTION rather than by a case: a `kind = "policy"` rule over `scope = "tree"` reads `input.tree` and has no spawn or socket surface at all, where the shell could in principle have reached the network and needed a case saying it did not + +// Panicking on setup failure is the idiomatic way for a test to fail loudly. +#![allow(clippy::unwrap_used, clippy::expect_used)] + +use crate::common; + +use std::path::{Path, PathBuf}; +use std::process::Output; + +use common::{Fixture, git_in, run, stderr, stdout}; + +/// A repository declaring only this rule, so any finding is the one under test. +fn budget_repo(name: &str, workflows: &[(&str, &str)]) -> PathBuf { + let mut fixture = Fixture::new(name).config( + "version = 1\n\n\ + [[pattern]]\n\ + id = \"workflow-job-key\"\n\ + regex = '^ [A-Za-z0-9_-]+:[[:space:]]*$'\n\n\ + [[pattern]]\n\ + id = \"workflow-top-level-key\"\n\ + regex = '^[a-z][A-Za-z0-9_-]*:'\n\n\ + [[pattern]]\n\ + id = \"job-timeout-line\"\n\ + regex = '^ timeout-minutes:[[:space:]]*[0-9]+'\n\n\ + [[pattern]]\n\ + id = \"timeout-budget-measured\"\n\ + regex = '^#[[:space:]]*budget:[[:space:]]*p95=([0-9]+)s[[:space:]]+x([0-9]+)[[:space:]]+measured=([0-9]{4}-[0-9]{2}-[0-9]{2})[[:space:]]*$'\n\n\ + [[pattern]]\n\ + id = \"timeout-budget-grandfathered\"\n\ + regex = '^#[[:space:]]*budget:[[:space:]]*grandfathered[[:space:]]+measured=[0-9]{4}-[0-9]{2}-[0-9]{2}[[:space:]]*$'\n\n\ + [[verdict]]\n\ + id = \"timer declare missing\"\n\ + gloss = \"a workflow job declares no timeout at all\"\n\ + class = \"An unbounded job burns a runner until the forge's own ceiling stops it.\"\n\n\ + [[verdict.route]]\n\ + id = \"prose read first\"\n\ + kind = \"document\"\n\ + target = \"AGENTS.md\"\n\n\ + [[verdict]]\n\ + id = \"timer carry unnamed\"\n\ + gloss = \"a timeout carries no budget comment beside it\"\n\ + class = \"A limit with no justification is boilerplate and never reports that a job got slower.\"\n\n\ + [[verdict.route]]\n\ + id = \"prose read first\"\n\ + kind = \"document\"\n\ + target = \"AGENTS.md\"\n\n\ + [[verdict]]\n\ + id = \"timer parse unclear\"\n\ + gloss = \"a budget comment does not match either accepted form\"\n\ + class = \"A budget a reader cannot parse is one a gate cannot check.\"\n\n\ + [[verdict.route]]\n\ + id = \"prose read first\"\n\ + kind = \"document\"\n\ + target = \"AGENTS.md\"\n\n\ + [[verdict]]\n\ + id = \"timer count wrong\"\n\ + gloss = \"a measured budget's declared minutes disagree with its own stated p95\"\n\ + class = \"A justification for a different number than the one enforced.\"\n\n\ + [[verdict.route]]\n\ + id = \"prose read first\"\n\ + kind = \"document\"\n\ + target = \"AGENTS.md\"\n\n\ + [[verdict]]\n\ + id = \"timer count other\"\n\ + gloss = \"a measured budget uses a multiplier other than the repository's\"\n\ + class = \"A per-job multiplier is a per-job argument.\"\n\n\ + [[verdict.route]]\n\ + id = \"prose read first\"\n\ + kind = \"document\"\n\ + target = \"AGENTS.md\"\n\n\ + [[verdict]]\n\ + id = \"job list empty\"\n\ + gloss = \"a workflow declares no jobs this rule can read\"\n\ + class = \"A file that cannot be parsed as a workflow is not one with no jobs.\"\n\n\ + [[verdict.route]]\n\ + id = \"prose read first\"\n\ + kind = \"document\"\n\ + target = \"AGENTS.md\"\n\n\ + [[rule]]\n\ + id = \"timeout-budget\"\n\ + kind = \"policy\"\n\ + scope = \"tree\"\n\ + line_sources = [\".github/workflows/*.yml\"]\n\ + module = \"policy/timeout-budget.rego\"\n\ + severity = \"deny\"\n", + ); + fixture = fixture.file("AGENTS.md", "the consumer's own authority\n"); + for (path, body) in workflows { + fixture = fixture.file(path, body); + } + let dir = fixture.git().build(); + common::write( + &dir, + "policy/timeout-budget.rego", + &std::fs::read_to_string(common::at_root("policy/timeout-budget.rego")).unwrap(), + ); + git_in(&dir, &["add", "-A"]); + git_in(&dir, &["commit", "-q", "-m", "base"]); + dir +} + +fn check(dir: &Path) -> Output { + run(dir, &["check", "--rule", "timeout-budget"]) +} + +/// One workflow whose `jobs:` block carries `jobs`. +fn flow(jobs: &str) -> String { + format!("on: push\njobs:\n{jobs}") +} + +const GRANDFATHERED: &str = " build:\n runs-on: ubuntu-latest\n timeout-minutes: 15 # budget: grandfathered measured=2026-08-01\n"; + +#[test] +fn a_grandfathered_budget_passes() { + // Dated debt is the day-one state: honest, gateable, and visibly not a + // justification. + let dir = budget_repo( + "budget-grandfathered", + &[(".github/workflows/ci.yml", &flow(GRANDFATHERED))], + ); + let output = check(&dir); + assert_eq!( + output.status.code(), + Some(0), + "out={} err={}", + stdout(&output), + stderr(&output) + ); +} + +#[test] +fn a_measured_budget_whose_arithmetic_agrees_passes() { + let dir = budget_repo( + "budget-measured", + &[( + ".github/workflows/ci.yml", + &flow(" build:\n timeout-minutes: 14 # budget: p95=267s x3 measured=2026-08-01\n"), + )], + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(0), "{}", stdout(&output)); +} + +#[test] +fn a_job_with_no_timeout_is_refused_and_the_pointer_names_the_job() { + let dir = budget_repo( + "budget-no-timeout", + &[( + ".github/workflows/ci.yml", + &flow(" build:\n runs-on: ubuntu-latest\n"), + )], + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); + assert!( + stdout(&output).contains("build"), + "the pointer names the job: {:?}", + stdout(&output) + ); +} + +#[test] +fn a_timeout_with_no_budget_comment_is_refused() { + let dir = budget_repo( + "budget-no-comment", + &[( + ".github/workflows/ci.yml", + &flow(" build:\n timeout-minutes: 15\n"), + )], + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); +} + +#[test] +fn a_malformed_budget_comment_is_refused() { + let dir = budget_repo( + "budget-malformed", + &[( + ".github/workflows/ci.yml", + &flow(" build:\n timeout-minutes: 15 # budget is generous here\n"), + )], + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); +} + +#[test] +fn a_budget_missing_its_measured_date_is_malformed() { + // The grammars are matched WHOLE, so a nearly-right comment is not accepted. + let dir = budget_repo( + "budget-no-date", + &[( + ".github/workflows/ci.yml", + &flow(" build:\n timeout-minutes: 15 # budget: grandfathered\n"), + )], + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); +} + +#[test] +fn arithmetic_that_disagrees_with_its_own_p95_is_refused() { + // The capability the retiring program said a `forbid` row did not have: a + // numeric comparison over a parsed field. + let dir = budget_repo( + "budget-arithmetic", + &[( + ".github/workflows/ci.yml", + &flow(" build:\n timeout-minutes: 30 # budget: p95=267s x3 measured=2026-08-01\n"), + )], + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); +} + +#[test] +fn a_per_job_multiplier_is_refused() { + // The point of a single number is that loosening it anywhere is visible as + // loosening it everywhere. + let dir = budget_repo( + "budget-multiplier", + &[( + ".github/workflows/ci.yml", + &flow(" build:\n timeout-minutes: 45 # budget: p95=267s x10 measured=2026-08-01\n"), + )], + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); +} + +#[test] +fn a_step_timeout_does_not_satisfy_the_jobs_obligation() { + let dir = budget_repo( + "budget-step-level", + &[( + ".github/workflows/ci.yml", + &flow( + " build:\n steps:\n - run: make\n timeout-minutes: 5 # budget: grandfathered measured=2026-08-01\n", + ), + )], + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); +} + +#[test] +fn a_matrix_job_passes_with_one_budget_covering_every_leg() { + // The structural oddity a separate budget table would have needed a key + // scheme to express. + let dir = budget_repo( + "budget-matrix", + &[( + ".github/workflows/release-artifacts.yml", + &flow( + " dist:\n strategy:\n matrix:\n target: [a, b, c]\n timeout-minutes: 30 # budget: grandfathered measured=2026-08-01\n", + ), + )], + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(0), "{}", stdout(&output)); +} + +#[test] +fn prose_that_merely_mentions_a_budget_satisfies_no_job() { + let dir = budget_repo( + "budget-prose", + &[( + ".github/workflows/ci.yml", + &format!( + "# budget: grandfathered measured=2026-08-01\n{}", + flow(" build:\n timeout-minutes: 15\n") + ), + )], + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); +} + +#[test] +fn every_offender_is_reported_in_one_pass() { + let dir = budget_repo( + "budget-many-offenders", + &[ + ( + ".github/workflows/ci.yml", + &flow(" build:\n timeout-minutes: 15\n"), + ), + ( + ".github/workflows/release.yml", + &flow(" publish:\n runs-on: ubuntu-latest\n"), + ), + ], + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); + let text = stdout(&output); + assert!( + text.contains("build") && text.contains("publish"), + "one run reports both: {text:?}" + ); +} + +#[test] +fn a_file_with_no_jobs_block_is_not_a_pass() { + let dir = budget_repo( + "budget-no-jobs-block", + &[(".github/workflows/ci.yml", "on: push\nname: ci\n")], + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); +} + +#[test] +fn a_jobs_block_with_no_job_keys_is_not_a_pass() { + let dir = budget_repo( + "budget-empty-jobs-block", + &[( + ".github/workflows/ci.yml", + "on: push\njobs:\n # every job was removed\n", + )], + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); +} + +#[test] +fn output_is_pointer_only() { + let dir = budget_repo( + "budget-pointer-only", + &[( + ".github/workflows/ci.yml", + &flow(" build:\n runs-on: a-distinctive-runner\n timeout-minutes: 15\n"), + )], + ); + let output = check(&dir); + assert_eq!(output.status.code(), Some(2)); + assert!( + !stdout(&output).contains("a-distinctive-runner"), + "the workflow body is payload: {:?}", + stdout(&output) + ); +} + +#[test] +fn the_real_workflows_all_carry_a_justified_budget() { + // The self-consumption case the retiring suite opened on. + let output = + common::run_at_real_root(&common::at_root(""), &["check", "--rule", "timeout-budget"]); + assert_eq!( + output.status.code(), + Some(0), + "a job here has no justified budget: {}", + stdout(&output) + ); +} diff --git a/crates/batten/tests/it/verdict.rs b/crates/batten/tests/it/verdict.rs new file mode 100644 index 000000000..492656bb8 --- /dev/null +++ b/crates/batten/tests/it/verdict.rs @@ -0,0 +1,164 @@ +//! One authority folds a run's findings and blind spots into an exit code, over +//! the compiled binary (CLOUD-1718). +//! +//! **What is decidable only here.** `exit.rs` unit-tests `ExitCode::combine`, and +//! those cases pin the fold. They cannot pin that a SHELL can reach it, which is +//! the whole justification for the verb: the bug dies with each program that +//! retires, so what pays for this row is what does NOT retire — the workflow +//! tree, which is permanently bash by declaration; the installer, which is bash +//! by construction; and consumer repositories, whose gates hit the identical +//! inversion with no campaign to save them. Every one of those calls a PROCESS +//! and branches on `$?`, so the claim under test is the process's status, and a +//! unit test over the enum cannot make it. +//! +//! The discriminating case is the inversion itself: the retiring corpus reads +//! `1` as a violation and `2` as could-not-look, and this table reads `2` as a +//! violation and `3` as could-not-look. A port that carried the old fold across +//! would not report a worse code — it would report the OPPOSITE meaning. + +// Panicking on setup failure is the idiomatic way for a test to fail loudly. +#![allow(clippy::unwrap_used, clippy::expect_used)] + +use crate::common; + +use std::process::Output; + +use common::{Fixture, run, stderr, stdout}; + +/// The verb reads two integers off its own command line, so the repository it +/// runs in cannot change its answer. A fixture is built anyway — that +/// independence is a claim worth holding rather than assuming. +fn anywhere() -> std::path::PathBuf { + Fixture::new("verdict-anywhere") + .config("version = 1\n") + .git() + .build() +} + +fn verdict(args: &[&str]) -> Output { + let dir = anywhere(); + let mut argv = vec!["verdict"]; + argv.extend_from_slice(args); + run(&dir, &argv) +} + +#[test] +fn a_clean_run_exits_success() { + let output = verdict(&["--findings", "0", "--unjudgeable", "0"]); + assert_eq!( + output.status.code(), + Some(0), + "out={} err={}", + stdout(&output), + stderr(&output) + ); +} + +#[test] +fn findings_alone_exit_violation() { + let output = verdict(&["--findings", "3", "--unjudgeable", "0"]); + assert_eq!(output.status.code(), Some(2), "{}", stderr(&output)); +} + +#[test] +fn a_blind_spot_alone_exits_internal() { + let output = verdict(&["--findings", "0", "--unjudgeable", "1"]); + assert_eq!(output.status.code(), Some(3), "{}", stderr(&output)); +} + +#[test] +fn a_blind_spot_outranks_a_finding() { + // THE PRECEDENCE, over the process rather than over the enum (CLOUD-251, + // CLOUD-1718). One retiring program deliberately inverted this, and nothing + // could notice because the rule lived in each epilogue's prose. A run that + // could not read part of its subject has an INCOMPLETE answer: a caller told + // `2` fixes what it names and sees green, where a caller told `3` learns the + // gate did not run. The findings reach stderr either way, so ranking the + // blind spot first costs nothing. + let output = verdict(&["--findings", "3", "--unjudgeable", "1"]); + assert_eq!(output.status.code(), Some(3), "{}", stderr(&output)); +} + +#[test] +fn the_two_contracts_disagree_on_every_nonclean_answer() { + // THE INVERSION, which is what makes this a correctness row rather than a + // duplication one. The retiring corpus's `1` is a violation and its `2` is + // could-not-look; here `2` is the violation and `3` is could-not-look. The + // dangerous half is the COLLISION: the corpus's could-not-look is this + // table's violation, so a fold carried across does not degrade, it inverts. + let shell_violation = 1; + let shell_could_not_look = 2; + + let found = verdict(&["--findings", "1"]); + assert_ne!( + found.status.code(), + Some(shell_violation), + "a finding must not take the corpus's violation code" + ); + assert_eq!( + found.status.code(), + Some(shell_could_not_look), + "and the collision is exact: this table's violation IS the corpus's could-not-look" + ); + + let blind = verdict(&["--unjudgeable", "1"]); + assert_ne!( + blind.status.code(), + Some(shell_could_not_look), + "a blind spot must not take the corpus's could-not-look code" + ); +} + +#[test] +fn an_absent_count_reads_as_zero() { + // The verb's job is to be callable from a shell epilogue, where an unset + // variable expands to the empty string. A caller with only findings to + // report should not have to say it saw no blind spots. + let output = verdict(&["--findings", "2"]); + assert_eq!(output.status.code(), Some(2), "{}", stderr(&output)); + let bare = verdict(&[]); + assert_eq!(bare.status.code(), Some(0), "{}", stderr(&bare)); +} + +#[test] +fn an_unparsable_count_reads_as_zero_rather_than_replacing_the_verdict() { + // The safe direction is the one that reports LESS: a miscounted finding is + // still on the caller's own stderr, where a usage error would replace the + // verdict entirely and put the caller back to hand-folding the case it came + // here to avoid. + let output = verdict(&["--findings", "", "--unjudgeable", "1"]); + assert_eq!(output.status.code(), Some(3), "{}", stderr(&output)); +} + +#[test] +fn the_fold_never_reports_a_failure_of_battens_own() { + // `1` is a statement about the INVOCATION. No count of findings or of + // unreadable subjects can make an invocation malformed, so it is unreachable + // through this verb — which is what lets a caller branch on `1` as "the gate + // is misconfigured" and never as "policy says no". + for findings in ["0", "1", "7"] { + for unjudgeable in ["0", "1", "7"] { + let output = verdict(&["--findings", findings, "--unjudgeable", unjudgeable]); + assert_ne!( + output.status.code(), + Some(1), + "{findings}/{unjudgeable} must not report a usage error" + ); + } + } +} + +#[test] +fn the_verb_reads_no_repository() { + // Its independence from the tree is what makes it callable from a consumer + // repository, an installer or a workflow step — none of which has this + // config, and one of which has no repository at all. + let bare = Fixture::new("verdict-no-config").git().build(); + let output = run(&bare, &["verdict", "--findings", "1"]); + assert_eq!( + output.status.code(), + Some(2), + "a tree with no batten.toml must still get a verdict: {}", + stderr(&output) + ); +} diff --git a/crates/batten/tests/it/workflow_shell_census.rs b/crates/batten/tests/it/workflow_shell_census.rs new file mode 100644 index 000000000..11ab4bcb1 --- /dev/null +++ b/crates/batten/tests/it/workflow_shell_census.rs @@ -0,0 +1,222 @@ +//! `.github/workflows/**` is counted, over the compiled binary (CLOUD-1709). +//! +//! **What these cases are for, and what `ratchet.rs` next door already covers.** +//! That tier proves the ratchet KIND: direction, counts, base movement, waivers, +//! byte stability. Nothing there is about this surface. What is decidable only +//! here is that the two literals this row picked actually count what a workflow +//! spells — and, more importantly, that they do NOT count what a workflow spells +//! that is not shell. +//! +//! **The anchoring is the case worth having.** `run:` unanchored also matches +//! `workflow_run:` and `check_run:`, which are trigger declarations rather than +//! steps; there are 15 of them in the tree today. A row that counted those would +//! report a rise whenever a workflow gained a trigger, and would report a fall +//! when one lost it — a census moving on something that is not shell at all. +//! `a_trigger_declaration_is_not_a_shell_step` is the discriminating case, and it +//! is the one a naive `pattern = "run:"` fails. +//! +//! The exit assertion is **2** throughout. The retiring shell corpus spells a +//! violation `1`; carrying that inversion in is the defect CLOUD-1718 names. + +// Panicking on setup failure is the idiomatic way for a test to fail loudly. +#![allow(clippy::unwrap_used, clippy::expect_used)] + +use crate::common; + +use std::path::{Path, PathBuf}; +use std::process::Output; + +use common::{Fixture, git_in, run, stdout}; + +/// The row as `batten.toml` declares it, minus the origin ref. +/// +/// `base = "main"` rather than `origin/main`: the fixtures carry no origin +/// literal (`no-origin-literal-in-fixtures`), and a local branch proves the same +/// plumbing. Both spellings are declared together because the pair is the +/// predicate — one row alone is a census with a hole in it, which is the whole +/// reason there are two. +fn census_config(admits: bool) -> String { + let permit = if admits { + "admits_with = \"# workflow-shell:\"\n" + } else { + "" + }; + format!( + "version = 1\n\n\ + [[rule]]\n\ + id = \"workflow-shell-not-growing\"\n\ + kind = \"ratchet\"\n\ + glob = \".github/workflows/**\"\n\ + pattern = \"\\n run:\"\n\ + direction = \"non_increasing\"\n\ + base = \"main\"\n\ + {permit}\ + severity = \"deny\"\n\ + scope = \"tree\"\n\n\ + [[rule]]\n\ + id = \"workflow-shell-not-growing-bare\"\n\ + kind = \"ratchet\"\n\ + glob = \".github/workflows/**\"\n\ + pattern = \"\\n - run:\"\n\ + direction = \"non_increasing\"\n\ + base = \"main\"\n\ + {permit}\ + severity = \"deny\"\n\ + scope = \"tree\"\n" + ) +} + +/// A workflow carrying one of each spelling, plus a trigger that is not a step. +/// +/// The `workflow_run:` trigger is in the BASE rather than added by a case, +/// because the property is that it is never counted — in either half of the +/// comparison. A fixture that only added one would leave the base side untested. +const BASE_WORKFLOW: &str = "\ +name: ci +on: + workflow_run: + workflows: [other] + types: [completed] +jobs: + build: + runs-on: ubuntu-latest + steps: + - name: named step + run: echo one + - run: echo two +"; + +fn census_repo(name: &str, admits: bool) -> PathBuf { + let dir = Fixture::new(name) + .config(&census_config(admits)) + .file(".github/workflows/ci.yml", BASE_WORKFLOW) + .git() + .build(); + git_in(&dir, &["add", "-A"]); + git_in(&dir, &["commit", "-q", "-m", "base"]); + dir +} + +fn check(dir: &Path) -> Output { + run(dir, &["check"]) +} + +#[test] +fn a_tree_at_the_ceiling_passes() { + let dir = census_repo("workflow-census-held", false); + let output = check(&dir); + assert_eq!(output.status.code(), Some(0), "{}", stdout(&output)); + assert!( + output.stdout.is_empty(), + "a ratchet that held says nothing: {}", + stdout(&output) + ); +} + +#[test] +fn one_named_step_over_the_ceiling_fails() { + // The 8-space spelling: a `run:` key under a `- name:`. + let dir = census_repo("workflow-census-named-over", false); + common::write( + &dir, + ".github/workflows/ci.yml", + &format!("{BASE_WORKFLOW} - name: added\n run: echo three\n"), + ); + + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); + let text = stdout(&output); + assert!( + text.contains("workflow-shell-not-growing"), + "the finding names the rule: {text:?}" + ); + assert!( + text.contains("1->2"), + "and carries both counts, so a reader sees how far it moved: {text:?}" + ); + assert!( + !text.contains("echo three"), + "pointer-only: the step's body is payload and never appears: {text:?}" + ); +} + +#[test] +fn one_bare_step_over_the_ceiling_fails_the_other_row() { + // The 6-space `- run:` spelling. A separate case rather than a second + // assertion, because the whole reason there are two rows is that one literal + // cannot see the other — so a case that only exercised the first would pass + // over a census with a live hole in it. + let dir = census_repo("workflow-census-bare-over", false); + common::write( + &dir, + ".github/workflows/ci.yml", + &format!("{BASE_WORKFLOW} - run: echo three\n"), + ); + + let output = check(&dir); + assert_eq!(output.status.code(), Some(2), "{}", stdout(&output)); + let text = stdout(&output); + assert!( + text.contains("workflow-shell-not-growing-bare"), + "the bare spelling is the OTHER row's, and it must be the one that fires: {text:?}" + ); +} + +#[test] +fn a_trigger_declaration_is_not_a_shell_step() { + // DISCRIMINATING, and the case a naive `pattern = "run:"` fails. Adding a + // `workflow_run:` trigger adds no shell, so neither row may move. Without the + // newline anchor both would count it and the census would report growth on a + // change that added no bash at all. + let dir = census_repo("workflow-census-trigger", false); + common::write( + &dir, + ".github/workflows/ci.yml", + &BASE_WORKFLOW.replace( + " types: [completed]\n", + " types: [completed]\n check_run:\n types: [created]\n", + ), + ); + + let output = check(&dir); + assert_eq!( + output.status.code(), + Some(0), + "a trigger is not shell and must not move the census: {}", + stdout(&output) + ); +} + +#[test] +fn a_declared_increase_is_admitted_and_an_undeclared_one_is_not() { + // The `admits_with` half, which is why the row carries the permit at all: a + // retirement that lands a `run: batten ` step raises this count while + // lowering real bash, and that firing is the campaign succeeding rather than + // a defect. Both halves in one case because the permit means nothing unless + // the same edit without it still fails. + let added = format!("{BASE_WORKFLOW} - name: added\n run: echo three\n"); + + let undeclared = census_repo("workflow-census-permit-absent", true); + common::write(&undeclared, ".github/workflows/ci.yml", &added); + assert_eq!( + check(&undeclared).status.code(), + Some(2), + "an undeclared increase is still refused when the column exists" + ); + + let declared = census_repo("workflow-census-permit-present", true); + common::write( + &declared, + ".github/workflows/ci.yml", + &format!( + "# workflow-shell: CLOUD-1709 the step invokes a retired program's successor\n{added}" + ), + ); + let output = check(&declared); + assert_eq!( + output.status.code(), + Some(0), + "a declared increase is owned rather than refused: {}", + stdout(&output) + ); +} diff --git a/hk.pkl b/hk.pkl index fad887a05..39af3848e 100644 --- a/hk.pkl +++ b/hk.pkl @@ -37,62 +37,8 @@ local gate = new Mapping { glob = List("hk.pkl", "mise.toml") check = "mise run hk-version" } - // CI installs a narrow, per-job tool set instead of the whole toolchain - // (ci.yml, install_args). That is worth ~10s per job, and its cost is a - // second place tool names are written down. `mise install` does not fail on - // an unknown name, so drift between the two files surfaces late as "command - // not found". This is that rule's mechanism: the names must resolve. - // The glob is every workflow, not `ci.yml` alone: since CLOUD-812 this gate - // also asks whether a `pull_request` workflow runs the mise action with no - // `install_args` at all, and that question is about files `ci.yml` does not - // name. A glob narrower than the subject means the pre-commit tier never - // selects the step that would have caught the very file being added. - ["ci-tools-check"] { - glob = List(".github/workflows/*.yml", "mise.toml") - check = "mise run ci-tools-check" - } - // The honesty gate for `rust.yml`'s `paths:` filter (CLOUD-398). It ships with - // the filter because a filter that is too NARROW fails silently: the four jobs - // are absent, `checks-green` accepts absent by design, and the run is green - // without their verdict. `mise.toml` is in the glob because the filter has to - // select it — the jobs invoke `mise run` tasks — so a change here is a change - // to what the filter must cover. - ["rust-paths-check"] { - glob = List(".github/workflows/rust.yml", "mise.toml") - check = "mise run rust-paths-check" - } - // The per-suite cost corpus, held to the tree it describes (CLOUD-352). Not - // `slow`: the predicate is set membership over committed text and answers in - // milliseconds — it re-runs NOTHING. The durations themselves are a clock and - // are deliberately not gated, since wall clock is not reproducible and a gate - // demanding byte equality on one would be red on every second run. - // - // NO GLOB, AND THAT IS THE WHOLE POINT (CLOUD-1216). It was globbed on the - // corpus and on the suites, because both directions rot — a suite added - // without regenerating is a file nothing records, and a suite deleted leaves a - // cost attached to nothing. **The glob could only ever deliver the first - // half.** hk selects a step by matching CHANGED PATHS against it, and a - // deleted path is not there to match, so the one commit shape the reverse - // direction exists for was the one shape that did not select the step. - // - // Measured on #770: `f45e214` deleted `tests/derived-check.bats`, passed the - // full pre-commit gate, and left the corpus recording a suite the tree no - // longer tracked — the exact state this check exists to refuse. It survived - // four more commits, every one of them green, and was found by running the - // task by hand. - // - // THE COST, measured rather than asserted, because the comment this replaces - // said "milliseconds" and a reader reads that as ~10: **531ms** (three runs, - // 531/530/531). That is the price of the step on every commit, and hk runs - // steps concurrently so the marginal wall-clock cost is usually less. It is - // paid to close a direction that was structurally unreachable, which is worth - // half a second. - // - // The class is wider than this step — any predicate about a file's ABSENCE, or - // about set equality against a tracked listing, inherits it. CLOUD-1216 carries - // the survey. ["suite-bench-check"] { - check = "mise run suite-bench-check" + check = "cargo run --quiet -p batten -- check --rule suite-cost-corpus" } // CLOUD-266's rule reaches the one budget nobody had looked at (CLOUD-668). // The MCP startup budget was the host's 30000ms default, never measured, and @@ -104,132 +50,9 @@ local gate = new Mapping { glob = List(".claude/settings.json", "mise-tasks/mcp-timeout-budget.sh") check = "mise run mcp-timeout-budget" } - // The floor is a derived copy of the toolchain pin (CLOUD-593), so the two - // must agree. This REPLACES the `msrv` job, which answered the same question - // by compiling the workspace a second time at a second toolchain — right while - // the numbers were independent, and pure cost once they are coupled. Note what - // the old gate could not catch: it was green through all twelve releases the - // pin sat stale, because the two numbers agreed. Agreeing on a stale number is - // what it was built to confirm. - ["msrv-pin-agreement"] { - glob = List("Cargo.toml", "mise.toml", "renovate.json5") - check = "mise run msrv-pin-agreement" - } - // An MSRV cap lives in two files — the manifest bound and the bot-side - // `allowedVersions` rule that mirrors it, because the bot cannot read a - // manifest cap. That mirror moved from `.github/dependabot.yml` to - // `renovate.json5` with CLOUD-660, which retired the bot the old file - // configured; the predicate is unchanged and both sets are still empty. - // Lift only the manifest side and everything passes while the crate stays - // frozen: the bot keeps withholding a version the manifest now admits, and - // nothing is red. That half-lift is the direction with no symptom, and it is - // why the row is set equality both ways rather than a one-sided check. - // - // Both sets are empty as of CLOUD-593, and the row is still load-bearing: it - // is a ratchet on the next cap added to either file. A gate that only worked - // while a list was non-empty would have been deleted with the list. - ["cap-drift"] { - glob = List("Cargo.toml", "renovate.json5") - check = "mise run cap-drift" - } - // The sibling question about the same two files, asked of the RESOLUTION - // rather than the requirement (CLOUD-767). `cap-drift` decides whether the - // manifest and the bot config agree about what may be proposed; - // `digest-major-agreement` decides whether what was actually resolved still - // composes. A caret requirement is silent on the second, which is why this - // reads `Cargo.lock` and that one does not. - // - // The rule it carries was a `Cargo.toml` comment asserting something untrue - // for as long as `gix` had been a dependency. Prose is feedforward only. - ["digest-major-agreement"] { - glob = List("Cargo.toml", "Cargo.lock") - check = "mise run digest-major-agreement" - } - // `coverage` and `scorecard` emit a number for a human and must never gate a - // branch — a coverage threshold rewards line-touching, and scorecard scores - // deliberate design decisions (trunk-based review, the landing lease's - // `actions: write`) as defects. Both said so in a comment and nothing held - // it: the only thing keeping them off the landing path was that nobody had - // added them to it (CLOUD-582). This is the inverse of `ci-parity`'s - // "every task CI runs, verify runs" — named reports appear in NEITHER - // `[tasks.verify]` nor any `pull_request` workflow. Kept separate from that - // gate because the two fail for opposite reasons and one deny cannot say both. - ["report-only-check"] { - glob = List(".github/workflows/*.yml", "mise.toml", "mise-tasks/report-only-check.sh") - check = "mise run report-only-check" - } - // Enforces the "research goes to Linear, not a repo docs/ tree" directive as - // a runnable gate, not prose (see AGENTS.md): fails if any docs/ path is - // tracked. Research deliverables and evidence notes attach to the Linear - // issue; the repo carries code and its close-in config, not research prose. - // - // Deliberately keeps NO glob where its two glob-less neighbours got one - // (CLOUD-224). Its input genuinely is the whole index: it fails on any TRACKED - // docs/ path, including one an earlier commit left behind, so a glob would let - // a violation persist unreported for every commit that did not touch it. It - // costs ~95ms, which is what makes that affordable. ["no-docs-tree"] { check = "mise run no-docs-tree" } - // The coverage half of the nextest swap (CLOUD-813). `test:cargo` runs - // `cargo nextest run`, which does not execute doctests; the swap was safe - // because `cargo test --doc --workspace` reported zero on this workspace, and - // that emptiness is a measurement rather than a property. This asserts it - // still holds, so a doc example cannot arrive as dead code that reads like a - // tested one. - // - // Globbed on the crate sources it scans, unlike its glob-less neighbour above: - // a fence can only appear in a tracked `.rs` file, so the input really is that - // set and a commit touching none of them cannot change the answer. - ["no-doctests"] { - glob = List("crates/**/*.rs") - check = "mise run no-doctests" - } - // The third half of the same non-negotiable. CONTRIBUTING.md's license table - // ends "Confirm each _to confirm_ entry before that tool is adopted in a - // shipped release" — a release precondition stated in prose, which nothing - // enforced while three of five rows sat unresolved (CLOUD-325). The table - // stays the authority; this only asserts that no row is still an open - // question. It cannot check that a recorded license is *correct* — that is a - // human reading an upstream LICENSE — so it proves the narrower thing an exit - // code can carry: nobody shipped while the question was open. - ["license-table-check"] { - check = "mise run license-table-check" - } - // The mechanism CLOUD-847 shipped without (CLOUD-860). That row landed - // `.coderabbit.yaml` after measuring what each key buys — a formal - // CHANGES_REQUESTED review so a verdict exists at all, review on drafts so it - // exists in the phase that costs nothing, and `gitleaks` kept because - // `mise run ci` is `if: draft == false` and a draft would otherwise have no - // secret scanning. Nothing then held the file to those readings, and the - // failure is silent in the worst direction: reviews simply stop, which looks - // exactly like nobody having pushed. - // - // Globbed on the one file it reads, unlike its glob-less neighbours above: a - // commit touching nothing else cannot change the answer, so the whole-index - // scan they need would only cost time here. - ["coderabbit-config-check"] { - glob = List(".coderabbit.yaml") - check = "mise run coderabbit-config-check" - } - // The two-tier gate ships with its mechanism (non-negotiable 2). The steps - // below marked `profiles = List("slow")` are skipped at pre-commit and run by - // `check`, and the dangerous direction is not the obvious one: a tier that - // stops being SKIPPED merely makes commits slow, while a tier that stops being - // RUN by `check` removes clippy, the test suite and `batten-check` from - // `mise run ci`, `verify` and CI at once — green everywhere, nothing tested. - // Deleting the config-level `profiles` line does exactly that, silently. This - // reds it, reading the plan rather than a timing, since a timing cannot tell a - // skipped step from a cached one (CLOUD-509). - // - // Globbed on both files that can move the verdict: hk.pkl declares the tier, - // and the hook is the one place it is switched off. A glob on hk.pkl alone - // would let the `--profile` flag be deleted from the hook with nothing red — - // the step would not even run for that commit. - // The step keeps its name and its glob and records rather than decides: the - // verdict is `policy/hook-profile.rego`'s now, off the `hk-plan` row - // (CLOUD-509, retired under CLOUD-1199). Only the plan half needs a producer — - // `hook-missing-profile-flag` reads this step's other globbed file directly. ["hook-profile-check"] { glob = List("hk.pkl", ".claude/hooks/git-hook.sh") check = "mise run record-verdicts" @@ -247,42 +70,6 @@ local gate = new Mapping { profiles = List("slow") check = "mise run hk-drift" } - // `rules/rust.md` carries no module tree — it defers to mem:core as - // the authority kept current instead. Nothing checked that authority, so a - // module could land with no row and leave the rule pointing at an incomplete - // map; `severity.rs` (CLOUD-168) did exactly that, past a green gate. - // memories-check does not cover this: it gates the graph's *edges*, which - // stay coherent while the map is missing rows (CLOUD-194). - ["module-map-check"] { - glob = List("crates/*/src/*.rs", ".serena/memories/core.md") - check = "mise run module-map-check" - } - // `rules/toolchain.md` forbids restating a value three times over and - // then restated four, two of which had drifted — `LAND_MAX_LAPS` quoted as 8 - // where `land` defaults to 2, and `contract-drift` described as running on an - // event nothing wires (CLOUD-506). These files are read by an agent that then - // acts without re-deriving, so a stale parenthetical is a false premise - // carrying the rule's authority. It fails a claim that is present and WRONG, - // never one that is absent: the file's own rule is to defer, and a gate - // pushing toward completeness would invert it. - // CLOUD-479. Same glob as `rules-drift` below and for the same reason: the - // verdict moves when a registration moves, when a task's tool use changes, or - // when `mise.toml` pins something new. Sub-second, so it declares no profile — - // `hook-profile-check`'s `slow-tier-empty` rule would refuse one. - ["hook-pin-check"] { - glob = List(".claude/settings.json", "mise-tasks/*", "mise.toml") - check = "mise run hook-pin-check" - } - // CLOUD-480. The census, never `mutant` itself: this step decides whether every - // gate in the tree is DECLARED, which is a property of the commit and costs one - // pass over the `#MISE description` lines. `mutant` proves the declarations - // DISCRIMINATE, which is a property of the suites and costs two filtered bats - // runs per row, so it stays off the landing path — the same split - // `lock-complete` and `lock-currency` were separated along. - // - // The glob is the three surfaces the verdict is a function of: a gate added or - // renamed, a module added, and the set itself. Without `mise.toml` a commit that - // only shortened the set would not fire the step that reads it. ["mutant-census"] { glob = List("mise-tasks/**", "policy/*.rego", "mise.toml") check = "mise run mutant-census" @@ -524,38 +311,6 @@ local gate = new Mapping { ) check = "mise run install-check" } - // Publishing to a registry with a long-lived token is the thing OIDC exists to - // retire, and today this repository does neither — `publish = false`, and no - // registry secret exists (CLOUD-109). That makes the rule an implication - // rather than an edit: while publishing is off the OIDC permission is not - // required, and the commit that turns publishing ON must carry - // `id-token: write` or fail here. Globbed on the config that decides it and on - // every workflow, since a registry credential is refused wherever it appears. - ["publish-credential-check"] { - glob = List("release-plz.toml", ".github/workflows/*") - check = "mise run publish-credential-check" - } - // The release job publishes a tag; this is what keeps it telling Linear so - // (CLOUD-618). Both invocations must be present, SHA-pinned, and bound to the - // tag, over a checkout deep enough to carry the commit range the action reads — - // each of those failing produces a GREEN job and either no release record or - // one with no issues attached, which is why prose could not hold it. The - // release path must also REFRESH tags before resolving one, and that assertion - // is an ordering rather than a presence: the tag is pushed from a clone the - // runner cannot see, so a refresh placed after the resolver is the workflow - // that ran 32 times and recorded nothing. Wiring it here rather than as a - // CI-only step is what puts it in `mise run ci` -> `verify` -> CI, so - // `ci-parity`'s "every task CI runs is one `verify` runs" keeps holding. - // - // Globbed on BOTH workflows it judges as well as the script that decides, for - // the reason the timeout budget below states: a gate that does not re-run when - // its subject is edited stops running for exactly the commits it exists to - // judge. `linear-release-backfill.yml` is the second subject — the operator - // path that records a tag which shipped before the recording worked — and it - // needs its own trigger here or it would be judged only when the release path - // happened to change. The batten.toml row's glob names only the script — that - // one is the rule's subject, and `batten-glob-check` must be able to prove it - // covered. ["release-tracking-check"] { glob = List( @@ -574,36 +329,6 @@ local gate = new Mapping { glob = List(".claude/settings.json") check = "mise run mcp-allow-check" } - // The same silence, one table over in the same file (CLOUD-471). The - // `PreToolUse` matcher decides which tool calls the host spawns the hook for - // AT ALL, so a `[[verb]]` naming a tool the matcher omits loads, validates and - // gates nothing — the process is never spawned, no envelope is decoded, and - // the symptom is an allow indistinguishable from a pass. `hooks-wiring-check` - // above deliberately compares only the EVENT and the COMMAND ("THE MATCHER IS - // NOT COMPARED"), because a matcher derived from the `Harness` enum would be - // the repo-agnostic core asserting a consumer's tool vocabulary; the coverage - // question therefore lands in a consumer gate, which is this one. - // - // A pattern reaching awk through `-v` goes through assignment escape - // processing first, and what that does to a backslash is undefined across - // implementations — gawk strips it, mawk keeps it. ready-lint matched its §8 - // label that way: green on mawk locally, matching NOTHING on the gawk runner, - // so the clause went back to passing silently and took three older tests with - // it. A gate that cannot match its own label does not fail, it passes. - ["awk-regex-check"] { - glob = List("mise-tasks/**", "*.sh") - check = "mise run awk-regex-check" - } - // `producer | grep -q P` under pipefail can report FAILURE on a match: grep - // exits at the first hit, a still-writing producer dies of SIGPIPE, and - // pipefail promotes 141. It is a race (2 in 300 runs on a two-commit range), - // so it passes review and passes tests. landed-check reported a clean board - // over three landed issues that way; issue-guard denied `gh pr ready` on a - // branch whose every commit carried Refs: CLOUD-186 — it blocked its own PR. - ["pipefail-grep-check"] { - glob = List("mise-tasks/**", "*.sh") - check = "mise run pipefail-grep-check" - } // Completeness of mise.lock, asked of the committed bytes. The old lock-check // asked it by running `mise lock` and diffing, which answers "did upstream // change" instead — and never removes or repairs an existing entry, so a @@ -618,71 +343,10 @@ local gate = new Mapping { glob = List("mise.lock", "mise.toml", ".github/workflows/*.yml") check = "mise run lock-complete" } - // Every job already declared a timeout, so the absence of a limit was never - // the problem — the numbers were boilerplate, at 3.4x to 75x of measured p95. - // A cap at 75x cannot fail a job that is merely broken-slow, and it is inert - // as feedback: `ci` could double and every gate would stay green. So this - // gates the JUSTIFICATION, not the limit: a budget comment beside each value, - // either `grandfathered` (dated debt, visibly not a justification) or a - // measured `p95=s x` whose arithmetic must agree with the declared - // minutes. Whether a budget still matches reality is a property of the world - // and runs on a clock instead (timeout-drift), the lock-complete split. - ["timeout-check"] { - glob = List(".github/workflows/*.yml") - check = "mise run timeout-check" - } - // We pin an UNRELEASED toolchain-install commit — - // uses: jdx/mise-action@9dda3952d607125725deac9ec10a5f0e245d266b — because it is - // the one that added the download retry, and that action ships its own built - // dist. The pin therefore no longer matches a release tag — while renovate.json5 - // tracks the github-actions ecosystem and auto-bot-land.yml lands bot bumps - // with no human in the loop. A bump resolving back to the pre-retry commit - // would be a silent downgrade, auto-landed, and the next transient would read - // as fresh. So the rule ships with its mechanism: this reds such a pin at check - // time, which is what stops that PR going green (CLOUD-404). A denylist rather - // than a required sha, so a real forward bump needs no edit here. - ["mise-action-floor"] { - glob = List(".github/workflows/*.yml") - check = "mise run mise-action-floor" - } - // Every agent pays AGENTS.md on every turn, so its size is a per-turn tax - // rather than a style question. Counted with any memory declared always-load, - // since moving a section into one of those moves the cost without cutting it. - // - // This was `mise run context-budget`, a ~50-line shell task, until CLOUD-50 - // moved the predicate into the engine. `batten.toml` now carries the counted - // set and both thresholds, so `batten.toml` joins the glob: changing the - // budget must re-run the gate that enforces it. - // - // `.serena/project.yml` joins it for the same reason (CLOUD-899): - // `[[budget.instructions.embedded]]` counts its `initial_prompt`, so it is a - // real input, and a step that does not list its own input stops running for - // exactly the commits it is meant to judge — the shape the `check` tier's - // memory glob already records for CLOUD-770. `batten-check` globs it too, but - // that step is `slow`-profile and skipped at commit time, so it covered - // `verify` and `ci` while the pre-commit tier stayed blind. That went unnoticed - // while the value was empty; CLOUD-864 put text in it. - // - // What is deliberately NOT here: `.serena/memories/always/*.md`, carried over - // from the deleted shell task. That directory has never existed — batten.toml - // says so where it declares the counted set — and CLOUD-298 removed the - // engine-side copy while this one survived, matching nothing and reading to the - // next author as evidence of an `always/` convention. ["policy-budget"] { glob = List("AGENTS.md", ".serena/project.yml", "batten.toml") check = "mise run policy-budget" } - // The other half of `batten-check`'s glob (CLOUD-224). That list is derived - // from batten.toml — every `[[rule]]` glob, every declared budget file — so - // writing it here makes hk.pkl a second authority over a set the config - // already defines, and a second authority narrows SILENTLY: add a rule whose - // glob names a path outside the list and the step stops running for commits - // that touch only that path, with nothing going red. A glob without this is - // half a change (non-negotiable 2). - ["batten-glob-check"] { - glob = List("batten.toml", "hk.pkl") - check = "mise run batten-glob-check" - } // The shipped skill is the dispositional surface of the interaction model, and // two of its properties are specification rather than taste: a ~300-line // budget somebody measured (a 341-line skill beat a 2,187-line one head to @@ -950,7 +614,7 @@ local gate = new Mapping { // // Deriving that union from batten.toml by hand makes hk.pkl a second authority // over it, which narrows silently the next time a rule adds a glob — half a - // change by non-negotiable 2. `batten-glob-check` below is the other half. + // change by non-negotiable 2. `glob-containment` below is the other half. // // The task runs `batten enforce` since CLOUD-229 — the committed // no-conflict-markers rule delegates to `hk util check-merge-conflict`, and a @@ -980,6 +644,13 @@ local gate = new Mapping { "Cargo.lock", "mise.toml", ".github/workflows/*.yml", + // CLOUD-1709's two `workflow-shell-not-growing` rows glob + // `.github/workflows/**`, which is BROADER than the `*.yml` entry above + // and not redundant with it: the narrow entry misses a `.yaml` workflow + // and misses a nested path. A step whose glob does not cover its own + // inputs stops running for exactly the commits it is meant to judge, so + // `glob-containment` refused the rows until this followed them. + ".github/workflows/**", // Recursive since CLOUD-328: the ratchet row over bats suites was // narrowed to one segment only to dodge a submodule the walker used to // count and the base rev did not. Both halves stop at a nested @@ -988,12 +659,12 @@ local gate = new Mapping { // CLOUD-383's portability row scopes a `forbid` to `mise-tasks/**`, so // those files are an input to `batten check` now — and a step whose glob // does not cover its own inputs stops running for exactly the commits it - // is meant to judge. `batten-glob-check` refused the rule until this + // is meant to judge. `glob-containment` refused the rule until this // followed it, which is the coupling working rather than a chore. "mise-tasks/**", // CLOUD-1457's `install-does-one-thing` row scopes a `forbid` to // `install.sh`, so the installer is an input to `batten check` — the same - // coupling as `mise-tasks/**` above, and `batten-glob-check` refused the + // coupling as `mise-tasks/**` above, and `glob-containment` refused the // rule until this followed it. The consequence of omitting it is sharper // here than elsewhere: `install.sh` is the one file exempt from the bash // moratorium, so a commit touching only it is exactly the commit the rule @@ -1002,13 +673,13 @@ local gate = new Mapping { // CLOUD-876's `no-rego-metadata` row scopes a `forbid` to // `policy/**/*.rego`, so the modules are an input to `batten check` too // — the same coupling as `mise-tasks/**` above, and refused by - // `batten-glob-check` until it followed. Without it the gate goes silent + // `glob-containment` until it followed. Without it the gate goes silent // for exactly the commits that add a policy module, which is when the // `rego.metadata.*` refusal has anything to catch. "policy/**/*.rego", // CLOUD-761's `no-tracker-key-in-modules` row scopes a `forbid` to // `policy/**` — the whole tree rather than the `.rego` files above it — - // so the broader glob has to follow it here, and `batten-glob-check` + // so the broader glob has to follow it here, and `glob-containment` // refused the commit until it did. The two entries are not redundant: // every tracked path under `policy/` is a module today, so they select // the same set, and the day one is not, the narrow entry would leave the @@ -1024,21 +695,21 @@ local gate = new Mapping { // The two `command` rows over the Rego corpus are globbed on their own // configuration rather than on `policy/**`, so that they run here and in // no fixture (CLOUD-614) — which makes these files batten inputs in the - // literal sense `batten-glob-check` reads, and it refused the commit + // literal sense `glob-containment` reads, and it refused the commit // until they were listed. ".regal/config.yaml", "schema/policy-input.schema.json", "AGENTS.md", ".serena/project.yml", // CLOUD-681's `fix-selection-complete` globs this very file, so editing - // the gate is now a reason to run the gate. `batten-glob-check` is what + // the gate is now a reason to run the gate. `glob-containment` is what // named it — the row landed, the list did not, and that suite went red // on the same commit rather than the step quietly not selecting. "hk.pkl", // CLOUD-1137's `claude-shell-not-growing` ratchets shebangs under // `.claude/**`, so those files are a `batten check` input now — the same // coupling `mise-tasks/**` and `policy/**/*.rego` above carry, and - // `batten-glob-check` refused the commit until this followed the row. It + // `glob-containment` refused the commit until this followed the row. It // is the entry that matters most for that row: a new shell program under // `.claude/` is exactly the commit the ratchet exists to catch, and // without this it is the commit the step would not run for. @@ -1046,7 +717,7 @@ local gate = new Mapping { // CLOUD-605's `no-denied-identity-prescribed` row scopes a `forbid` to // Markdown — where the user-level hook's remedy would get pasted — so // every tracked `.md` is an input now. The bare `**` below does not - // discharge this: `batten-glob-check`'s subsumption is a `P/**` prefix + // discharge this: `glob-containment`'s subsumption is a `P/**` prefix // test and a slashless `**` matches only verbatim, so an explicit entry // is what the gate accepts. It is also what this list wants — each line // records WHICH rule made its path an input. diff --git a/man/batten-exec.1 b/man/batten-exec.1 index 87c107187..74eb9a1ff 100644 --- a/man/batten-exec.1 +++ b/man/batten-exec.1 @@ -4,7 +4,7 @@ .SH NAME batten\-exec \- Run a command — or a `:::` bundle — and report a pointer to what it wrote .SH SYNOPSIS -\fBbatten exec\fR [\fB\-\-capture\-only\fR] [\fB\-\-tee\fR] [\fB\-\-jobs\fR] [\fB\-\-continue\-on\-error\fR] [\fB\-\-format\fR] [\fB\-\-style\fR] [\fB\-h\fR|\fB\-\-help\fR] <\fIcommand\fR> +\fBbatten exec\fR [\fB\-\-capture\-only\fR] [\fB\-\-tee\fR] [\fB\-\-jobs\fR] [\fB\-\-continue\-on\-error\fR] [\fB\-\-lock\fR] [\fB\-\-lock\-path\fR] [\fB\-\-lock\-attempts\fR] [\fB\-\-lock\-label\fR] [\fB\-\-format\fR] [\fB\-\-style\fR] [\fB\-h\fR|\fB\-\-help\fR] <\fIcommand\fR> .SH DESCRIPTION Run a command — or a `:::` bundle — and report a pointer to what it wrote .SH OPTIONS @@ -21,6 +21,18 @@ How many of a `:::` bundle\*(Aqs commands run at once \fB\-\-continue\-on\-error\fR Run the rest of a `:::` bundle after a command fails .TP +\fB\-\-lock\fR +Hold this clone\*(Aqs named singleton lock for the child\*(Aqs lifetime +.TP +\fB\-\-lock\-path\fR +Hold the lock at this path, for a resource the clone does not own +.TP +\fB\-\-lock\-attempts\fR +How many times to ask for the lock before reporting it held +.TP +\fB\-\-lock\-label\fR +What the wait is for, named by the caller for the refusal line +.TP \fB\-\-format\fR [default: human] How Batten\*(Aqs own record is encoded (hk\*(Aqs axis) .br diff --git a/man/batten-verdict.1 b/man/batten-verdict.1 new file mode 100644 index 000000000..bcab5fcbf --- /dev/null +++ b/man/batten-verdict.1 @@ -0,0 +1,19 @@ +.ie \n(.g .ds Aq \(aq +.el .ds Aq ' +.TH batten-verdict 1 batten +.SH NAME +batten\-verdict \- Fold a run\*(Aqs findings and blind spots into this tool\*(Aqs exit code +.SH SYNOPSIS +\fBbatten verdict\fR [\fB\-\-findings\fR] [\fB\-\-unjudgeable\fR] [\fB\-h\fR|\fB\-\-help\fR] +.SH DESCRIPTION +Fold a run\*(Aqs findings and blind spots into this tool\*(Aqs exit code +.SH OPTIONS +.TP +\fB\-\-findings\fR +How many blocking findings the run produced +.TP +\fB\-\-unjudgeable\fR +How many subjects the run could not read +.TP +\fB\-h\fR, \fB\-\-help\fR +Print help diff --git a/man/batten.1 b/man/batten.1 index 311d6c802..6e3aac9f9 100644 --- a/man/batten.1 +++ b/man/batten.1 @@ -103,6 +103,9 @@ Decide whether this repository\*(Aqs gates discriminate, rather than merely pars batten\-policy(1) Inspect the thresholds and path sets this repository holds itself to .TP +batten\-verdict(1) +Fold a run\*(Aqs findings and blind spots into this tool\*(Aqs exit code +.TP batten\-commit(1) The shape a commit must take here: what its subject may say .TP diff --git a/mise-tasks/awk-regex-check.sh b/mise-tasks/awk-regex-check.sh deleted file mode 100755 index 3f6c66e87..000000000 --- a/mise-tasks/awk-regex-check.sh +++ /dev/null @@ -1,62 +0,0 @@ -#!/usr/bin/env bash -#MISE description="Gate: no shell task hands awk a regex through -v, where escape handling is implementation-defined" -# -# A pattern passed through `awk -v` goes through the assignment's escape -# processing before awk ever sees it as a regex, and what that does to a -# backslash is not defined across implementations. gawk strips `\(` to `(` with -# a warning; mawk keeps it. So the same pattern is a literal paren on one -# machine and a capturing group on the other. -# -# This is not theoretical. `ready-lint` matched its §8 label that way. It worked -# on mawk here and, on the gawk runner, matched NOTHING — so the clause that -# exists to catch a blocker claimed without a relation went back to passing -# silently, and three tests that predated the change went red with it. A gate -# that cannot match its own label does not fail; it passes. -# -# The predicate is the USE, not the value: a literal with no backslash is safe -# today and unsafe the moment someone adds one, and a variable's runtime content -# is invisible to any static check. So this flags a `-v` name that the awk -# program then uses as a regex — `~ name` or `match(…, name)` — regardless of -# what the value looks like at the call site. -# -# `-v` for a plain VALUE stays fine, which is most of its use: comparing with -# `==`, printing, arithmetic. Only regex position is judged. -# -# The fix is always the same shape and needs no new tool: let `grep` find what -# the pattern matches and let awk work in literal patterns, or inline the pattern -# in the awk program where no assignment processing happens. -# A gate listed in $MUTANT_GATES with no row here fails `mise run mutant`. -#MUTANT regex-through-v-passes|s/^exit "\$fail"$/exit 0/|a -v name used with ~ is reported - -set -euo pipefail - -fail=0 -report() { - [[ "$fail" = 0 ]] && echo "::error:: a regex reaches awk through -v, where escape handling is implementation-defined (see mem:toolchain-and-hooks):" >&2 - printf ' %s\n' "$1" >&2 - fail=1 -} - -while IFS= read -r hit; do - [[ -n "$hit" ]] || continue - # -H, not -n alone: grep omits the filename when handed a single path, which - # silently turns the pointer into "lineno:text" and misreports the location. - file=${hit%%:*} - rest=${hit#*:} - lineno=${rest%%:*} - text=${rest#*:} - - # Every -v name assigned on this line. - for name in $(grep -oE '(^|[[:space:]])-v[[:space:]]*[A-Za-z_][A-Za-z0-9_]*=' <<<"$text" | - grep -oE '[A-Za-z_][A-Za-z0-9_]*=' | tr -d '=' | sort -u); do - # Used in regex position? `$0 ~ name`, `x ~ name`, or match(s, name). - if grep -qE "~[[:space:]]*$name([^A-Za-z0-9_]|$)" <<<"$text" || - grep -qE "match\([^)]*[,[:space:]]$name([^A-Za-z0-9_]|\))" <<<"$text"; then - report "$file:$lineno: \`$name\` is assigned with -v and used as a regex" - fi - done -done < <(git ls-files -z 'mise-tasks/*' '*.sh' 'mise.toml' 2>/dev/null | - xargs -0 grep -HnI 'awk' 2>/dev/null | grep -- '-v' || true) - -[[ "$fail" = 0 ]] && echo "awk-regex-check: no regex reaches awk through -v" -exit "$fail" diff --git a/mise-tasks/batten-glob-check.sh b/mise-tasks/batten-glob-check.sh deleted file mode 100755 index c34df7946..000000000 --- a/mise-tasks/batten-glob-check.sh +++ /dev/null @@ -1,190 +0,0 @@ -#!/usr/bin/env bash -#MISE description="Gate: hk.pkl's batten-check glob covers every path batten.toml makes an input — a pure function of the two committed files" -# -# CLOUD-224. `batten-check` used to carry no glob at all, on the reasoning that -# any file can carry a violation, so `cargo run -p batten -- check` rebuilt the -# engine on every commit whatever it touched. Giving it a glob is what makes a -# docs-only commit cheap; the cost of a glob is that the step's trigger is now a -# SECOND authority over a set `batten.toml` already defines, and a second -# authority narrows silently. -# -# The failure that would produce is the one the whole gate model exists to -# prevent: add a `[[rule]]` whose glob names a path outside the list, and the -# gate simply stops running for commits that touch only that path. Nothing goes -# red. `hk check --all` still covers it in CI, so the symptom is a rule that -# quietly does not gate a branch — feedback deleted, verdict preserved. -# -# So this asserts the containment directly, of the committed bytes. What -# `batten check` reads is three things (crates/batten/src/lib.rs, run_check_with): -# -# 1. every `[[rule]]`, over that rule's own `glob` -# 2. `budget::measure_all` — `[budget.instructions] files` and every -# `[[budget.instructions.embedded]] path`. A declared budget is a gate under -# `check`, not only under `policy budget` (CLOUD-50), which is why AGENTS.md -# is an input and "Markdown cannot change this verdict" is false. -# 3. `defects::gate`, which reads a ledger path only when [defects] is declared -# -# 1 and 2 are what this gate reads out of batten.toml. 3 declares no path in this -# repository today; a [defects] table that names one is the case to extend here. -# -# NOT checked, and not checkable by a glob at all: a `ratchet` rule's verdict -# also moves when its `base` (origin/main) moves, with no file in this repository -# changing. That is a property of the world, and `hk check --all` in CI is what -# covers it — the same split `lock-complete`/`lock-currency` documents. -# -# Output is a pointer, never a payload (non-negotiable 4): the uncovered glob and -# the file that demands it. -# A gate listed in $MUTANT_GATES with no row here fails `mise run mutant`. -#MUTANT uncovered-glob-passes|s/^\tif \[\[ "\$ok" = 0 \]\]; then$/\tif false; then/|absent from the list is caught - -set -euo pipefail - -# Both inputs are ARGUMENTS defaulting to the real files, for the same reason -# `ci-tools-check` takes both of its: the decision is the part worth testing, and -# it only tests if the suite can point it at fixtures carrying drift the real -# tree must never have. -cd "$(git rev-parse --show-toplevel)" - -readonly CONFIG="${1:-batten.toml}" -readonly HOOKS="${2:-hk.pkl}" - -for f in "$CONFIG" "$HOOKS"; do - if [[ ! -f "$f" ]]; then - echo "::error:: batten-glob-check: $f not found" >&2 - exit 2 - fi -done - -# --- what batten.toml makes an input --------------------------------------- -# -# Three shapes, one per table that can name a path. Each prints `\t` -# so a finding can point at the line that demands it. -# -# `[[rule]]` blocks: the `glob` key. A rule with no `glob` selects nothing extra. -# `[budget.instructions]`: `files = [...]`, a one-line array in this config and -# gated as such — a multi-line array would need continuation tracking, and the -# `no-budget-entries` guard below is what notices if one ever appears. -# `[[budget.instructions.embedded]]`: the `path` key. -required=$(awk ' - /^\[\[rule\]\]/ { table = "rule"; next } - /^\[budget\.instructions\]/ { table = "budget"; next } - /^\[\[budget\.instructions\.embedded\]\]/ { table = "embedded"; next } - /^\[/ { table = ""; next } - table == "rule" && /^glob = / { - line = $0; sub(/^glob = /, "", line); gsub(/"/, "", line) - print line "\t" NR - next - } - table == "embedded" && /^path = / { - line = $0; sub(/^path = /, "", line); gsub(/"/, "", line) - print line "\t" NR - next - } - table == "budget" && /^files = \[/ { - line = $0 - sub(/^files = \[/, "", line); sub(/\].*$/, "", line) - n = split(line, parts, ",") - for (i = 1; i <= n; i++) { - gsub(/^[ \t]+|[ \t]+$/, "", parts[i]) - gsub(/"/, "", parts[i]) - if (parts[i] != "") print parts[i] "\t" NR - } - next - } -' "$CONFIG") - -# A config this gate can parse nothing out of is not a config with no inputs — -# it is a parse that failed, and passing on it would be the vacuous green a -# containment check can most easily produce. -if [[ -z "$required" ]]; then - echo "::error:: batten-glob-check: parsed no rule glob or budget path out of $CONFIG — a config batten check reads nothing from is not a thing this repo has" >&2 - exit 2 -fi - -# --- what hk.pkl's batten-check step selects -------------------------------- -# -# The `glob = List(...)` belonging to the `["batten-check"]` step, and only that -# one: the file carries a dozen others. Bounded by the step's own header and the -# next step's, so a later step's list cannot be read as this step's. -covered=$(awk ' - /^ \["batten-check"\]/ { in_step = 1; next } - in_step && /^ \["/ { in_step = 0 } - in_step && /glob =/ { in_glob = 1 } - # A COMMENT INSIDE THE LIST IS NOT LIST SYNTAX, and reading it as such made - # this gate lie. The entries carry a comment each recording which rule made - # the path an input; one of them contained `(CLOUD-614)`, whose `)` ended the - # list here — so every entry BELOW it went uncovered and the gate reported - # four paths that were listed all along. A containment check that mis-parses - # in the reporting direction is survivable; the same parse silently DROPPING - # entries from `required` would not be, which is why this skips rather than - # tries to be clever about the paren. - in_glob && /^[ \t]*\/\// { next } - in_glob { - # One entry per quoted string, however the list is wrapped: pkl format - # breaks a long List across lines, so neither a one-line nor a - # one-per-line shape can be assumed. - rest = $0 - while (match(rest, /"[^"]*"/)) { - print substr(rest, RSTART + 1, RLENGTH - 2) - rest = substr(rest, RSTART + RLENGTH) - } - if (index($0, ")") > 0) in_glob = 0 - } -' "$HOOKS") - -if [[ -z "$covered" ]]; then - echo "::error:: batten-glob-check: found no \`glob = List(...)\` on the [\"batten-check\"] step in $HOOKS. A glob-less step runs on every commit — which is what CLOUD-224 removed, so its absence is a regression, not a default." >&2 - exit 1 -fi - -# --- containment ------------------------------------------------------------ -# -# Covered means: present verbatim, or subsumed by a `P/**` entry whose prefix the -# required glob starts with. That second clause is the whole reason the list can -# stay short — one `crates/**` stands for `crates/**/*.rs`, -# `crates/batten/tests/**` and `crates/batten/tests/**/*.rs`. -# -# It is deliberately NOT general glob subsumption, which is undecidable in the -# directions that matter and would be a matcher this repo would then have to own. -# A prefix test is the narrow, honest case; anything it cannot prove must be -# listed verbatim, which fails CLOSED — the direction a containment check has to -# fail in. -fail=0 -reported=0 -while IFS=$'\t' read -r want line; do - [[ -n "$want" ]] || continue - - ok=0 - while IFS= read -r have; do - [[ -n "$have" ]] || continue - if [[ "$have" = "$want" ]]; then - ok=1 - break - fi - case $have in - */'**') - prefix=${have%'**'} - case $want in - "$prefix"*) - ok=1 - break - ;; - esac - ;; - esac - done <<<"$covered" - - if [[ "$ok" = 0 ]]; then - if [[ "$reported" = 0 ]]; then - echo "::error:: hk.pkl's batten-check glob does not cover every path batten.toml makes an input, so the gate silently stops running for commits that touch only these (CLOUD-224):" >&2 - reported=1 - fi - echo " $CONFIG:$line: \`$want\` — add it to the \`glob = List(...)\` on the [\"batten-check\"] step in $HOOKS" >&2 - fail=1 - fi -done <<<"$required" - -if [[ "$fail" = 0 ]]; then - echo "batten-glob-check: $HOOKS's batten-check glob covers every path $CONFIG makes an input" -fi -exit "$fail" diff --git a/mise-tasks/cap-drift.sh b/mise-tasks/cap-drift.sh deleted file mode 100755 index 81f9e7831..000000000 --- a/mise-tasks/cap-drift.sh +++ /dev/null @@ -1,166 +0,0 @@ -#!/usr/bin/env bash -#MISE description="Gate: an MSRV cap in Cargo.toml and its bot-side `allowedVersions` rule exist together or not at all (CLOUD-593)" -# -# THE HALF-LIFT, which is the one failure mode in this change that lands GREEN. -# -# An MSRV cap lives in two files and nothing kept them in step. `Cargo.toml` -# carries the upper bound; the bot config restates it, because the bot cannot -# read a manifest cap and would otherwise propose a bump the manifest has already -# refused — one unbuildable member reddening a whole grouped batch (CLOUD-344). -# -# THE BOT SIDE MOVED FILES, NOT MEANING (CLOUD-660). It was -# `.github/dependabot.yml`'s `ignore: - dependency-name:` list; that file is -# deleted with the bot it configured, and the mirror is now `renovate.json5`'s -# `packageRules[].allowedVersions` — Renovate's spelling of the same withholding. -# Both sets are empty at the commit that moves them, so this is a repoint with no -# change of verdict, which is the only kind of repoint a ratchet can survive. -# -# Lift only the manifest side and everything passes. The manifest now admits -# 0.4.33, the bot still withholds it, nothing proposes it ever again, and no -# check anywhere is red. The freeze survives the change that was supposed to end -# it, and the issue closes. That is worse than the original defect, because the -# original had a symptom. -# -# So the predicate is set equality, in BOTH directions, and both directions are -# real: -# -# cap-without-ignore the manifest refuses a version the bot will propose, -# so the batch goes red the way CLOUD-344 measured. -# ignore-without-cap the bot withholds a version the manifest admits, so -# the crate silently never advances. This is the half- -# lift, and it is the direction with no symptom. -# -# STILL LOAD-BEARING WITH BOTH SETS EMPTY, which is the state CLOUD-593 leaves -# them in. It is a ratchet: the next cap added to either file has to be added to -# the other, and the gate says so at the commit that adds it rather than at the -# release that notices the freeze. A gate that only functions while a list is -# non-empty would have been deleted with the list. -# -# WHAT COUNTS AS A CAP, and the narrowness is deliberate. Only an upper bound on -# a `[workspace.dependencies]` entry — `<`, `<=`, or a `,`-joined range carrying -# one. A bare `"0.4"` is a caret requirement, which bounds the major and is not -# an MSRV decision; treating it as one would demand a dependabot entry for every -# dependency in the file and the gate would be switched off within a day. -# -# Pointer-only per non-negotiable rule 4: the crate name and which file its -# counterpart is missing from. Never a version range, never a manifest line. -# -# Exit 0 the two sets agree / 1 they differ / 2 could not look. -# -# The mutation drops the second direction, leaving only cap-without-ignore. That -# is the direction that already has a symptom — CI goes red on its own — so the -# suite still passes every row except the half-lift, which is the row this gate -# exists for and the only one that lands green unaided. -#MUTANT only-checks-cap-without-ignore|s/^for crate in \$ignored; do$/for crate in ; do/|THE HALF-LIFT: an allowedVersions rule with no cap is refused, and named -#PIN-OK: jq -set -uo pipefail - -manifest="${CAP_DRIFT_MANIFEST:-Cargo.toml}" -bot="${CAP_DRIFT_RENOVATE:-renovate.json5}" - -while [[ $# -gt 0 ]]; do - case "$1" in - --manifest) - manifest="${2:-}" - shift 2 - ;; - --renovate) - bot="${2:-}" - shift 2 - ;; - *) - echo "usage: cap-drift [--manifest ] [--renovate ]" >&2 - exit 2 - ;; - esac -done - -for f in "$manifest" "$bot"; do - if [[ ! -r "$f" ]]; then - echo "::error:: cap-drift: cannot read $f — a gate that cannot look must not report agreement" >&2 - exit 2 - fi -done - -# Capped crates: a `[workspace.dependencies]` line whose version requirement -# carries a `<`. Read from the section alone, so a `<` in a comment or in another -# table cannot mint a phantom cap. -capped=$(awk ' - /^\[workspace\.dependencies\]/ { in_deps = 1; next } - /^\[/ { in_deps = 0 } - in_deps && /^[a-zA-Z0-9_-]+[[:space:]]*=/ { - name = $1 - line = $0 - if (line ~ /version[[:space:]]*=[[:space:]]*"[^"]*<[^"]*"/ || line ~ /^[a-zA-Z0-9_-]+[[:space:]]*=[[:space:]]*"[^"]*<[^"]*"/) { - print name - } - } -' "$manifest" | sort -u) - -# Withheld crates: every name in a `packageRules` entry that carries -# `allowedVersions`. Scoped to the RULE rather than to the file, so a -# `matchPackageNames` used for grouping — which withholds nothing — is not read -# as a cap mirror. `//` comments are stripped first, for the reason -# `ci-local-parity` gives: that file argues for its keys at length, and a gate a -# comment can satisfy is a gate satisfied by deleting the key the comment -# explains. The strip requires the `//` at line start or after whitespace, so the -# `https://` inside `$schema` survives. -# -# Rules are found by BRACE DEPTH inside the `packageRules` array rather than by -# line shape, so a rule written inline and one spread over five lines read the -# same — a formatter's choice must not change a verdict. -ignored=$(sed -e 's|^//.*$||' -e 's|[[:space:]]//.*$||' "$bot" | awk ' - /packageRules[[:space:]]*:/ { inrules = 1 } - !inrules { next } - { - line = $0 - while (length(line) > 0) { - c = substr(line, 1, 1) - line = substr(line, 2) - if (c == "{") { - depth++ - if (depth == 1) buf = "" - } - if (depth >= 1) buf = buf c - if (c == "}") { - depth-- - if (depth == 0) { - if (buf ~ /allowedVersions/) { - names = buf - sub(/.*matchPackageNames[^[]*\[/, "", names) - sub(/\].*$/, "", names) - gsub(/["\x27[:space:]]/, "", names) - n = split(names, parts, ",") - for (i = 1; i <= n; i++) if (parts[i] != "") print parts[i] - } - buf = "" - } - } - } - } -' | sort -u) - -violations=0 -report() { - echo "::error:: cap-drift: $1" >&2 - violations=$((violations + 1)) -} - -for crate in $capped; do - grep -qx -- "$crate" <<<"$ignored" || - report "$crate is capped in $manifest and has no allowedVersions rule in $bot — the bot will propose a version the manifest refuses, and one unbuildable member reddens the whole grouped batch (CLOUD-344)" -done - -for crate in $ignored; do - grep -qx -- "$crate" <<<"$capped" || - report "$crate has an allowedVersions rule in $bot and no cap in $manifest — the bot withholds a version the manifest admits, so it silently never advances and nothing goes red. This is the half-lift (CLOUD-593)" -done - -if [[ "$violations" != 0 ]]; then - exit 1 -fi - -n=0 -[[ -n "$capped" ]] && n=$(grep -c . <<<"$capped") -echo "cap-drift: $n MSRV cap(s) in $manifest, each mirrored in $bot" -exit 0 diff --git a/mise-tasks/ci-tools-check.sh b/mise-tasks/ci-tools-check.sh deleted file mode 100755 index 4fea0b3ad..000000000 --- a/mise-tasks/ci-tools-check.sh +++ /dev/null @@ -1,263 +0,0 @@ -#!/usr/bin/env bash -#MISE description="Gate: install_args names resolve to real [tools] entries, and no PR workflow runs mise-action without a binding list" -# -# CI installs a NARROW tool set per job (see .github/workflows/ci.yml): the `ci` -# job takes the hk gate's tools, `cross` takes rust, `darwin-link` takes rust + -# zig + cargo-zigbuild. That is the largest single CI speed-up available — every -# job used to install all 18 tools, and zig (17.3s) set the wall clock in three -# jobs that never invoke it. -# -# The cost of narrowing is a second place where tool names are written down, and -# the failure mode is drift: rename or re-pin a tool in mise.toml and the -# install_args list silently keeps the old name. `mise install` does NOT fail on -# an unknown tool name, so the drift surfaces much later as "command not found" -# in whichever step happened to need it — a missing TOOL wearing a missing-STEP -# costume, which is the same class of late, misattributed failure `doctor` -# exists to kill. -# -# This is that rule's mechanism (AGENTS.md non-negotiable 2): a computable -# predicate over the two files, exit 0 or 1. It asserts NAMES resolve, which is -# the half that can rot silently. It deliberately does not try to infer which -# tools a job "should" need — that is a judgement, not a predicate, and the -# comments in ci.yml carry the reasoning. -# -# Output is a pointer, never a payload (non-negotiable 4): the tool name and the -# file it is missing from. -set -uo pipefail - -# Both inputs are ARGUMENTS defaulting to the real files, for the same reason -# doctor-check takes `installed` as an argument: the decision is the part worth -# testing, and it only tests if the suite can point it at fixtures carrying -# drift the real tree must never have. -# `|| exit 1` rather than `set -e`: the checks below deliberately let non-zero -# exits through (an empty `grep` is a real outcome here, not a fatal error), so -# this script cannot take -e the way doctor does. -cd "$(git rev-parse --show-toplevel)" || exit 1 - -readonly WORKFLOW="${1:-.github/workflows/ci.yml}" -readonly CONFIG="${2:-mise.toml}" -# The second pass's input: a DIRECTORY, because absence is the thing a -# single-file gate structurally cannot see. -# -# It defaults to the first argument's own directory rather than to a literal -# path, and that is what keeps the two passes pointed at one tree. A caller -# aiming the gate at a fixture workflow is asking about that fixture; resolving -# the second pass to `.github/workflows` regardless would silently mix a -# fixture's verdict with the committed tree's, which is the one thing a suite -# built on fixtures must not do. With no arguments at all it resolves to -# `.github/workflows`, which is the real question. -readonly WORKFLOW_DIR="${3:-$(dirname "$WORKFLOW")}" -# The policy file the third pass reads, an ARGUMENT for the same reason -# WORKFLOW_DIR is: a caller aiming this at a fixture workflow is asking about -# that fixture, and reading the committed `batten.toml` against it would mix one -# tree's spawns with another's install list — the one thing a suite built on -# fixtures must not do. -readonly POLICY="${4:-batten.toml}" - -for f in "$WORKFLOW" "$CONFIG"; do - if [[ ! -f "$f" ]]; then - echo "::error:: ci-tools-check: $f not found" >&2 - exit 1 - fi -done - -# The [tools] table's keys, as mise sees them: everything from `[tools]` up to -# the next table header, left of the `=`, unquoted. Comments and blanks drop out -# because they carry no bare `key =`. -declared=$( - awk ' - /^\[tools\]/ { in_tools = 1; next } - /^\[/ { in_tools = 0 } - in_tools && /^[^#]/ { - eq = index($0, "=") - if (eq == 0) next - key = substr($0, 1, eq - 1) - gsub(/^[ \t]+|[ \t]+$/, "", key) - gsub(/^"|"$/, "", key) - if (key != "") print key - } - ' "$CONFIG" -) - -if [[ -z "$declared" ]]; then - echo "::error:: ci-tools-check: no [tools] entries parsed from $CONFIG" >&2 - exit 1 -fi - -# Every install_args value in the workflow, flattened to one tool name per line. -# Each list is a one-line plain scalar, which ci.yml states and this relies on: -# a YAML block scalar would need continuation-line tracking, and several tool -# names begin with `aqua:` so they read as new keys under any such heuristic. -requested=$( - awk ' - /^[ \t]*install_args:/ { - sub(/^[ \t]*install_args:[ \t]*/, "") - print - } - ' "$WORKFLOW" | tr -s '[:blank:]' '\n' | grep -v '^$' | sort -u -) - -if [[ -z "$requested" ]]; then - echo "::error:: ci-tools-check: no install_args lists found in $WORKFLOW — CI would install the whole toolchain in every job" >&2 - exit 1 -fi - -status=0 -count=0 -while IFS= read -r tool; do - count=$((count + 1)) - if ! grep -qxF "$tool" <<<"$declared"; then - echo "::error:: $WORKFLOW installs '$tool', which is not a [tools] entry in $CONFIG. CI would skip it and fail later as 'command not found'." >&2 - status=1 - fi -done <<<"$requested" - -# --------------------------------------------------------------------------- -# THE SECOND DIRECTION, AND WHY THE FIRST ONE COULD NOT ASK IT (CLOUD-812). -# -# Everything above judges the names IN a list. A workflow that declares no list -# at all has no names to judge, so it passes — and that is not a hole in the -# predicate, it is a hole in the trigger: the gate is absence-blind by -# construction, because it was pointed at one file that happened to be compliant. -# -# CLOUD-180 narrowed `ci.yml` and its Done list said so precisely — "per-job -# `install_args` in `ci.yml`, with auto-install off so the lists bind." Measured -# 2026-08-20, `commit-lint.yml` and `zizmor.yml` had never been in that scope and -# were still installing all 28 `[tools]` entries on every PR push, to run a -# commit-subject regex and one static analyzer. Nothing was red. Nothing could be. -# -# So this asks the question the other direction, over every workflow that spends -# a runner on a pull request: -# -# 1. Every `mise-action` step carries an `install_args` list. Counted -# rather than parsed per step: one list per use is the invariant, and a -# count is decidable without tracking YAML block structure — the same -# constraint the `requested` scan above already works within. -# 2. Both auto-install variables are set. A list without them is DECORATIVE: -# mise re-installs the missing tools at task time, which is the failure -# CLOUD-180 measured in the `cross` job (rust in 13s, then the whole -# toolchain rebuilt inside the work step) and which is indistinguishable -# from a fix by reading the workflow. -# -# Scheduled and `workflow_run` workflows are out of scope deliberately: they are -# not on the PR path, they are not what the ledger is measuring, and widening -# this to them would be a different decision with a different cost argument. -# -# Pointer-only (non-negotiable 4): the workflow path and which half is missing. -# Never the tool set, never a log line. -# The two declarations sit HERE, at column 0, rather than beside the arms they -# corrupt. `mutant` reads them with `sed -n 's/^#MUTANT //p'`, so an indented -# one is invisible to it — and `shfmt` reindents any comment inside a block, so -# a column-0 comment beside those arms cannot survive the formatter either. The -# slugs name which arm each one reverts. -#MUTANT spawned-tool-need-not-be-installed|s@^ if ! grep -qxF "\$tool" <<<"\$installed"; then$@ if false; then@|no list installs is refused -#MUTANT pr-workflow-may-omit-install-args|s@^\t\tif \[\[ "\$lists" -ne "\$uses" \]\]; then$@\t\tif false; then@|with no install_args fails -#MUTANT pr-workflow-list-may-be-nonbinding|s@^\t\tif ! grep -qE "\$binding_task".*@\t\tif false; then@|without the auto-install variables fails -if [[ -d "$WORKFLOW_DIR" ]]; then - pr_workflows=0 - for wf in "$WORKFLOW_DIR"/*.yml; do - [[ -f "$wf" ]] || continue - grep -qE '^[ \t]*pull_request:' "$wf" || continue - uses=$(grep -cF 'uses: jdx/mise-action' "$wf") - [[ "$uses" -gt 0 ]] || continue - pr_workflows=$((pr_workflows + 1)) - - lists=$(grep -cE '^[ \t]*install_args:' "$wf") - if [[ "$lists" -ne "$uses" ]]; then - echo "::error:: $wf runs mise-action $uses time(s) but declares $lists install_args list(s), so a pull_request job installs the whole toolchain. Narrow it to the tools that job invokes." >&2 - status=1 - fi - - # The ASSIGNMENT, set to false — not a mention. A substring search would - # pass a workflow whose only occurrence is the comment explaining why the - # variable matters, and it would pass one that sets it to "true", which is - # the same hole wearing a fix's clothing. - binding_task='^[ \t]*MISE_TASK_RUN_AUTO_INSTALL:[ \t]*"?false"?[ \t]*$' - binding_exec='^[ \t]*MISE_EXEC_AUTO_INSTALL:[ \t]*"?false"?[ \t]*$' - if ! grep -qE "$binding_task" "$wf" || ! grep -qE "$binding_exec" "$wf"; then - echo "::error:: $wf narrows install_args but does not set MISE_TASK_RUN_AUTO_INSTALL and MISE_EXEC_AUTO_INSTALL, so mise re-installs the rest at task time and the list decides nothing (CLOUD-180)." >&2 - status=1 - fi - - # AND THE NAMES IN *THAT* WORKFLOW RESOLVE TOO. The pass above judges one - # file, the one named by the first argument, and that was the whole gate - # when `ci.yml` was the only workflow with per-job lists. It is not any - # more: CLOUD-398 moved four jobs into `rust.yml`, taking their - # `install_args` with them, and a rename in `mise.toml` that missed those - # lists would have failed nowhere — `mise install` does not error on an - # unknown tool name, so it surfaces later as "command not found" in - # whichever step needed it. Same predicate, every file it applies to. - if [[ "$wf" != "$WORKFLOW" ]]; then - while IFS= read -r tool; do - [[ -n "$tool" ]] || continue - if ! grep -qxF "$tool" <<<"$declared"; then - echo "::error:: $wf installs '$tool', which is not a [tools] entry in $CONFIG. CI would skip it and fail later as 'command not found'." >&2 - status=1 - fi - done < <( - awk ' - /^[ \t]*install_args:/ { - sub(/^[ \t]*install_args:[ \t]*/, "") - print - } - ' "$wf" | tr -s '[:blank:]' '\n' | grep -v '^$' | sort -u - ) - fi - done -fi - -# --------------------------------------------------------------------------- -# THE THIRD DIRECTION, AND IT IS THE ONE THAT COST TWO CI RUNS (CLOUD-480). -# -# Both directions above are about the install list and `[tools]`. Neither can see -# a tool a `batten.toml` row SPAWNS: `policy-modules-type-check` runs `opa check -# -s schema/ policy/` and `policy-lint-rule-tests` runs `regal test -# .regal/rules`, both were declared in `[tools]`, and neither was in `ci.yml`'s -# list. With `MISE_EXEC_AUTO_INSTALL: false` making that list binding, the two -# rows did not run slowly — they failed CLOSED, at `deny`, while passing locally -# where the tools are installed. `land` refuses that as a verify/CI disagreement -# and is right to, but nothing named the cause. -# -# SCOPED TO WHAT MISE OWNS, which is derived rather than allowlisted. A spawned -# binary can arrive three other ways: bundled with a bigger tool (`cargo` with -# `rust`), pre-installed on the runner (`gh`), or vendored as a submodule -# (`bats`). None of those belongs in an install list, and all three came back as -# findings on the first run of this block. Rather than carry a second list of -# exemptions — the drifting authority this task exists to refuse — the question -# narrows to tools mise DECLARES: if `[tools]` owns it, the install list must name -# it, and if it does not, mise was never going to install it anyway. -# -# Basename matching on both sides, deliberately: `[tools]` and the list hold -# backend-qualified keys (`aqua:open-policy-agent/opa`) and the spawn names a -# binary (`opa`), so the comparison is over the last path component. Approximate -# in the safe direction — it can fail to catch a mismatch, never invent one — and -# the precise mapping is mise's business rather than something to restate here. -owned=$(sed -E 's@.*[:/]@@' <<<"$declared" | sort -u) -spawned=$( - grep -oE 'mise exec -- [a-z][a-z0-9._-]*' "$POLICY" 2>/dev/null | - awk '{print $NF}' | sort -u | - grep -xF -f <(printf '%s\n' "$owned") || true -) -if [[ -n "${spawned//[[:space:]]/}" ]]; then - installed=$( - awk ' - /^[ \t]*install_args:/ { - sub(/^[ \t]*install_args:[ \t]*/, "") - print - } - ' "$WORKFLOW" | tr -s '[:blank:]' '\n' | grep -v '^$' | - sed -E 's@.*[:/]@@' | sort -u - ) - while IFS= read -r tool; do - [[ -n "$tool" ]] || continue - if ! grep -qxF "$tool" <<<"$installed"; then - echo "::error:: $POLICY spawns '$tool' from a rule's check, but no install_args list in $WORKFLOW installs it. With auto-install off the row does not run slowly, it fails CLOSED — green locally, red in CI." >&2 - status=1 - fi - done <<<"$spawned" -fi - -if [[ "$status" -eq 0 ]]; then - echo "ci-tools-check: all $count tools named in $WORKFLOW are declared in $CONFIG, every tool a $POLICY row spawns is installed there, and every pull_request workflow in $WORKFLOW_DIR carries a binding install_args list" -fi -exit "$status" diff --git a/mise-tasks/coderabbit-config-check.sh b/mise-tasks/coderabbit-config-check.sh deleted file mode 100755 index e287c5b76..000000000 --- a/mise-tasks/coderabbit-config-check.sh +++ /dev/null @@ -1,142 +0,0 @@ -#!/usr/bin/env bash -#MISE description="Gate: .coderabbit.yaml still carries the keys the review lifecycle depends on (pointer-only)" -# -# CLOUD-860, and the missing half of CLOUD-847. That row landed `.coderabbit.yaml` -# and measured every key in it; nothing then held the file to those readings, so -# a rule shipped without a mechanism — the shape non-negotiable rule 2 names. -# -# THE THREE KEYS ARE NOT A STYLE PREFERENCE, they are what the lifecycle rests on: -# -# request_changes_workflow findings arrive as a formal CHANGES_REQUESTED review, -# so `reviewDecision` carries an answer. Off, the bot -# only COMMENTS and the decision stays null. -# auto_review.drafts the draft phase is the free phase (every job in -# ci.yml is `if: draft == false`). Off, nothing reviews -# it, and the review can only arrive after the ready — -# which is the whole defect CLOUD-847 measured. -# tools.gitleaks.enabled the ONLY secret scanning a draft gets, for the same -# reason: `mise run ci` does not run on drafts. The -# other linters are deliberately off because our gates -# already run them; this one is deliberately kept. -# -# WHY THE FAILURE IS WORTH A GATE RATHER THAN A COMMENT. Flipping `drafts` back is -# a one-line diff, and its symptom is silence: reviews stop happening, which looks -# exactly like nobody having pushed. The gate that consumes the config would then -# refuse every PR for want of a review the config quietly stopped producing, and -# the visible failure would be the gate rather than the cause. -# -# ABSENT IS NOT PASSING for the first two, and that asymmetry is the point: a key -# nobody wrote and a key someone deleted are the same file, and both leave the -# default in force — which is the value this gate exists to refuse. `gitleaks` is -# the inverse, because its default IS enabled: only an explicit `false` is a -# violation there, so absence passes. -# -# Pointer-only per non-negotiable rule 4: `path:line key=value` and a count, never -# a byte of the file. A config can carry instructions and paths, and a gate that -# echoed them would put them in every CI log. -# -# Exit 0 every required key holds / 2 a key is missing or flipped. No fail-open -# arm, unlike a gate that reads GitHub: the input is a tracked file in this -# checkout, so "could not look" means the file is gone, which is itself the -# violation this refuses. -# -# The mutation drops the empty-file guard, so a file with no keys reports zero -# violations — the vacuous pass this gate is shaped to avoid, and only the -# empty-fixture case can catch it. -#MUTANT empty-file-is-a-pass|s/if \[\[ "\$keys" -eq 0 \]\]/if false/|a comment-only file is a failure, not a vacuous pass -# A gate listed in $MUTANT_GATES with no row here fails `mise run mutant`. -#MUTANT flipped-key-passes|s/^if \[\[ "\$violations" -ne 0 \]\]; then$/if false; then/|request_changes_workflow flipped off fails - -set -euo pipefail - -CFG="${1:-.coderabbit.yaml}" - -if [[ ! -f "$CFG" ]]; then - echo "::error:: coderabbit-config-check: $CFG is absent — the review lifecycle has no configuration to rest on" >&2 - exit 2 -fi - -violations=0 -report() { # $1 = line (0 when the key is absent), $2 = pointer - echo "$CFG:$1 $2" >&2 - violations=$((violations + 1)) -} - -# A key line at any indentation, reported with its line number. The file is ours -# and its shape is reviewed; this reads the key rather than the tree, which is -# what keeps the gate to one awk pass and no YAML dependency. -key_line() { # $1 = key name -> "linevalue", empty when absent - awk -v key="$1" ' - $0 ~ "^[[:space:]]*#" { next } - { - pattern = "^[[:space:]]*" key ":[[:space:]]*" - if ($0 ~ pattern) { - value = $0 - sub(pattern, "", value) - sub("[[:space:]]*#.*$", "", value) - sub("[[:space:]]+$", "", value) - print NR "\t" value - exit - } - } - ' "$CFG" -} - -# `enabled:` appears once per tool, so this one is scoped: find the tool, then read -# the first `enabled:` inside its block. Reading it unscoped would answer about -# whichever tool happened to come first in the file. -tool_enabled() { # $1 = tool name -> "linevalue", empty when the tool is absent - awk -v tool="$1" ' - $0 ~ "^[[:space:]]*#" { next } - !seen && $0 ~ ("^[[:space:]]*" tool ":[[:space:]]*$") { seen = 1; next } - seen && $0 ~ "^[[:space:]]*enabled:[[:space:]]*" { - value = $0 - sub("^[[:space:]]*enabled:[[:space:]]*", "", value) - sub("[[:space:]]*#.*$", "", value) - sub("[[:space:]]+$", "", value) - print NR "\t" value - exit - } - seen && $0 ~ "^[[:space:]]*[A-Za-z0-9_-]+:[[:space:]]*$" { exit } - ' "$CFG" -} - -# The vacuity guard, and the reason it is first: every check below is an assertion -# ABOUT a key, so a file carrying none of them satisfies all of them by having -# nothing to judge. Counting real keys turns "no violations found" back into a -# statement about a file that was actually read. -keys=$(grep -cE '^[[:space:]]*[A-Za-z0-9_-]+:' "$CFG" || true) -if [[ "$keys" -eq 0 ]]; then - echo "::error:: coderabbit-config-check: $CFG carries no keys at all, so every assertion below would pass vacuously" >&2 - exit 2 -fi - -require_true() { # $1 = key name - local hit line value - hit=$(key_line "$1") - if [[ -z "$hit" ]]; then - report 0 "$1 absent (default is in force)" - return - fi - line=${hit%% *} - value=${hit#* } - [[ "$value" = "true" ]] || report "$line" "$1=$value (want true)" -} - -require_true "request_changes_workflow" -require_true "drafts" - -# The inverse arm: gitleaks defaults to enabled, so absence is compliant and only -# an explicit `false` is the violation. -hit=$(tool_enabled "gitleaks") -if [[ -n "$hit" ]]; then - line=${hit%% *} - value=${hit#* } - [[ "$value" != "false" ]] || report "$line" "tools.gitleaks.enabled=false (drafts would have no secret scanning)" -fi - -if [[ "$violations" -ne 0 ]]; then - echo "::error:: coderabbit-config-check: $violations violation(s) in $CFG — see CLOUD-847 for what each key was measured to do" >&2 - exit 2 -fi -echo "coderabbit-config-check: $CFG holds the three keys the review lifecycle depends on" diff --git a/mise-tasks/container-preflight.sh b/mise-tasks/container-preflight.sh deleted file mode 100755 index 033d58e44..000000000 --- a/mise-tasks/container-preflight.sh +++ /dev/null @@ -1,131 +0,0 @@ -#!/usr/bin/env bash -#MISE description="Gate: this container can actually do the work — egress reaches api.github.com and the credential carries the claims the lifecycle needs. Halts for manual repair; never repairs." -# -# CLOUD-261. `doctor` asserts the provisioning mise does not own. This asserts -# the things NOTHING in the repo owns: the container's egress policy and the -# credential it was handed. Both are supplied from outside, both are invisible -# until a task deep in the lifecycle fails in someone else's name, and neither -# is repairable from inside the session. -# -# Measured, on the container this was written for: -# -# * ambient NO_PROXY omitted api.github.com, so mise's own release resolver -# 403'd and `mise install` died naming `ubi:rust-cross/cargo-zigbuild` — a -# proxy policy wearing a third-party-tool costume. -# * the credential read the repo fine but carried no `checks=read`, so -# `ci-wait` could not see CI. A session verified green, readied a PR, and -# then could not land it — and read the toolchain as broken rather than the -# token as under-scoped. -# -# WHY THIS HALTS INSTEAD OF REPAIRING, which is the opposite of `doctor`. -# doctor's failures have one correct outcome an agent can reach: check out a -# submodule, install a target. These need a human to change a token's scopes or -# a container's egress policy. An agent that improvises past them does not -# recover — it produces work it cannot land, and concludes the repo is at fault. -# So the verdict is a stop sign, not a to-do list. -# -# WHAT THIS IS NOT SAYING. A correct container passes this silently, and main's -# landed history is the evidence that passing is the norm — this is a detector -# for a DEVIATION, not a description of the environment. Nothing it reports is a -# reason to rewrite a task, a memory, or AGENTS.md to accommodate a broken -# container. If it fires, the container is what gets fixed. -# -# Every probe is READ-ONLY, and the toolchain-dependent ones are skipped under -# `--degraded` (the hook passes it when provisioning already failed), so a -# missing binary is never reported as a missing permission. -# A gate listed in $MUTANT_GATES with no row here fails `mise run mutant`. -#MUTANT missing-claim-passes|s/^exit 1$/exit 0/|a missing read claim halts with exit 1 - -set -uo pipefail - -# `set -e` is deliberately absent — this collects every root cause rather than -# aborting on the first — so the cd guards itself. A preflight that silently ran -# from the wrong directory would probe the wrong repo. -cd "$(git rev-parse --show-toplevel)" || exit 1 - -degraded=no -[[ "${1:-}" = "--degraded" ]] && degraded=yes - -# Root causes are collected rather than raised one at a time: a broken container -# is usually broken in more than one way, and a human repairing it wants the -# whole list, not the first item followed by another session to find the second. -broken=() -detail="" - -# --- egress: can mise resolve a release at all? ------------------------------- -# -# Pure, instant, and deliberately first: it is the cause the failed install -# downstream is a symptom of, so naming it here is what stops the next reader -# debugging cargo-zigbuild. The decision itself lives in `egress-check` so it is -# testable without a proxy — same split as doctor / doctor-check. -egress=$("$(dirname "$0")/egress-check.sh" "${HTTPS_PROXY:-${https_proxy:-}}" "${NO_PROXY:-${no_proxy:-}}") -if [[ "$egress" = unfenced ]]; then - broken+=("EGRESS — api.github.com is proxied and not fenced out of NO_PROXY. - mise resolves every tool's release through that host, so \`mise install\` - fails on the first third-party tool and blames the tool. - REPAIR (outside the session): add api.github.com, objects.githubusercontent.com, - codeload.github.com and uploads.github.com to the container's AMBIENT NO_PROXY. - mise.toml's [env] already appends them, and CANNOT help here: mise applies - [env] to the processes it runs, after its own resolver has made the call.") -fi - -# --- credential: can it drive the lifecycle? ---------------------------------- -# -# gh-preflight already walks the read endpoints the tasks call and reports the -# claim GitHub itself names in each 403 (X-Accepted-GitHub-Permissions). It is -# the diagnosis and it already exits non-zero; what was missing was anything -# running it before a session spent an hour rediscovering the same fact one task -# at a time. Adopted rather than rebuilt (AGENTS.md: adopt prior art). -if [[ "$degraded" = yes ]]; then - echo "container-preflight: toolchain incomplete — skipping the GitHub probes (a missing gh is not a missing permission)" -else - preflight=$(mise run gh-preflight 2>&1) - rc=$? - detail="$preflight" - case "$rc" in - 0) ;; - 1) - broken+=("CREDENTIAL — the token reads the repo but is missing read claim(s) - the lifecycle needs. Work can be verified and pushed, and then never landed: - ci-wait cannot see CI state and land cannot see the merge. - REPAIR (outside the session): grant the claims gh-preflight names below, or - supply a classic PAT scoped \`repo\`, which bundles checks=read.") - ;; - *) - # Deliberately a fork, not an assertion: an Actions/API incident looks - # exactly like an absent token from here, and sending someone to rotate a - # working credential during an outage is its own expensive mistake. - broken+=("GITHUB UNREACHABLE — could not probe the API at all. Either the token - is absent/unauthenticated, or GitHub is having an incident. - CHECK FIRST: https://www.githubstatus.com/api/v2/summary.json — during an - incident this clears on its own and no credential should be touched.") - ;; - esac -fi - -if [[ "${#broken[@]}" -eq 0 ]]; then - echo "container-preflight: egress fenced, credential carries every probed read claim" - exit 0 -fi - -{ - echo - echo "::error:: container-preflight: THIS CONTAINER CANNOT DO THE WORK." - echo - for cause in "${broken[@]}"; do - echo " * $cause" - echo - done - if [[ -n "$detail" ]]; then - echo " gh-preflight said:" - printf '%s\n' "$detail" | sed 's/^/ /' - echo - fi - echo " HALT. None of the above is a defect in this repository and none is" - echo " repairable from inside the session — that is why this stops rather than" - echo " working around them. Do not rewrite tasks, memories or AGENTS.md to" - echo " accommodate what you find here, and do not improvise a workaround and" - echo " carry on: the work will verify, push, and then fail to land." - echo " Report these root causes, repair the container, start a new session." -} >&2 -exit 1 diff --git a/mise-tasks/digest-major-agreement.sh b/mise-tasks/digest-major-agreement.sh deleted file mode 100755 index c322d5935..000000000 --- a/mise-tasks/digest-major-agreement.sh +++ /dev/null @@ -1,182 +0,0 @@ -#!/usr/bin/env bash -#MISE description="Gate: the crypto crates this workspace declares itself all resolve the same `digest` major (CLOUD-767)" -# -# THE RULE THIS REPLACES WAS A COMMENT, AND THE COMMENT WAS WRONG. -# -# `Cargo.toml` pinned `hmac = "0.12"` under a justification that read, in full, -# "0.13 wants `digest 0.11` and would put two majors of the same hashing -# substrate in the tree". Measured 2026-08-20 on `main`: the committed -# `Cargo.lock` already carried `digest 0.10.7` AND `digest 0.11.3`, the second -# arriving through `gix-hash` -> `sha2 0.11`. The claim had been false for as -# long as `gix` had been a dependency, nothing noticed, and the pin it justified -# went on being enforced by nobody but the reader. -# -# That is non-negotiable rule 2 stated from the failure end. Prose is feedforward -# only; a decision with no exit code is one that rots silently and is discovered -# by a bot proposing the bump it forbade (#503, CLOUD-767). -# -# WHAT IS ACTUALLY DECIDABLE, and the narrowing is the whole design. "One -# `digest` major in the tree" is NOT it: `gix` vendors what it vendors, the -# workspace has no say, and a gate asserting it would be red on the commit that -# introduced it and every commit after. A gate nobody can keep green is switched -# off within a day, and switching it off takes the real rule with it. -# -# The real rule is narrower and is true today: **the crypto crates this workspace -# declares FOR ITSELF must agree with each other.** `hmac` and `sha2` are both -# `[workspace.dependencies]` entries, both resolve a `digest`, and -# `crates/batten/src/identity.rs` composes them in one expression — -# ` as KeyInit>::new_from_slice`. Split them across majors and that -# type does not exist. Unlike `cap-drift`'s half-lift this direction does have a -# symptom; what it does not have is a symptom BEFORE a runner is spent, which is -# what this buys. -# -# READ FROM THE LOCKFILE, NEVER THE MANIFEST. `hmac = "0.12"` is a caret -# requirement, not a resolution: which `digest` it lands on is `Cargo.lock`'s -# answer and cannot be derived from the manifest without re-implementing the -# resolver. The manifest is consulted for ONE thing — which crates the workspace -# declares for itself — because that is the question `Cargo.lock` cannot answer: -# the lock cannot tell our direct dependency from `gix`'s transitive one, and -# conflating the two is exactly the mistake the old comment made. -# -# A DEPENDENCY-OF-A-DEPENDENCY IS NOT OUR AGREEMENT TO KEEP. `sha1-checked` and -# `sha1` sit under `gix` and resolve whatever they resolve. They are excluded by -# construction, being absent from `[workspace.dependencies]`. -# -# Pointer-only per non-negotiable rule 4: crate names and the majors they landed -# on. Never a version requirement, never a manifest line, never a lock stanza. -# -# Exit 0 they agree (or fewer than two are declared) / 1 they split / 2 could not -# look. -# -# The mutation stops reading the resolution and hardcodes agreement, which is the -# shortcut this gate exists to refuse: every pair then "agrees" and a lockfile -# that split them across two `digest` majors reads clean. -#MUTANT reads-no-resolution|s/^\tmajor=\$(digest_major_of "\$crate")$/\tmajor=0.10/|THE HALF-BUMP IS REFUSED — one crate moved and the other left behind -#PIN-OK: awk -set -uo pipefail - -manifest="${DIGEST_AGREEMENT_MANIFEST:-Cargo.toml}" -lock="${DIGEST_AGREEMENT_LOCK:-Cargo.lock}" - -# The crates whose `digest` major must agree. Named rather than sniffed: "does -# this crate depend on `digest`" is a question about the whole registry, and a -# gate that answered it by scanning the lock would silently widen to every -# transitive hasher the day one appeared. This is the workspace's own composition -# decision, and it is one line long. -CRYPTO_CRATES="${DIGEST_AGREEMENT_CRATES:-hmac sha2}" - -while [[ $# -gt 0 ]]; do - case "$1" in - --manifest) - manifest="${2:-}" - shift 2 - ;; - --lock) - lock="${2:-}" - shift 2 - ;; - *) - echo "usage: digest-major-agreement [--manifest ] [--lock ]" >&2 - exit 2 - ;; - esac -done - -for f in "$manifest" "$lock"; do - if [[ ! -r "$f" ]]; then - echo "::error:: digest-major-agreement: cannot read $f — a gate that cannot look must not report agreement" >&2 - exit 2 - fi -done - -# Is this crate one the workspace declares for itself? Scoped to the -# `[workspace.dependencies]` table alone, so a name in a comment, in -# `[workspace.lints]`, or as another table's key cannot enrol a crate `gix` owns. -declared() { - awk -v want="$1" ' - /^\[workspace\.dependencies\]/ { in_deps = 1; next } - /^\[/ { in_deps = 0 } - in_deps && $1 == want { found = 1 } - END { exit(found ? 0 : 1) } - ' "$manifest" -} - -# The `digest` major this crate resolved to, read out of its own `[[package]]` -# stanza. Cargo writes a bare `"digest"` when one major is in the tree and -# `"digest "` when several are, so both spellings are handled — `bare` -# is returned for the first and resolved by the caller, since with one major -# there is nothing to disambiguate. -# -# Prints the major, or nothing when the crate is absent or names no `digest`. -digest_major_of() { - awk -v want="$1" ' - /^\[\[package\]\]/ { inpkg = 0 } - /^name = "/ { - line = $0 - gsub(/^name = "|"$/, "", line) - if (line == want) inpkg = 1 - } - inpkg && /^ "digest/ { - line = $0 - gsub(/^ "|",$|"$/, "", line) - n = split(line, parts, " ") - if (n >= 2) { split(parts[2], v, "."); print v[1] "." v[2] } - else print "bare" - exit - } - ' "$lock" -} - -# Every `digest` major in the lock, for resolving a bare reference. More than one -# and a bare reference cannot occur, so a caller seeing several has been handed a -# lockfile that does not describe itself. -digest_majors_in_lock() { - awk ' - /^\[\[package\]\]/ { isdigest = 0 } - /^name = "digest"$/ { isdigest = 1; next } - isdigest && /^version = "/ { - line = $0 - gsub(/^version = "|"$/, "", line) - split(line, v, ".") - print v[1] "." v[2] - isdigest = 0 - } - ' "$lock" | sort -u -} - -seen="" -for crate in $CRYPTO_CRATES; do - declared "$crate" || continue - major=$(digest_major_of "$crate") - if [[ -z "$major" ]]; then - echo "::error:: digest-major-agreement: $crate is declared in $manifest and resolves no digest in $lock — the lockfile does not describe the manifest, so nothing here can be decided" >&2 - exit 2 - fi - if [[ "$major" = bare ]]; then - major=$(digest_majors_in_lock) - if [[ "$(printf '%s\n' "$major" | grep -c .)" != 1 ]]; then - echo "::error:: digest-major-agreement: $crate names digest without a version while $lock carries several — cannot tell which it resolved" >&2 - exit 2 - fi - fi - seen="$seen$crate $major"$'\n' -done - -declared_count=$(printf '%s' "$seen" | grep -c . || true) -if [[ "$declared_count" -lt 2 ]]; then - echo "digest-major-agreement: $declared_count declared crypto crate(s) — fewer than two can disagree" - exit 0 -fi - -majors=$(printf '%s' "$seen" | awk '{ print $2 }' | sort -u) -if [[ "$(printf '%s\n' "$majors" | grep -c .)" != 1 ]]; then - echo "::error:: digest-major-agreement: the workspace's own crypto crates resolved different digest majors, so \`Hmac\` composes two incompatible substrates and the workspace will not build:" >&2 - printf '%s' "$seen" | while read -r crate major; do - [[ -n "$crate" ]] && echo "::error:: $crate -> digest $major" >&2 - done - echo "::error:: bump them together or not at all — the manifest comments on both entries say so" >&2 - exit 1 -fi - -echo "digest-major-agreement: $declared_count declared crypto crate(s), all on digest $majors" -exit 0 diff --git a/mise-tasks/egress-check.sh b/mise-tasks/egress-check.sh deleted file mode 100755 index 74e4e6425..000000000 --- a/mise-tasks/egress-check.sh +++ /dev/null @@ -1,72 +0,0 @@ -#!/usr/bin/env bash -#MISE description="The pure decision behind the session preflight: can mise's own release resolver reach api.github.com, given the ambient proxy environment?" -# -# mise resolves every tool's release through GitHub's API host, api.github.com. -# Where an egress proxy fronts the network and injects a repo-scoped token, that -# host answers 403 for THIRD-PARTY tool repos (`GitHub access to this repository -# is not enabled for this session`), and `mise install` fails on the first such -# tool. mise.toml's [env] block already appends api.github.com to NO_PROXY for -# exactly this reason. -# -# THE PART THAT MAKES THIS A SEPARATE CHECK: that guard cannot fix itself. mise -# applies its own [env] to the processes it RUNS, after its release resolver has -# already made the HTTP call that picks the tool version. So a container whose -# AMBIENT NO_PROXY omits api.github.com fails during `mise install` — before any -# mise.toml setting is in play — and the failure names the tool repo rather than -# the proxy, which is what makes it expensive to read. -# -# THIS IS NOT THE NORM, AND MUST NOT BE READ AS ONE. A correctly provisioned -# container either has no proxy at all or already carries the fence in its -# ambient environment; main's landed history is the evidence that this is the -# usual case. This check exists to catch the DEVIATION at the start of a session -# rather than three tasks in, and its verdict is a statement about one container, -# never about the repo or the toolchain. -# -# Split from the hook that calls it for the reason doctor-check is split from -# doctor: the decision is the part worth testing, and it only tests if it is -# callable without the thing it decides about. Both values are ARGUMENTS, never -# reads of the live environment, so the suite can drive every combination. -# -# Verdicts, one word on stdout: -# -# ok no proxy is set (nothing to fence), or the proxy is set and -# api.github.com is fenced out of it. mise can resolve releases. -# unfenced a proxy is set and api.github.com is NOT fenced. `mise install` -# will fail on the first third-party tool, and no mise.toml setting -# can prevent it. -# -# Exit is 0 for a delivered verdict and 2 for a malformed call: this classifies, -# it does not adjudicate. The caller decides what a verdict is worth. -set -uo pipefail - -usage() { - echo "usage: egress-check " >&2 - exit 2 -} - -# Both arguments are required but either may legitimately be EMPTY — "no proxy -# set" and "no NO_PROXY set" are the two most interesting inputs. So arity is -# checked, never emptiness. -[[ "$#" -eq 2 ]] || usage - -proxy="$1" -no_proxy="$2" - -# No proxy fronting the network: mise talks to GitHub directly and there is -# nothing to fence. This is the ordinary developer machine and most CI runners. -if [[ -z "$proxy" ]]; then - echo "ok" - exit 0 -fi - -# The host mise's resolver calls. Matched as a substring rather than by splitting -# on commas, because NO_PROXY has no single normative syntax — entries appear -# bare, dot-prefixed, and wildcard-prefixed across tools — and any of those forms -# containing this host means some implementation will honour it. A false "ok" -# here would be worse than a false "unfenced": the first hides the diagnosis the -# session needs, the second only asks a human to look. -case "$no_proxy" in -*api.github.com*) echo "ok" ;; -*) echo "unfenced" ;; -esac -exit 0 diff --git a/mise-tasks/hook-pin-check.sh b/mise-tasks/hook-pin-check.sh deleted file mode 100755 index 167d9731f..000000000 --- a/mise-tasks/hook-pin-check.sh +++ /dev/null @@ -1,153 +0,0 @@ -#!/usr/bin/env bash -#MISE description="Gate: no hook registered BY PATH may shell out to a mise-pinned tool, because a by-path invocation does not get mise's env" -# -# CLOUD-479's own trap, made computable rather than remembered. -# -# A hook registered as `mise run -q ` pays ~203ms of task-runner startup -# per call — measured, against 15-19ms for the same script invoked by path, on -# the one path where an agent cannot background anything. Registering by path is -# therefore the obvious fix, and it has one non-obvious cost: **a by-path -# invocation does not get mise's env**, so every tool the script assumes is the -# pinned one resolves to whatever the ambient PATH offers, or to nothing. -# -# That is not a latency question, it is a correctness one. Every hook here is -# fail-open by design — `|| exit 0`, `2>/dev/null`, an empty read that means -# "allow" — so a missing parser does not error. It silently allows. A gate that -# reports a clean session while checking nothing is strictly worse than the -# 203ms it saved, and it is invisible: nothing turns red. -# -# So the pairing is refused: registered by path AND shelling out to a tool -# `mise.toml` pins. Either invoke it through `mise run` and pay the startup, or -# stop depending on the pinned tool — which is what `stop-guard` and -# `contract-drift` did, moving their payload reads to `payload-field` and the -# compiled binary that is already on this path. -# -# ONE EXEMPTION, declared IN THE SCRIPT rather than listed here: a task may carry -# a `#PIN-OK: ` line stating that it asserts the tool's presence itself. -# `mcp-attach-check` is the case that forced it — its two `jq` calls read a -# settings file and an MCP log, neither of which is a hook payload, so no -# extraction surface can serve them. It checks `command -v jq` and exits 2, -# loudly, rather than reading nothing. The exemption lives beside the assertion -# it describes so the two cannot drift apart; a list here would be a second -# authority on a fact the file already states. -# -# Not judged: `mise run` registrations. They get mise's env by construction, and -# this gate has nothing to say about them. -# -# Pointer-only per non-negotiable rule 4: the task name and the tool name, never -# a line of either file. -# -# Exit 0 clean / 1 a by-path registration depends on a pinned tool / 2 could not -# look (no settings file, no manifest). -# -# The mutation drops the by-path filter, so `mise run` registrations are judged -# too and the fixture that pairs `mise run` with a pinned tool reddens — a gate -# that cannot tell the two invocation shapes apart is not this gate. -#MUTANT pin-check-ignores-invocation-shape|s/case "\$command" in/case "mise run x" in/|a by-path hook shelling out to a pinned tool is refused -# A gate listed in $MUTANT_GATES with no row here fails `mise run mutant`. -#MUTANT pinned-tool-in-path-hook-passes|s/^\texit 1$/\texit 0/|shelling out to a pinned tool is refused - -set -uo pipefail - -settings="${HOOK_PIN_SETTINGS:-.claude/settings.json}" -manifest="${HOOK_PIN_MANIFEST:-mise.toml}" -tasks_dir="${HOOK_PIN_TASKS:-mise-tasks}" - -if [[ ! -f "$settings" ]]; then - echo "::error:: hook-pin-check: no $settings — nothing to judge" >&2 - exit 2 -fi -if [[ ! -f "$manifest" ]]; then - echo "::error:: hook-pin-check: no $manifest — cannot tell which tools are pinned" >&2 - exit 2 -fi - -# The pinned set, read from `[tools]` and nothing else. A key is either bare -# (`zizmor`, `node`) or a backend coordinate (`"aqua:jqlang/jq"`), and it is the -# LAST path segment that names the executable — `aqua:jqlang/jq` is `jq`. Read -# from the manifest rather than restated, so pinning a tool enrols it here with -# no second edit. -pinned=$(awk ' - /^\[tools\]/ { in_tools = 1; next } - /^\[/ { in_tools = 0 } - in_tools && /^["a-zA-Z0-9]/ { - key = $1 - gsub(/"/, "", key) - n = split(key, parts, "/") - print parts[n] - } -' "$manifest" | sort -u) - -if [[ -z "$pinned" ]]; then - echo "::error:: hook-pin-check: no [tools] entries in $manifest — cannot tell which tools are pinned" >&2 - exit 2 -fi - -fail=0 -judged=0 - -# Every registered hook command, one per line. `grep`+`sed` rather than `jq`, -# deliberately: a gate about depending on a pinned tool must not itself depend on -# one. It reads the committed text, which is the only thing it judges. -while IFS= read -r command; do - [[ -n "$command" ]] || continue - - # BY PATH is the case this judges: a command naming a file under the tasks - # directory. A `mise run` registration gets mise's env and is not this gate's - # business. - # - # Matched on the tasks directory's BASENAME, not on `$tasks_dir` itself. The - # two are the same in this repository and are not the same under test, where - # the fixture directory is an absolute path — and a registration always names - # the repo-relative segment. Conflating "where do I read the script" with - # "what does the registration look like" made every fixture silently skip, - # which is the shape of a gate that judges nothing while reporting clean. - segment="${tasks_dir##*/}" - case "$command" in - *"/$segment/"*) ;; - *) continue ;; - esac - - task="${command##*/}" - script="$tasks_dir/$task" - [[ -f "$script" ]] || continue - judged=$((judged + 1)) - - # Comments stripped ONCE, into a variable, and deliberately not piped into - # the `grep -q` below. `grep -q` exits on its first match, which SIGPIPEs the - # producer, and under `pipefail` that makes the pipeline report failure — - # so a MATCH would read as "no match" and the violation would vanish, the - # more reliably the larger the file. `pipefail-grep-check` refuses that shape - # and refused this one while it was being written. - stripped=$(sed -e 's/#.*//' "$script") - - while IFS= read -r tool; do - [[ -n "$tool" ]] || continue - # A call to the tool, not a mention of it: the word at the start of a - # command position. Comments are stripped first, so the paragraphs - # explaining why a tool was dropped do not read as a dependency on it — - # which is exactly what `stop-guard` and `contract-drift` now contain. - if grep -qE "(^|[;&|(]|\\\$\\()[[:space:]]*${tool}[[:space:]]" <<<"$stripped" 2>/dev/null; then - # The declared exemption, beside the assertion it describes. - if grep -qE "^#PIN-OK:.*\\b${tool}\\b" "$script"; then - continue - fi - echo "$task $tool" >&2 - fail=1 - fi - done <<<"$pinned" -done <<<"$(grep -o '"command"[[:space:]]*:[[:space:]]*"[^"]*"' "$settings" | sed 's/.*"command"[[:space:]]*:[[:space:]]*"//; s/"$//')" - -# A gate that judged nothing must not look like one that found nothing -# (CLOUD-418's class). Every registration being `mise run` is a legitimate state; -# a settings file this could not parse at all is not. -if [[ "$judged" -eq 0 ]]; then - echo "hook-pin-check: no by-path hook registrations to judge" >&2 -fi - -if [[ "$fail" -ne 0 ]]; then - echo "::error:: hook-pin-check: a hook registered BY PATH shells out to a mise-pinned tool, named above. By-path invocation does not get mise's env, so that tool resolves unpinned or not at all — and every hook here fails OPEN, so an absent one allows silently instead of erroring. Either register it as \`mise run -q \`, or drop the dependency (see \`mise-tasks/payload-field.sh\`), or assert the tool yourself and declare \`#PIN-OK: \`." >&2 - exit 1 -fi - -echo "hook-pin-check: $judged by-path hook registration(s), none depending on a mise-pinned tool" diff --git a/mise-tasks/license-table-check.sh b/mise-tasks/license-table-check.sh deleted file mode 100755 index 83bad6131..000000000 --- a/mise-tasks/license-table-check.sh +++ /dev/null @@ -1,85 +0,0 @@ -#!/usr/bin/env bash -#MISE description="Gate: every adopted tool's license row is resolved — the release precondition CONTRIBUTING.md states in prose, as a predicate" -# -# CONTRIBUTING.md's license table ends with "Confirm each _to confirm_ entry -# before that tool is adopted in a shipped release." That sentence is a release -# precondition, and it had no runnable check: three of five rows carried -# `_to confirm_` in both columns and nothing failed. A rule without its mechanism -# is half a change (AGENTS.md, non-negotiable rule 2), so this is the other half. -# -# The table is the data and this is only the assertion over it — the verdicts are -# NOT restated here. A second copy would be a second authority for one fact, and -# the two would drift. -# -# Deliberately narrow. It judges whether a row is *resolved*, never whether the -# recorded license is *correct*: correctness is a human reading an upstream -# LICENSE file, which no exit code can stand in for. What the gate can prove is -# that nobody shipped while the question was still open. -# A gate listed in $MUTANT_GATES with no row here fails `mise run mutant`. -#MUTANT empty-table-passes|s/^if \[\[ "\$rows" -eq 0 \]\]; then$/if false; then/|a table with no rows is a failure - -set -uo pipefail - -DOC="${1:-CONTRIBUTING.md}" - -if [[ ! -r "$DOC" ]]; then - echo "::error:: license-table-check: cannot read $DOC" >&2 - exit 1 -fi - -# The compatibility column is a closed set. An unrecognised glyph is a failure -# rather than a pass, because "some other marker" is exactly how an unresolved -# row would slip through a check that only looked for the literal placeholder. -readonly YES='✅' -readonly NO='❌' - -fail=0 -rows=0 - -# Read the table between its heading and the next blank-line-terminated block. -# Rows are `| cell | cell | cell | cell |`; the header and the `---` separator -# are skipped by shape, not by line number, so inserting a row cannot shift the -# parse. -while IFS= read -r line; do - case "$line" in - '|'*'|') ;; - *) continue ;; - esac - case "$line" in - *'---'*) continue ;; - esac - - tool=$(printf '%s' "$line" | cut -d'|' -f2 | sed 's/^ *//; s/ *$//') - license=$(printf '%s' "$line" | cut -d'|' -f4 | sed 's/^ *//; s/ *$//') - compat=$(printf '%s' "$line" | cut -d'|' -f5 | sed 's/^ *//; s/ *$//') - - # The header row names the columns rather than a tool. - [[ "$tool" = "Tool" ]] && continue - [[ -z "$tool" ]] && continue - - rows=$((rows + 1)) - - if [[ -z "$license" ]] || [[ "$license" != "${license#*to confirm}" ]]; then - [[ "$fail" = 0 ]] && echo "::error:: license-table-check: a row's license is unresolved. Read the upstream LICENSE file and record the SPDX id:" >&2 - printf ' %s — license is %s\n' "$tool" "${license:-empty}" >&2 - fail=1 - continue - fi - - if [[ "$compat" != "$YES" ]] && [[ "$compat" != "$NO" ]]; then - [[ "$fail" = 0 ]] && echo "::error:: license-table-check: a row's Apache-2.0 verdict is unresolved or outside the closed set ($YES / $NO):" >&2 - printf ' %s — verdict is %s\n' "$tool" "${compat:-empty}" >&2 - fail=1 - fi -done <"$DOC" - -if [[ "$rows" -eq 0 ]]; then - # A table that parses to zero rows passes every per-row assertion vacuously, - # which is the false green this branch exists to kill: a renamed heading or a - # reformatted table would otherwise read as "all rows resolved". - echo "::error:: license-table-check: no license rows found in $DOC — the table moved or its shape changed" >&2 - exit 1 -fi - -[[ "$fail" = 0 ]] && echo "license-table-check: $rows adopted-tool rows, every license and verdict resolved" -exit "$fail" diff --git a/mise-tasks/mise-action-floor.sh b/mise-tasks/mise-action-floor.sh deleted file mode 100755 index f6a0a7028..000000000 --- a/mise-tasks/mise-action-floor.sh +++ /dev/null @@ -1,164 +0,0 @@ -#!/usr/bin/env bash -#MISE description="Gate: no workflow pins a toolchain-install action commit known to predate its download retry (CLOUD-404)" -# -# CLOUD-404. The toolchain install action fetched its own mise binary with a bare -# `curl -fsSL` and no retry, so a transient release-asset error (curl 22/503, -# curl 60/TLS) killed a job in provisioning — spending its minutes, redding the -# branch, and answering nothing. Three occurrences in two days. Upstream fixed it -# in `9dda3952d` (`retryDownload`, 5 attempts at 2s, wrapping both download tools -# and retrying on any thrown error) after our report, and we adopted that commit -# directly because it ships its own built `dist/`. -# -# WHY THIS GATE EXISTS, AND IT IS NOT ABOUT HUMANS FORGETTING. We adopted an -# UNRELEASED commit, so the pin no longer corresponds to a release tag. The pin -# comment used to name the major version alone, and `renovate.json5` -# tracks the `github-actions` ecosystem while `auto-bot-land.yml` lands bot -# bumps **with no human in the loop** — every check green is the only condition. -# A bot resolving that major back to the pre-retry commit would therefore be a -# silent DOWNGRADE to the un-retried install, auto-landed. That is strictly worse -# than the transient it reverts, because nothing announces it and the next -# occurrence reads as fresh. -# -# So the rule "do not slide back behind the retry" ships with its mechanism -# (non-negotiable 2): this gate reds such a pin at check time, the bot's PR -# cannot go green, and `auto-bot-land` cannot land it. The accepted cost is -# that such a PR then sits open and red until somebody closes it — stated here -# rather than discovered later. -# -# A DENYLIST OF KNOWN-BAD PINS, NOT A REQUIRED SHA. An "equals the expected -# commit" gate would fail every legitimate forward bump and demand a hand edit in -# lockstep with the bot — which is how a gate earns a bypass and then gets -# switched off. A denylist is silent on the next release and every one after it, -# and speaks only for the backslide. It also cannot answer "is this pin new -# enough", which is the honest limit: ancestry needs the network, so it lives on -# the issue as a checkable acceptance line rather than being faked offline here. -# -# WHICH BYTES: the INDEX (`git show :`), the `timeout-check` and -# `lock-complete` idiom — exactly the bytes a commit would carry, identical in CI -# and in a sandbox, immune to whatever an editor left in the tree. Explicit path -# arguments win, which is how the bats suite drives fixtures without a git repo. -# -# Exit 0 pass, 1 a denylisted pin, 2 could-not-look. `2` is the `lock-complete` -# doctrine — "the gate could not read what it was asked to judge" — and a gate -# reporting green over bytes it failed to read is what gets a gate switched off. -# A gate listed in $MUTANT_GATES with no row here fails `mise run mutant`. -#MUTANT pre-retry-pin-passes|s/^\texit 1$/\texit 0/|a pre-retry pin fails and is named - -set -euo pipefail - -# ONE COORDINATE, and both the action name and the denylist are derived from it. -# Written as a real pinned coordinate rather than a bare vendor string on purpose: -# `attribution-check` exempts a line that NAMES a dependency (`uses:`, `@<40 hex>`) -# and flags the same name in prose as an appeal to authority — and the exemption is -# per LINE, so the spelling has to live where it reads as a coordinate. Deriving -# also means the sha appears once, so the name and the denylisted pin cannot drift. -# -# This is the pin that predates the retry: the latest release and the floating -# major tag both resolve to it, which is exactly what makes it the reachable -# backslide rather than a hypothetical one. -PRE_RETRY_COORDINATES=" -uses: jdx/mise-action@7e36c90d9ab29c415a2384db3006f3ec8a8cc654 -" - -# The action every message and pattern below expands, taken from the first -# coordinate: everything left of the `@`, with the `uses: ` prefix dropped. -# `<<<` rather than a pipe into `grep -m1`: an early-exiting grep SIGPIPEs its -# producer, and under `pipefail` that makes a MATCH report failure — the inversion -# `pipefail-grep-check` gates, which caught exactly this line. -first_coordinate=$(grep -m1 '@' <<<"$PRE_RETRY_COORDINATES" || true) -if [[ -z "$first_coordinate" ]]; then - echo "::error:: mise-action-floor: PRE_RETRY_COORDINATES declares no pinned coordinate, so this gate has nothing to judge against." >&2 - exit 2 -fi -ACTION=${first_coordinate#*: } -ACTION=${ACTION%@*} - -# The shas themselves. Add a coordinate above when a pin is found to predate a fix -# we depend on; never remove one, because a commit does not stop being pre-retry. -PRE_RETRY_PINS=$(sed -n 's/.*@//p' <<<"$PRE_RETRY_COORDINATES") - -# Pointer-only per non-negotiable rule 4: the workflow, the line number and a -# short sha. Never a line of workflow content. -violations=0 -report() { - echo "::error:: $1" >&2 - violations=$((violations + 1)) -} - -# The files to judge. With arguments, those paths; without, every workflow in the -# index. Fixture mode is the argument form. -declare -a labels=() -declare -a sources=() -scratch="" -cleanup() { [[ -z "$scratch" ]] || rm -rf "$scratch"; } -trap cleanup EXIT - -if [[ "$#" -gt 0 ]]; then - for path in "$@"; do - if [[ ! -f "$path" ]]; then - echo "::error:: mise-action-floor: $path not found" >&2 - exit 2 - fi - labels+=("$path") - sources+=("$path") - done -else - scratch="$(mktemp -d)" - tracked="$(git ls-files '.github/workflows/*.yml')" - if [[ -z "$tracked" ]]; then - echo "::error:: mise-action-floor: no tracked .github/workflows/*.yml — run from the repo, or pass paths" >&2 - exit 2 - fi - while IFS= read -r path; do - [[ -n "$path" ]] || continue - blob="$scratch/$(basename "$path")" - if ! git show ":$path" >"$blob" 2>/dev/null; then - echo "::error:: mise-action-floor: $path is not in the index — stage it, or pass a path" >&2 - exit 2 - fi - labels+=("$path") - sources+=("$blob") - done <<<"$tracked" -fi - -# Every 40-hex pin of the action, as `