Skip to content

PRD: Merge-queue latency — docs-only fast path on merge_group + Test Suite sharding #2116

Open
2 of 2 issues completed

Description

@filipeforattini

Problem Statement

Merging a one-line docs change takes ~15 minutes. The docs shim (ci-docs.yml, issue #1307) fast-greens all required checks for docs-only PRs — but it only listens to pull_request, not merge_group, so in the merge queue the full ci.yml runs regardless (GitHub does not support path filters on merge_group). Measured on the last green queue run, the wall clock is one job: Test Suite at 901s (~15 min); everything else finishes in ≤ 2.3 min. This tax lands on every queued merge — and will compound hard under AFK drains with dozens of merges a day. Cost is not the issue (public repo, free GitHub runners; only one Blacksmith job is paid) — latency is.

Solution

Two levers, both accepted: (1) close the shim gap on merge_group — a first job in ci.yml computes whether the merge-group diff is docs-only and, if so, the heavy jobs report immediate success (same contract-locked job-name mechanism the shim already uses); docs merges drop to ~1 minute. (2) Shard the Test Suite — the tests are already grouped into 12 bins; split across 3–4 parallel jobs (free runners, parallelism costs nothing) with cargo-nextest inside each shard; 901s becomes ~4–5 min for every merge, not just docs.

User Stories

  1. As a maintainer landing a glossary term, I want a docs-only merge to complete in about a minute, so that documentation discipline isn't taxed 15 minutes per line.
  2. As an AFK fleet operator, I want queue latency under 5 minutes, so that a drain of dozens of slices per day doesn't serialize behind Test Suite wall clock.
  3. As a release manager, I want the docs-only fast path to be contract-locked to the required-checks list, so that the shim cannot drift from the gate (the existing CI gate gap: docs-only (**.md) PRs can't satisfy required checks → permanently BLOCKED #1307 lock-step test extends to the merge_group path).
  4. As a code contributor, I want a code-touching merge group to still run the full suite, so that the green ratchet (ADR 0059) is never weakened.
  5. As a CI maintainer, I want Test Suite shards balanced over the existing test bins, so that shard wall clocks stay even as bins grow.
  6. As a cost owner, I want the sharding on free public runners, so that the speedup adds no Blacksmith spend.

Human Decisions

  • Decision: Levers 1 + 2 enter the backlog; merge-queue batching (grouping multiple PRs per CI run) is deferred.
  • Why: the shim fix is ~30 lines of YAML with immediate return; sharding attacks the root (the 15-minute tax on every code merge, which will hurt badly under AFK drains); batching only pays with real concurrent PR volume.
  • Alternatives considered: shim fix only (rejected: leaves the code-merge tax); full package with batching (deferred).

Implementation Decisions

  • A docs-only detection job on merge_group computes the diff of the merge group against its base; heavy jobs condition on its output and report success without running when docs-only (paths: the same set ci.yml path-ignores and the shim triggers on).
  • The scripts/ci-docs-shim-contract.test.mjs lock-step test extends to cover the merge_group fast path, so required-context names cannot drift.
  • Test Suite sharding partitions the existing grouped test bins across 3–4 matrix jobs; cargo-nextest runs within shards; the required context becomes the matrix aggregate (or per-shard contexts added to the required list — decided at implementation against the branch-protection mechanics).
  • No change to which tests exist or what they assert; this is scheduling only.

Testing Decisions

  • The CI change is validated by the repo's own meta-tests: the shim contract test, plus a demonstration docs-only PR and a code-touching PR through the queue showing fast-green and full-run respectively.
  • Shard balance is observable in the run summary; a drifting shard (>2× median) is a maintenance signal, noted in the workflow comments.

Acceptance Criteria

  • A docs-only PR merges through the queue in under 2 minutes (measured on a real merge).
  • A code-touching PR still runs the full suite in the merge group (pinned by the contract test).
  • Test Suite wall clock in the merge group is under 6 minutes on a representative run.
  • The shim lock-step test covers the merge_group path and fails on required-context drift.
  • No new Blacksmith minutes attributable to the sharding (job runner labels audited).

Out of Scope

  • Merge-queue batching configuration (deferred until concurrent-PR volume justifies it).
  • Changing the required-checks list or branch protection posture (ADR 0059).
  • Test content changes, flaky-test hunting, or build caching overhauls.

Further Notes

Governing: ADR 0059 (binding merge gate / green ratchet), issue #1307 (docs shim design and its lock-step contract). Context: recent Blacksmith cost-cap work (#2103) established the free-runner posture this Spec preserves.

Metadata

Metadata

Assignees

No one assigned

    Labels

    type:prdPermanent type marker: PRD planning artifact, not implementable

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions