Repository navigation
release: publish ahead of a merge, promote after it - #73
Merged
Merged
Conversation
test_damage_since_an_older_build_is_forced built its patches once and entered them on each pass of its loop. The ExitStack the first pass exited no longer patched anything, so the second pass reached the live GitHub API and failed whenever that answer did not parse. The patches are now made fresh for each pass. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Target releases were tagged by target, arch and kernel version alone, so a rebuild whose inputs changed but whose kernel did not had to replace the release master fetches, and could only be uploaded after the change merged. Uploads take hours. Releases are now tagged rel-<target>-<arch>-<kver>[-<variant>]-<key>, where the key digests the publishing checkout's container, kernel and image inputs, the declared ZFS, the release schema and a BUILD_GENERATION string that is replaced whenever build code changes what artifacts contain. Notes name each release's target, arch, declared kernel, variant, ltvm version and commit, and carry a completion mark written last. From a branch, publish uploads the release as a draft; fetch never sees drafts. After the merge, `ltvm target promote` on the default branch makes public the drafts whose key matches it, in seconds. From the default branch, publish goes public directly. Fetch takes the published release whose key matches the checkout, otherwise the newest published one with a note, and asks for an update when that came from a newer ltvm. On a publish or promote at the default branch's tip, published rel- releases of the same kernel and variant 14 or more days old are pruned, keeping the newest two. Old unprefixed releases are never written or pruned: older ltvm sees only those. Publish now checks every component it packages with the builds' own staleness rules, refuses a dirty checkout or inputs not in the commit, records the build container by image id and exports it by that id, keeps a release a draft until every asset and its notes are written, and refuses a second manifest in a release. Kernels are always built in the base variant's container. Builds made before this change stay current for ordinary use; publishing needs a rebuild. docs/RELEASING.md describes the workflow and every check. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rebuilt target releases can now be uploaded from a pull request branch before it merges, and made public in seconds after it does. Until now a rebuild had to wait for the merge, and the uploads take hours: #60's 25 releases took about three, and anyone fetching on master meanwhile got artifacts their checkout called stale.
How it works
rel-<target>-<arch>-<kver>[-<variant>]-<key>. The key digests what the publishing checkout would build from: the container, kernel and image input hashes, the declared ZFS, the release schema, andBUILD_GENERATION, a string replaced (never incremented) when ltvm's build code changes what artifacts contain. Two PRs bumping it conflict in git.ltvm target promoteon a clean default-branch checkout makes public each complete draft whose key matches what that checkout computes, newest build per kernel only. A PR whose inputs no longer match master after the merge promotes nothing, safely.rel-releases of the same kernel and variant 14 or more days old are deleted, keeping the newest two. Drafts and old unprefixed releases are never pruned.<target>-, never seesrel-releases, and keeps fetching the existing releases, which are never written again.Publish checks more than before
Publish now refuses a release unless every component it packages is current by the builds' own staleness rules (container, kernel, image against the Lustre baked into it, Lustre against the kernel's Module.symvers, ZFS, MOFED kmods), the checkout is clean and every file the inputs read is committed, and each kernel and image records this build generation and the container it was built in. It exports the build container by the image id it recorded, keeps a release a draft until every asset and its notes (with a completion mark written last) are uploaded, and refuses a second manifest in one release.
That fixes several things that were wrong before this change:
--no-lustrestill shipped Lustre baked into the image;--tagcould put two kernels' manifests in one release, or let a bootable publish overwrite a target release (target releases no longer take--tag).Also:
tests/test_release_build.py's damage test reused oneExitStackacross loop passes and could reach the live GitHub API (separate commit).Cost
Input hashes and their goldens are unchanged, so nobody's existing builds go stale. Artifacts built before this change cannot be published until rebuilt, because they record no build generation or container id. That is a one-off for the next release rebuild.
docs/RELEASING.md has the workflow and every check.
Tested
build --iidfile,save -o,load -i) were checked against podman's client code, not run on a Mac.Review
Eighteen rounds of independent review (Codex gpt-6.1-sol). The first three found that a fallback based on git ancestry and version ordering could not be made safe across squash merges, forks and installed copies; the design was replaced by drafts plus promote. Later rounds found progressively narrower problems in publish's provenance checks, ending with publish reusing the builds' own staleness functions. Round 17 found nothing; round 18's two findings are fixed here.
Declined, with reasons:
stableand a declared series resolving to the same version collide. Mainline releases are not published, and the marker predates this change.🤖 Generated with Claude Code