feat: hash-suffixed GCS deploy + upload-release-to-gcs script #21
+154
−10
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,122 @@ | ||
| #!/usr/bin/env bash | ||
| # Uploads vmlinux-*-{amd64,arm64}.bin assets from a fc-kernels GitHub release | ||
| # to GCS at: | ||
| # gs://<bucket>/vmlinux-<version>-<short_hash>/<arch>/vmlinux.bin | ||
| # | ||
| # Existing objects are never overwritten. The legacy non-arch release asset | ||
| # (vmlinux-<version>.bin) is intentionally skipped — under a fresh | ||
| # hash-suffixed version name there is no pre-existing flat layout to be | ||
| # backwards-compatible with. | ||
| # | ||
| # Usage: | ||
| # ./scripts/upload-release-to-gcs.sh --hash <hash> --bucket <bucket> [--dry-run] [--repo <repo>] | ||
| # | ||
| # Options: | ||
| # --hash <hash> Commit hash (full or short prefix) of the build to upload. | ||
| # --bucket <bucket> Target bucket (with optional path prefix), e.g. | ||
| # my-bucket | ||
| # my-bucket/kernels | ||
| # gs://my-bucket/kernels | ||
| # --repo <repo> GitHub repo (default: e2b-dev/fc-kernels). | ||
| # --dry-run Print what would be uploaded without writing. | ||
| # -h, --help Show this help. | ||
|
|
||
| set -euo pipefail | ||
|
|
||
| REPO="e2b-dev/fc-kernels" | ||
| HASH="" | ||
| BUCKET="" | ||
| DRY_RUN=false | ||
|
|
||
| usage() { sed -n '2,/^$/p' "$0" | sed 's/^# \{0,1\}//'; exit "${1:-0}"; } | ||
|
|
||
| while [[ $# -gt 0 ]]; do | ||
| case "$1" in | ||
| --hash) HASH="${2:?--hash needs a value}"; shift 2 ;; | ||
| --bucket) BUCKET="${2:?--bucket needs a value}"; shift 2 ;; | ||
| --repo) REPO="${2:?--repo needs a value}"; shift 2 ;; | ||
| --dry-run) DRY_RUN=true; shift ;; | ||
| -h|--help) usage 0 ;; | ||
| *) echo "Unknown argument: $1" >&2; usage 1 ;; | ||
| esac | ||
| done | ||
|
|
||
| [[ -n "$HASH" ]] || { echo "ERROR: --hash is required" >&2; usage 1; } | ||
| [[ -n "$BUCKET" ]] || { echo "ERROR: --bucket is required" >&2; usage 1; } | ||
|
|
||
| command -v gh >/dev/null || { echo "ERROR: gh CLI not found" >&2; exit 1; } | ||
| command -v gcloud >/dev/null || { echo "ERROR: gcloud CLI not found" >&2; exit 1; } | ||
|
|
||
| BUCKET="${BUCKET#gs://}" | ||
| BUCKET="${BUCKET%/}" | ||
| BUCKET_URI="gs://${BUCKET}" | ||
|
|
||
| if ! FULL_HASH=$(gh api "repos/$REPO/commits/$HASH" --jq '.sha' 2>/dev/null) \ | ||
| || [[ -z "$FULL_HASH" || "$FULL_HASH" == "null" ]]; then | ||
| echo "ERROR: commit '$HASH' not found in $REPO" >&2 | ||
| exit 1 | ||
| fi | ||
| SHORT_HASH="${FULL_HASH:0:7}" | ||
|
|
||
| # The release workflow writes "Built from commit <full_sha>" into the body, so | ||
| # we locate the matching release by scanning bodies. | ||
| RELEASE_TAG=$(gh api "repos/$REPO/releases?per_page=100" --paginate \ | ||
| --jq ".[] | select((.body // \"\") | contains(\"$FULL_HASH\")) | .tag_name" \ | ||
| | head -1) | ||
|
There was a problem hiding this comment. SIGPIPE breaks script under
|
||
|
|
||
| if [[ -z "$RELEASE_TAG" ]]; then | ||
| echo "ERROR: no release in $REPO references commit $FULL_HASH" >&2 | ||
| exit 1 | ||
| fi | ||
|
|
||
| echo "Release: $RELEASE_TAG (commit ${SHORT_HASH})" | ||
| echo "Target: ${BUCKET_URI}" | ||
| $DRY_RUN && echo "Mode: dry-run" | ||
|
|
||
| mapfile -t ASSETS < <(gh release view "$RELEASE_TAG" --repo "$REPO" --json assets \ | ||
| --jq '.assets[] | select(.name | test("^vmlinux-.*\\.bin$")) | .name') | ||
|
|
||
| if [[ "${#ASSETS[@]}" -eq 0 ]]; then | ||
| echo "ERROR: release $RELEASE_TAG has no vmlinux-*.bin assets" >&2 | ||
| exit 1 | ||
| fi | ||
|
|
||
| TMP_DIR=$(mktemp -d) | ||
| trap 'rm -rf "$TMP_DIR"' EXIT | ||
|
|
||
| uploaded=0 | ||
| skipped=0 | ||
| for asset in "${ASSETS[@]}"; do | ||
| if [[ ! "$asset" =~ ^vmlinux-(.+)-(amd64|arm64)\.bin$ ]]; then | ||
| # Legacy non-arch release asset or unrecognized name — not uploaded. | ||
| continue | ||
| fi | ||
| version="${BASH_REMATCH[1]}" | ||
| arch="${BASH_REMATCH[2]}" | ||
| dst="${BUCKET_URI}/vmlinux-${version}-${SHORT_HASH}/${arch}/vmlinux.bin" | ||
|
|
||
| if gcloud storage ls "$dst" >/dev/null 2>&1; then | ||
| echo " EXISTS $dst" | ||
| skipped=$((skipped + 1)) | ||
| continue | ||
| fi | ||
|
|
||
| if $DRY_RUN; then | ||
| echo " WOULD $asset -> $dst" | ||
| continue | ||
| fi | ||
|
|
||
| echo " UPLOAD $asset -> $dst" | ||
| gh release download "$RELEASE_TAG" --repo "$REPO" \ | ||
| --pattern "$asset" --dir "$TMP_DIR" --clobber >/dev/null | ||
| gcloud storage cp "$TMP_DIR/$asset" "$dst" | ||
| rm -f "$TMP_DIR/$asset" | ||
| uploaded=$((uploaded + 1)) | ||
| done | ||
|
|
||
| echo "" | ||
| if $DRY_RUN; then | ||
| echo "Dry run complete. Already in GCS: $skipped." | ||
| else | ||
| echo "Done. Uploaded: $uploaded, already in GCS: $skipped." | ||
| fi | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Deploy matrix missing the
publicenvironmentHigh Severity
The PR description explicitly states the deploy matrix covers four environments (
staging,juliett,foxtrot,public) and mentions "The four environments must exist," but the matrix only lists three:[staging, juliett, foxtrot]. Thepublicenvironment is missing, meaning production/public buckets won't receive kernel deployments.Reviewed by Cursor Bugbot for commit 5e08233. Configure here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Bugbot Autofix determined this is a false positive.
The removal of 'public' from the deploy matrix was intentional as evidenced by commit 5e08233 'drop public from deploy matrix'.
You can send follow-ups to the cloud agent here.