Conversation
…(release gate)
Source: artifact-keeper PR #3253 scripts/dogfood-vscode-openvsx.sh, the
`probe` subcommand (manifest fetch, gallery extensionquery POST with the
exact VS Code client body shape, {publisher}/{name}/latest lookup, the
code-server compatibility alias /gallery/vscode/{p}/{n}/latest, and the
"every assetUri/fallbackAssetUri/files[].source is AK-owned" assertions).
The dogfood script's other subcommands (vscodium-ui, code-server-ui,
install-*, update-*, TRACE_NETWORK) drive a real VS Code-family client
against an isolated profile and don't fit a headless, 120s-per-script
release-gate suite, so they do not survive the port.
Fixes a real bug while porting rather than copying it: the dogfood script's
inline jq asset-URL assertions (its lines ~167-170 and ~183-186) built
`[.assetUri?, .fallbackAssetUri?, (.files[]?.source?)]` without filtering
nulls before `all(startswith($prefix))`. `X?` on an ABSENT key still
produces `null`, and `null | startswith(_)` is a jq runtime error, which
aborts the whole `jq -e` and inverts the diagnosis (reads as "retains a
non-AK asset URL" when the real story is "field absent"). The new
_assert_asset_urls_ak_owned helper null-filters first, matching the shape
of the dogfood script's own already-fixed assert_ak_urls helper (~lines
105-116) instead of its buggy inline duplicates.
Registered in .github/workflows/release-gate.yml (node batch,
matrix.batch.scripts) and .github/workflows/format-tests.yml (node batch)
so the suite actually runs; tests/formats is not auto-discovered.
cazlo
marked this pull request as ready for review
August 15, 2026 21:51
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
fixes #375
Summary
artifact-keeper/artifact-keeper#3253 merged 2026-08-15, shipping AK as a governed VS Code extension gateway (composed Open VSX gallery pull-through, every asset URL rewritten onto AK). The PR's own dogfood harness carried a note that its
probesubcommand should move to the release gate; it never did. This addstests/formats/test-vscode-remote.sh, ported fromscripts/dogfood-vscode-openvsx.sh'sprobesubcommand, and registers it in both format-tests.yml and release-gate.yml (node batch in each).What it covers
{publisher}/{name}/latestresolves the same extension, all asset URLs AK-owned./gallery/vscode/{publisher}/{name}/latest) resolves identically to the primary latest lookup, checked as its own direct probe so a regression in the alias specifically can't hide behind the primary path passing.Not ported: the dogfood script's vscodium-ui, code-server-ui, install-, update-, and TRACE_NETWORK subcommands drive a real VS Code-family client against an isolated profile. That doesn't fit a headless, 120s-per-script release-gate suite, and the gate runners have no VSCodium/code-server binary on PATH.
I also fixed a real bug while porting rather than copying it forward: the dogfood script's inline jq asset-URL assertions build
[.assetUri?, .fallbackAssetUri?, (.files[]?.source?)]without filtering nulls beforeall(startswith($prefix)). The?suffix suppresses type errors, not missing keys, so a version object lackingfallbackAssetUristill yieldsnullfor that slot andnull | startswith(_)is a jq runtime error, which aborts the wholejq -eand inverts the diagnosis (reads as "retains a non-AK asset URL" when the truth is "field absent"). The new_assert_asset_urls_ak_ownedhelper null-filters first.Validation against the shipped composed gateway
artifact-keeper/artifact-keeper#3253 landed a gallery skeleton+synthesis rework on 2026-08-15, after I first wrote this port, so I re-validated every assertion against a live composed-gateway backend rather than trusting the pre-rework version of the script.
Bounded extensionquery responses (I measured 30 stable + 29 prerelease versions returned for the probe extension, against the newest-30-per-platform-x-channel cap, instead of raw upstream counts), the
/latestage-gate walk-back (200 with the newest allowed version instead of 404), and the flags-conditional presence offiles/assetUrion synthesized entries are all confirmed compatible with this script as written. None of the assertions check an exact or all-versions count, so the bounding doesn't reach them, and the script always queries withflags=511, which requests full asset detail, so it's never exposed to the omitted-field case. No assertion needed changing.I then ran the script end to end against that backend: 9 passed, 0 failed, 0 skipped in 3s, against the gate's 120s-per-script budget. The run also confirms the cleanup path — the throwaway repo it creates is gone afterwards — and that the backend accepts
format: vscodewith an Open VSX gallery-adapterupstream_urlthrough the ordinary repository-creation API.Red-proof
This is new-feature coverage, not a regression test for an existing defect, so there's no prior "broken behavior" to construct a red state against. The natural red state is that these routes didn't exist at all before artifact-keeper/artifact-keeper#3253 merged (404, not a wrong-answer 200) -- I'm treating route-absence as sufficient rather than adding a synthetic failure-injection leg, consistent with how this repo's other remote-proxy format tests (npm-remote, go-remote, etc.) were added as straight coverage additions without a constructed red leg.
Also found while here
Having to register this script by hand sent me looking at what else isn't registered.
tests/formatsis the only suite without auto-discovery — fourteen other suites run throughscripts/run-suite.sh, which globstests/<suite>/**/test-*.sh, while formats enumerates scripts by name in the workflow YAML. That gap leaves 59 of the 119tests/formats/test-*.shscripts named in no workflow at all, including a regression gate written for artifact-keeper/artifact-keeper#2477 that has never executed. Filed separately as #374 rather than folding a repo-wide fix into this PR.Test plan