CI: pin and verify sccache via a shared composite action (#3248)

CI: add temporary runner certificate trust store probe (#3251)

Add a report-only workflow (weekly schedule + workflow_dispatch with an
optional label-filter regex) that checks whether each hosted and
self-hosted runner label still trusts the compromised eMudhra
"emSign ECC Root CA - G3" anchor. For each label it probes:

- the OS trust store: emSign G3 absent from the CA bundle or Windows
  Root store, and curl / Invoke-WebRequest reject the test endpoint;
- the runner agent's bundled Node.js: Node 24 >= v24.22.1, and https.get
  rejects the test endpoint (plus an informational node20 check).

Linux GPU labels are probed in the ubuntu:24.04 container used by test
jobs; ubuntu-latest is probed before and after upgrading ca-certificates.
Probe files are added to IGNORED_PATHS so probe-only changes skip the
wheel builds.

Delete the workflow and ci/tools/runner_trust_probe* once all labels
pass and the rollout is complete.

Refs #3195

Author: juenglin

.github/actions/install-sccache/action.yml

@@ -0,0 +1,75 @@
+# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+#
+# SPDX-License-Identifier: Apache-2.0
+
+name: Install pinned rapidsai/sccache
+description: Download, verify, and install a pinned rapidsai/sccache binary on Linux
+
+# Regenerate hashes when bumping SCCACHE_VERSION:
+#   for arch in x86_64 aarch64; do
+#     curl -fsSL "https://github.com/rapidsai/sccache/releases/download/<tag>/sccache-${arch}-unknown-linux-musl.tar.gz" | sha256sum
+#   done
+# Cross-check against the `.sha256` files published on the GitHub release.
+#
+# Security note (eMudhra/emSign CA incident):
+# - Every version bump MUST repeat the embedded-roots check on the extracted
+#   `sccache` binary:
+#     grep -ac 'ISRG Root X1' sccache             # >0 means a root list is embedded
+#     grep -ac 'emSign ECC Root CA - G3' sccache  # must be 0
+#   Pass criteria: either no embedded root list (both return 0; binary defers
+#   to the OS trust store via `rustls-platform-verifier`, as verified for
+#   `rapidsai/sccache` v0.18.0-rapids.7), or an embedded `webpki-roots` list
+#   at 1.0.10+ where `ISRG Root X1` matches and `emSign ECC Root CA - G3`
+#   returns 0. Match the full root name: 1.0.10 removes only that emSign root
+#   and still ships other emSign roots.
+# - Do NOT switch to or bump `mozilla/sccache` to v0.19.0 (or later without
+#   repeating the roots check): `mozilla/sccache` v0.19.0 re-embeds
+#   `webpki-roots` 1.0.5, which still trusts `emSign ECC Root CA - G3`.
+
+runs:
+  using: composite
+  steps:
+    - name: Install pinned rapidsai/sccache
+      shell: bash --noprofile --norc -xeuo pipefail {0}
+      env:
+        SCCACHE_VERSION: v0.18.0-rapids.7
+      run: |
+        if [[ "$(uname -s)" != "Linux" ]]; then
+          echo "::error::install-sccache only supports Linux (got $(uname -s))" >&2
+          exit 1
+        fi
+
+        arch="$(uname -m)"
+        case "${arch}" in
+          x86_64)
+            expected_sha256="44106a2e01aeff99767e66b1a5ae6be074306c518458b367f75287de4532974c"
+            ;;
+          aarch64)
+            expected_sha256="1bad10208481786f2c2e943d7cca8006eaa7e620d122859ef2d61861a733d3d1"
+            ;;
+          *)
+            echo "::error::Unsupported architecture for install-sccache: ${arch}" >&2
+            exit 1
+            ;;
+        esac
+
+        tarball="sccache-${arch}-unknown-linux-musl.tar.gz"
+        url="https://github.com/rapidsai/sccache/releases/download/${SCCACHE_VERSION}/${tarball}"
+
+        tmp_dir="$(mktemp -d)"
+        trap 'rm -rf "${tmp_dir}"' EXIT
+
+        curl -fsSL -o "${tmp_dir}/${tarball}" "${url}"
+        (
+          cd "${tmp_dir}"
+          echo "${expected_sha256}  ${tarball}" | sha256sum --check --strict -
+        )
+
+        if [[ "$(id -u)" -eq 0 ]]; then
+          tar -C /usr/local/bin -xvzf "${tmp_dir}/${tarball}" --wildcards --strip-components=1 -x '*/sccache'
+        else
+          sudo tar -C /usr/local/bin -xvzf "${tmp_dir}/${tarball}" --wildcards --strip-components=1 -x '*/sccache'
+        fi
+
+        echo "SCCACHE_PATH=/usr/local/bin/sccache" >> "${GITHUB_ENV}"
+        /usr/local/bin/sccache --version

.github/workflows/build-wheel.yml

@@ -79,12 +79,9 @@ jobs:
         # the job level instead of linking into the diff.
         run: echo "::add-matcher::.github/matchers/compiler.json"
 
-      - name: Install latest rapidsai/sccache
+      - name: Install rapidsai/sccache
         if: ${{ startsWith(inputs.host-platform, 'linux') && (env.BUILD_BINDINGS == 'true' || env.BUILD_CORE == 'true') }}
-        run: |
-          curl -fsSL "https://github.com/rapidsai/sccache/releases/latest/download/sccache-$(uname -m)-unknown-linux-musl.tar.gz" \
-            | sudo tar -C /usr/local/bin -xvzf - --wildcards --strip-components=1 -x '*/sccache'
-          echo "SCCACHE_PATH=/usr/local/bin/sccache" >> "$GITHUB_ENV"
+        uses: ./.github/actions/install-sccache
 
       # The runner only exposes these values to actions, not to `run:` steps or
       # the cibuildwheel container, so re-export them for sccache's GHA backend.

.github/workflows/test-sdist-linux.yml

@@ -87,15 +87,10 @@ jobs:
           pathfinder_uri="file://$(realpath "${pathfinder_wheels[0]}")"
           printf 'cuda-pathfinder @ %s\n' "${pathfinder_uri}" | tee wheel-constraints/cuda-bindings.txt
 
-      # Cython packages need CTK + sccache.
-      # The env vars ACTIONS_CACHE_SERVICE_V2, ACTIONS_RESULTS_URL, and ACTIONS_RUNTIME_TOKEN
-      # are exposed by this action.
-      - name: Enable sccache
+      # Cython packages need CTK + sccache (local disk cache only in this job).
+      - name: Install rapidsai/sccache
         if: ${{ env.BUILD_BINDINGS == 'true' || env.BUILD_CORE == 'true' }}
-        uses: mozilla-actions/sccache-action@fc920bf0ec8de6ee65d409111f7ec508035751ba  # 0.0.11
-        with:
-          version: v0.18.0
-          disable_annotations: 'true'
+        uses: ./.github/actions/install-sccache
 
       - name: Setup proxy cache
         if: ${{ env.BUILD_BINDINGS == 'true' || env.BUILD_CORE == 'true' }}