Repository navigation
ci: normalize toolkit configuration and isolate CUDA 13 versions #3314
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -65,7 +65,7 @@ jobs: | |
|
|
||
| - name: Run CI tools unit tests | ||
| run: | | ||
| python -m pip install pytest pyyaml==6.0.3 | ||
| python -m pip install pytest pyyaml==6.0.3 setuptools-scm | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Pin pytest likewise to the other packages ( |
||
| # Standalone CI tool tests; skip repo-root conftest.py (imports cuda.pathfinder). | ||
| python -m pytest -v --noconftest ci/tools/tests | ||
|
|
||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -258,8 +258,15 @@ jobs: | |
| echo "Reusable artifacts: run \`${run_id}\` at \`${merge_base}\` on \`${BASE_REF}\`." | ||
| } >> "$GITHUB_STEP_SUMMARY" | ||
|
|
||
| - name: Test CI workplan planner | ||
| run: python3 -m unittest ci/tools/tests/test_compute_ci_plan.py | ||
| - name: Set up Python for CI tool tests | ||
| uses: ./.github/actions/setup-hardened-python | ||
| with: | ||
| python-version: "3.12" | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Why Python 3.12? We could use the latest, but probably the right conservative thing here to avoid breaking any outside users who might want to compile cuda-python themselves is to use the oldest supported (currently 3.11). |
||
|
|
||
| - name: Test CI tools | ||
| run: | | ||
| python -m pip install pytest==9.1.0 pyyaml==6.0.3 setuptools-scm | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Pin version to same as used in production (
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
EDIT: It looks like #3315 does this. |
||
| python -m pytest -v --noconftest ci/tools/tests | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is there an explicit reason for |
||
|
|
||
| - name: Compute CI workplan | ||
| id: workplan | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -80,10 +80,10 @@ repos: | |
|
|
||
| - id: check-pixi-cuda-version | ||
| name: Check pixi cuda-version pins track ci/versions.yml | ||
| entry: python ./ci/tools/check_pixi_cuda_version.py | ||
| entry: python -m ci.tools.check_pixi_cuda_version | ||
| language: python | ||
| additional_dependencies: [pyyaml==6.0.3] | ||
| files: '^(ci/versions\.yml|cuda_bindings_13/pixi\.toml|cuda_core/pixi\.toml)$' | ||
| files: '^(ci/versions\.yml|ci/tools/(bindings_config|check_pixi_cuda_version)\.py|cuda_bindings_13/pixi\.toml|cuda_core/pixi\.toml)$' | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Ditto on the issue above (to include 12 in here -- if we can't right now at least at some point in the future). |
||
| pass_filenames: false | ||
|
|
||
| - id: check-mempool-hygiene | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,2 @@ | ||
| # SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. | ||
| # SPDX-License-Identifier: Apache-2.0 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,2 @@ | ||
| # SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. | ||
| # SPDX-License-Identifier: Apache-2.0 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,60 @@ | ||
| # SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
|
|
||
| """Read bindings package roots and release roles from the existing toolkit pins.""" | ||
|
|
||
| from __future__ import annotations | ||
|
|
||
| from dataclasses import dataclass | ||
| from pathlib import Path | ||
|
|
||
| import yaml | ||
|
|
||
| DEFAULT_CONFIG = Path(__file__).resolve().parents[2] / "ci" / "versions.yml" | ||
|
|
||
|
|
||
| @dataclass(frozen=True) | ||
| class BindingsPackage: | ||
| package_root: str | ||
| toolkit: tuple[int, int, int] | ||
| release_status: str | ||
|
|
||
| @property | ||
| def toolkit_version(self) -> str: | ||
| return ".".join(map(str, self.toolkit)) | ||
|
|
||
| @property | ||
| def ctk_target(self) -> str: | ||
| return f"{self.toolkit[0]}.{self.toolkit[1]}" | ||
|
|
||
| @property | ||
| def cuda_major(self) -> str: | ||
| return str(self.toolkit[0]) | ||
|
|
||
| @property | ||
| def cuda_variant(self) -> str: | ||
| return f"cu{self.cuda_major}" | ||
|
|
||
|
|
||
| @dataclass(frozen=True) | ||
| class BindingsConfig: | ||
| package_roots: tuple[BindingsPackage, ...] | ||
|
|
||
| def get_package(self, package_root: str) -> BindingsPackage: | ||
| for package in self.package_roots: | ||
| if package.package_root == package_root: | ||
| return package | ||
| raise ValueError(f"unknown CUDA bindings package root: {package_root!r}") | ||
|
|
||
| def package_for_release_status(self, release_status: str) -> BindingsPackage: | ||
| return next(package for package in self.package_roots if package.release_status == release_status) | ||
|
|
||
|
|
||
| def load_config(path: Path = DEFAULT_CONFIG) -> BindingsConfig: | ||
| """Normalize legacy build/prev_build fields without changing their consumers.""" | ||
| raw = yaml.safe_load(path.read_text(encoding="utf-8")) | ||
| packages = [] | ||
| for field, release_status in (("build", "current"), ("prev_build", "maintenance")): | ||
| major, minor, patch = map(int, raw["cuda"][field]["version"].split(".")) | ||
| packages.append(BindingsPackage(f"cuda_bindings_{major}", (major, minor, patch), release_status)) | ||
| return BindingsConfig(tuple(packages)) |
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
|
|
@@ -9,31 +9,31 @@ | |||||
| import tomllib | ||||||
| from pathlib import Path | ||||||
|
|
||||||
| import yaml | ||||||
| from .bindings_config import load_config | ||||||
|
|
||||||
| ROOT = Path(__file__).resolve().parents[2] | ||||||
| VERSIONS_FILE_PATH = ROOT / "ci" / "versions.yml" | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Nit: Duplicated from |
||||||
| PIXI_FILES = [ROOT / d / "pixi.toml" for d in ("cuda_bindings_13", "cuda_core")] | ||||||
|
|
||||||
|
|
||||||
| def main() -> int: | ||||||
| """Verify cuda_bindings_13/cuda_core pixi pins match ci/versions.yml.""" | ||||||
| """Verify current bindings and cuda_core pixi pins match ci/versions.yml.""" | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Makes total sense for now. Maybe it's already higher up in the stack, but we should make sure to come back and add this check to legacy (12) as well. If not already in the stack, let's create an issue or just add a TODO comment here so it doesn't fall through the cracks. |
||||||
| if not VERSIONS_FILE_PATH.is_file(): | ||||||
| print(f"error: {VERSIONS_FILE_PATH} not found", file=sys.stderr) | ||||||
| return 2 | ||||||
|
Comment on lines
20
to
22
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Remove (see comment above). |
||||||
| try: | ||||||
| build_version = yaml.safe_load(VERSIONS_FILE_PATH.read_text(encoding="utf-8"))["cuda"]["build"]["version"] | ||||||
| except (KeyError, TypeError): | ||||||
| print(f"error: cuda.build.version not found in {VERSIONS_FILE_PATH}", file=sys.stderr) | ||||||
| current = load_config(VERSIONS_FILE_PATH).package_for_release_status("current") | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
| except (KeyError, TypeError, ValueError) as exc: | ||||||
| print(f"error: invalid CUDA toolkit pins in {VERSIONS_FILE_PATH}: {exc}", file=sys.stderr) | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
| return 2 | ||||||
|
|
||||||
| major, minor, *_ = build_version.split(".") | ||||||
| expected = f"{major}.{minor}.*" | ||||||
| cuda_feature = f"cu{major}" | ||||||
| build_version = current.toolkit_version | ||||||
| expected = f"{current.ctk_target}.*" | ||||||
| cuda_feature = current.cuda_variant | ||||||
|
|
||||||
| errors: list[str] = [] | ||||||
| checked: list[str] = [] | ||||||
| for path in PIXI_FILES: | ||||||
| for package_root in (current.package_root, "cuda_core"): | ||||||
| path = ROOT / package_root / "pixi.toml" | ||||||
| if not path.is_file(): | ||||||
| print(f"error: {path} not found", file=sys.stderr) | ||||||
| return 2 | ||||||
|
|
@@ -69,7 +69,7 @@ def main() -> int: | |||||
|
|
||||||
| if errors: | ||||||
| print( | ||||||
| f"error: cuda_bindings_13/cuda_core pixi cuda-version pins out of sync with " | ||||||
| f"error: {current.package_root}/cuda_core pixi cuda-version pins out of sync with " | ||||||
| f"ci/versions.yml cuda.build.version={build_version!r} " | ||||||
| f"(expected pin {expected!r}):", | ||||||
| file=sys.stderr, | ||||||
|
|
||||||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,107 @@ | ||
| # SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
|
|
||
| from __future__ import annotations | ||
|
|
||
| import runpy | ||
| import shutil | ||
| import subprocess | ||
| import tomllib | ||
| from pathlib import Path | ||
|
|
||
| import pytest | ||
| import setuptools | ||
| from packaging.version import Version | ||
| from setuptools_scm import get_version | ||
|
|
||
| from ci.tools import check_pixi_cuda_version | ||
| from ci.tools.bindings_config import BindingsPackage, load_config | ||
|
|
||
| ROOT = Path(__file__).resolve().parents[3] | ||
|
|
||
|
|
||
| @pytest.fixture | ||
| def legacy_config(tmp_path): | ||
| path = tmp_path / "versions.yml" | ||
| path.write_text('cuda:\n build:\n version: "13.4.2"\n prev_build:\n version: "12.9.1"\n') | ||
| return path | ||
|
|
||
|
|
||
| @pytest.mark.agent_authored(model="gpt-6") | ||
| def test_normalizes_legacy_toolkit_pins(legacy_config): | ||
| config = load_config(legacy_config) | ||
| current = config.package_for_release_status("current") | ||
| maintenance = config.get_package("cuda_bindings_12") | ||
| assert current == BindingsPackage("cuda_bindings_13", (13, 4, 2), "current") | ||
| assert maintenance == config.package_for_release_status("maintenance") | ||
| assert maintenance.toolkit_version == "12.9.1" | ||
| assert maintenance.ctk_target == "12.9" | ||
| assert maintenance.cuda_major == "12" | ||
| assert maintenance.cuda_variant == "cu12" | ||
|
|
||
|
|
||
| @pytest.fixture | ||
| def pixi_checkout(tmp_path, monkeypatch, legacy_config): | ||
| for package_root in ("cuda_bindings_13", "cuda_core"): | ||
| path = tmp_path / package_root / "pixi.toml" | ||
| path.parent.mkdir() | ||
| path.write_text( | ||
| '[workspace.build-variants]\ncuda-version = ["13.4.*"]\n' | ||
| '[feature.cu13.dependencies]\ncuda-version = "13.4.*"\n' | ||
| ) | ||
| monkeypatch.setattr(check_pixi_cuda_version, "ROOT", tmp_path) | ||
| monkeypatch.setattr(check_pixi_cuda_version, "VERSIONS_FILE_PATH", legacy_config) | ||
| return tmp_path | ||
|
|
||
|
|
||
| @pytest.mark.agent_authored(model="gpt-6") | ||
| def test_pixi_check_keeps_current_bindings_and_core_contract(pixi_checkout, capsys): | ||
| # The hook requires only the current bindings workspace. | ||
| assert check_pixi_cuda_version.main() == 0 | ||
| output = capsys.readouterr().out | ||
| assert "cuda_bindings_13/pixi.toml" in output | ||
| assert "cuda_core/pixi.toml" in output | ||
|
|
||
|
|
||
| @pytest.mark.agent_authored(model="gpt-6") | ||
| def test_pixi_check_reports_core_pin_drift(pixi_checkout, capsys): | ||
| path = pixi_checkout / "cuda_core" / "pixi.toml" | ||
| path.write_text(path.read_text().replace('cuda-version = "13.4.*"', 'cuda-version = "13.3.*"')) | ||
| assert check_pixi_cuda_version.main() == 1 | ||
| assert "cuda_core/pixi.toml: feature.cu13.dependencies.cuda-version='13.3.*'" in capsys.readouterr().err | ||
|
|
||
|
|
||
| @pytest.mark.agent_authored(model="gpt-6") | ||
| def test_cuda12_milestone_does_not_change_current_package_versions(tmp_path, monkeypatch): | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is of medium importance -- it will just affect the ability to run the test locally. :bot: ci/tools/tests/test_bindings_config.py:270 — MEDIUM — test robustness The test's
|
||
| current = load_config(ROOT / "ci" / "versions.yml").package_for_release_status("current") | ||
| for package_root, filename in (("cuda_bindings_13", "pyproject.toml"), ("cuda_python", "setup.py")): | ||
| destination = tmp_path / package_root / filename | ||
| destination.parent.mkdir() | ||
| shutil.copyfile(ROOT / package_root / filename, destination) | ||
|
|
||
| def git(*args): | ||
| return subprocess.run( # noqa: S603 | ||
| ["git", "-c", "user.name=CI test", "-c", "user.email=ci-test@example.invalid", *args], # noqa: S607 | ||
| cwd=tmp_path, | ||
| check=True, | ||
| capture_output=True, | ||
| text=True, | ||
| ) | ||
|
|
||
| git("init", "--quiet") | ||
| git("add", ".") | ||
| git("commit", "--quiet", "-m", "Current CUDA 13 release") | ||
| git("tag", f"v{current.toolkit_version}") | ||
| (tmp_path / "maintenance-change").touch() | ||
| git("add", ".") | ||
| git("commit", "--quiet", "-m", "CUDA 12 maintenance milestone") | ||
| git("tag", "v12.9.11a0") | ||
|
|
||
| with (tmp_path / "cuda_bindings_13" / "pyproject.toml").open("rb") as stream: | ||
| scm = tomllib.load(stream)["tool"]["setuptools_scm"] | ||
| bindings_version = get_version(root=tmp_path, git_describe_command=scm["git_describe_command"]) | ||
| metadata = {} | ||
| monkeypatch.setattr(setuptools, "setup", lambda **kwargs: metadata.update(kwargs)) | ||
| runpy.run_path(str(tmp_path / "cuda_python" / "setup.py")) | ||
| assert Version(bindings_version).release[:2] == current.toolkit[:2] | ||
| assert metadata["version"] == bindings_version | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -112,4 +112,4 @@ markers = [ | |
| [tool.setuptools_scm] | ||
| root = ".." | ||
| version_file = "cuda/bindings/_version.py" | ||
| git_describe_command = ["git", "describe", "--dirty", "--tags", "--long", "--match", "v*[0-9]*"] | ||
| git_describe_command = ["git", "describe", "--dirty", "--tags", "--long", "--match", "v13.4.*"] | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Note to me: this is a UNIX Do we need If it /must/ be this way, we need to update the release TODO instructions to add this extra step. By the time we tag the release, it will be too late to update this number, so it shouldn't be forgotten. |
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -11,7 +11,7 @@ | |
| version = get_version( | ||
| root="..", | ||
| relative_to=__file__, | ||
| git_describe_command=["git", "describe", "--dirty", "--tags", "--long", "--match", "v*[0-9]*"], | ||
| git_describe_command=["git", "describe", "--dirty", "--tags", "--long", "--match", "v13.4.*"], | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Ditto as above. |
||
| ) | ||
|
|
||
|
|
||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pin version to same as used in production (
setuptools_scm[simple]>=8,!=10.1")