Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/ci-nightly.yml
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ jobs:

- name: Run CI tools unit tests
run: |
python -m pip install pytest pyyaml==6.0.3
python -m pip install pytest pyyaml==6.0.3 setuptools-scm

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pin version to same as used in production (setuptools_scm[simple]>=8,!=10.1")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pin pytest likewise to the other packages (pytest==9.1.0)

# Standalone CI tool tests; skip repo-root conftest.py (imports cuda.pathfinder).
python -m pytest -v --noconftest ci/tools/tests

Expand Down
11 changes: 9 additions & 2 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -258,8 +258,15 @@ jobs:
echo "Reusable artifacts: run \`${run_id}\` at \`${merge_base}\` on \`${BASE_REF}\`."
} >> "$GITHUB_STEP_SUMMARY"

- name: Test CI workplan planner
run: python3 -m unittest ci/tools/tests/test_compute_ci_plan.py
- name: Set up Python for CI tool tests
uses: ./.github/actions/setup-hardened-python
with:
python-version: "3.12"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why Python 3.12? We could use the latest, but probably the right conservative thing here to avoid breaking any outside users who might want to compile cuda-python themselves is to use the oldest supported (currently 3.11).


- name: Test CI tools
run: |
python -m pip install pytest==9.1.0 pyyaml==6.0.3 setuptools-scm

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pin version to same as used in production (setuptools_scm[simple]>=8,!=10.1")

@mdboom mdboom Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: Since ci/tools is now a package, it should have a pyproject.toml and specify its dependencies there (making it easier to run the tests locally). That can be a "side quest", though. Maybe create an issue to come back to it.

EDIT: It looks like #3315 does this.

python -m pytest -v --noconftest ci/tools/tests

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there an explicit reason for --noconftest? That will be confusing if we ever want a conftest in the future.


- name: Compute CI workplan
id: workplan
Expand Down
4 changes: 2 additions & 2 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -80,10 +80,10 @@ repos:

- id: check-pixi-cuda-version
name: Check pixi cuda-version pins track ci/versions.yml
entry: python ./ci/tools/check_pixi_cuda_version.py
entry: python -m ci.tools.check_pixi_cuda_version
language: python
additional_dependencies: [pyyaml==6.0.3]
files: '^(ci/versions\.yml|cuda_bindings_13/pixi\.toml|cuda_core/pixi\.toml)$'
files: '^(ci/versions\.yml|ci/tools/(bindings_config|check_pixi_cuda_version)\.py|cuda_bindings_13/pixi\.toml|cuda_core/pixi\.toml)$'

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto on the issue above (to include 12 in here -- if we can't right now at least at some point in the future).

pass_filenames: false

- id: check-mempool-hygiene
Expand Down
2 changes: 2 additions & 0 deletions ci/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
2 changes: 2 additions & 0 deletions ci/tools/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
60 changes: 60 additions & 0 deletions ci/tools/bindings_config.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

"""Read bindings package roots and release roles from the existing toolkit pins."""

from __future__ import annotations

from dataclasses import dataclass
from pathlib import Path

import yaml

DEFAULT_CONFIG = Path(__file__).resolve().parents[2] / "ci" / "versions.yml"


@dataclass(frozen=True)
class BindingsPackage:
package_root: str
toolkit: tuple[int, int, int]
release_status: str

@property
def toolkit_version(self) -> str:
return ".".join(map(str, self.toolkit))

@property
def ctk_target(self) -> str:
return f"{self.toolkit[0]}.{self.toolkit[1]}"

@property
def cuda_major(self) -> str:
return str(self.toolkit[0])

@property
def cuda_variant(self) -> str:
return f"cu{self.cuda_major}"


@dataclass(frozen=True)
class BindingsConfig:
package_roots: tuple[BindingsPackage, ...]

def get_package(self, package_root: str) -> BindingsPackage:
for package in self.package_roots:
if package.package_root == package_root:
return package
raise ValueError(f"unknown CUDA bindings package root: {package_root!r}")

def package_for_release_status(self, release_status: str) -> BindingsPackage:
return next(package for package in self.package_roots if package.release_status == release_status)


def load_config(path: Path = DEFAULT_CONFIG) -> BindingsConfig:
"""Normalize legacy build/prev_build fields without changing their consumers."""
raw = yaml.safe_load(path.read_text(encoding="utf-8"))
packages = []
for field, release_status in (("build", "current"), ("prev_build", "maintenance")):
major, minor, patch = map(int, raw["cuda"][field]["version"].split("."))
packages.append(BindingsPackage(f"cuda_bindings_{major}", (major, minor, patch), release_status))
return BindingsConfig(tuple(packages))
22 changes: 11 additions & 11 deletions ci/tools/check_pixi_cuda_version.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,31 +9,31 @@
import tomllib
from pathlib import Path

import yaml
from .bindings_config import load_config

ROOT = Path(__file__).resolve().parents[2]
VERSIONS_FILE_PATH = ROOT / "ci" / "versions.yml"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: Duplicated from bindings_config.py which should own knowing where the canonical location of this file is. Rather than checking for its existence here, bindings_config.py should (and already does) raise an exception when the file doesn't exist.

PIXI_FILES = [ROOT / d / "pixi.toml" for d in ("cuda_bindings_13", "cuda_core")]


def main() -> int:
"""Verify cuda_bindings_13/cuda_core pixi pins match ci/versions.yml."""
"""Verify current bindings and cuda_core pixi pins match ci/versions.yml."""

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes total sense for now. Maybe it's already higher up in the stack, but we should make sure to come back and add this check to legacy (12) as well. If not already in the stack, let's create an issue or just add a TODO comment here so it doesn't fall through the cracks.

if not VERSIONS_FILE_PATH.is_file():
print(f"error: {VERSIONS_FILE_PATH} not found", file=sys.stderr)
return 2
Comment on lines 20 to 22

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove (see comment above).

try:
build_version = yaml.safe_load(VERSIONS_FILE_PATH.read_text(encoding="utf-8"))["cuda"]["build"]["version"]
except (KeyError, TypeError):
print(f"error: cuda.build.version not found in {VERSIONS_FILE_PATH}", file=sys.stderr)
current = load_config(VERSIONS_FILE_PATH).package_for_release_status("current")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
current = load_config(VERSIONS_FILE_PATH).package_for_release_status("current")
current = load_config().package_for_release_status("current")

except (KeyError, TypeError, ValueError) as exc:
print(f"error: invalid CUDA toolkit pins in {VERSIONS_FILE_PATH}: {exc}", file=sys.stderr)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
print(f"error: invalid CUDA toolkit pins in {VERSIONS_FILE_PATH}: {exc}", file=sys.stderr)
print(f"error: invalid CUDA toolkit pins in {bindings_config.DEFAULT_CONFIG}: {exc}", file=sys.stderr)

return 2

major, minor, *_ = build_version.split(".")
expected = f"{major}.{minor}.*"
cuda_feature = f"cu{major}"
build_version = current.toolkit_version
expected = f"{current.ctk_target}.*"
cuda_feature = current.cuda_variant

errors: list[str] = []
checked: list[str] = []
for path in PIXI_FILES:
for package_root in (current.package_root, "cuda_core"):
path = ROOT / package_root / "pixi.toml"
if not path.is_file():
print(f"error: {path} not found", file=sys.stderr)
return 2
Expand Down Expand Up @@ -69,7 +69,7 @@ def main() -> int:

if errors:
print(
f"error: cuda_bindings_13/cuda_core pixi cuda-version pins out of sync with "
f"error: {current.package_root}/cuda_core pixi cuda-version pins out of sync with "
f"ci/versions.yml cuda.build.version={build_version!r} "
f"(expected pin {expected!r}):",
file=sys.stderr,
Expand Down
107 changes: 107 additions & 0 deletions ci/tools/tests/test_bindings_config.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

from __future__ import annotations

import runpy
import shutil
import subprocess
import tomllib
from pathlib import Path

import pytest
import setuptools
from packaging.version import Version
from setuptools_scm import get_version

from ci.tools import check_pixi_cuda_version
from ci.tools.bindings_config import BindingsPackage, load_config

ROOT = Path(__file__).resolve().parents[3]


@pytest.fixture
def legacy_config(tmp_path):
path = tmp_path / "versions.yml"
path.write_text('cuda:\n build:\n version: "13.4.2"\n prev_build:\n version: "12.9.1"\n')
return path


@pytest.mark.agent_authored(model="gpt-6")
def test_normalizes_legacy_toolkit_pins(legacy_config):
config = load_config(legacy_config)
current = config.package_for_release_status("current")
maintenance = config.get_package("cuda_bindings_12")
assert current == BindingsPackage("cuda_bindings_13", (13, 4, 2), "current")
assert maintenance == config.package_for_release_status("maintenance")
assert maintenance.toolkit_version == "12.9.1"
assert maintenance.ctk_target == "12.9"
assert maintenance.cuda_major == "12"
assert maintenance.cuda_variant == "cu12"


@pytest.fixture
def pixi_checkout(tmp_path, monkeypatch, legacy_config):
for package_root in ("cuda_bindings_13", "cuda_core"):
path = tmp_path / package_root / "pixi.toml"
path.parent.mkdir()
path.write_text(
'[workspace.build-variants]\ncuda-version = ["13.4.*"]\n'
'[feature.cu13.dependencies]\ncuda-version = "13.4.*"\n'
)
monkeypatch.setattr(check_pixi_cuda_version, "ROOT", tmp_path)
monkeypatch.setattr(check_pixi_cuda_version, "VERSIONS_FILE_PATH", legacy_config)
return tmp_path


@pytest.mark.agent_authored(model="gpt-6")
def test_pixi_check_keeps_current_bindings_and_core_contract(pixi_checkout, capsys):
# The hook requires only the current bindings workspace.
assert check_pixi_cuda_version.main() == 0
output = capsys.readouterr().out
assert "cuda_bindings_13/pixi.toml" in output
assert "cuda_core/pixi.toml" in output


@pytest.mark.agent_authored(model="gpt-6")
def test_pixi_check_reports_core_pin_drift(pixi_checkout, capsys):
path = pixi_checkout / "cuda_core" / "pixi.toml"
path.write_text(path.read_text().replace('cuda-version = "13.4.*"', 'cuda-version = "13.3.*"'))
assert check_pixi_cuda_version.main() == 1
assert "cuda_core/pixi.toml: feature.cu13.dependencies.cuda-version='13.3.*'" in capsys.readouterr().err


@pytest.mark.agent_authored(model="gpt-6")
def test_cuda12_milestone_does_not_change_current_package_versions(tmp_path, monkeypatch):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is of medium importance -- it will just affect the ability to run the test locally.

:bot: ci/tools/tests/test_bindings_config.py:270 — MEDIUM — test robustness

The test's git helper inherits the developer's global git config. A contributor with commit.gpgsign=true, tag.gpgSign=true or global hooks will see git commit or git tag fail or prompt. -c user.name and -c user.email are set, but nothing else is isolated.

  • Fix: pass -c commit.gpgsign=false -c tag.gpgsign=false -c core.hooksPath=/dev/null, or set GIT_CONFIG_GLOBAL=/dev/null, GIT_CONFIG_NOSYSTEM=1 and GIT_DIR/GIT_WORK_TREE unset in the subprocess env.
  • Failure message: if the selector falls behind the toolkit, get_version raises a setuptools-scm "no tag matched" error instead of a clear assertion. Asserting the selector prefix explicitly would make the failure readable.

current = load_config(ROOT / "ci" / "versions.yml").package_for_release_status("current")
for package_root, filename in (("cuda_bindings_13", "pyproject.toml"), ("cuda_python", "setup.py")):
destination = tmp_path / package_root / filename
destination.parent.mkdir()
shutil.copyfile(ROOT / package_root / filename, destination)

def git(*args):
return subprocess.run( # noqa: S603
["git", "-c", "user.name=CI test", "-c", "user.email=ci-test@example.invalid", *args], # noqa: S607
cwd=tmp_path,
check=True,
capture_output=True,
text=True,
)

git("init", "--quiet")
git("add", ".")
git("commit", "--quiet", "-m", "Current CUDA 13 release")
git("tag", f"v{current.toolkit_version}")
(tmp_path / "maintenance-change").touch()
git("add", ".")
git("commit", "--quiet", "-m", "CUDA 12 maintenance milestone")
git("tag", "v12.9.11a0")

with (tmp_path / "cuda_bindings_13" / "pyproject.toml").open("rb") as stream:
scm = tomllib.load(stream)["tool"]["setuptools_scm"]
bindings_version = get_version(root=tmp_path, git_describe_command=scm["git_describe_command"])
metadata = {}
monkeypatch.setattr(setuptools, "setup", lambda **kwargs: metadata.update(kwargs))
runpy.run_path(str(tmp_path / "cuda_python" / "setup.py"))
assert Version(bindings_version).release[:2] == current.toolkit[:2]
assert metadata["version"] == bindings_version
2 changes: 1 addition & 1 deletion cuda_bindings_13/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -112,4 +112,4 @@ markers = [
[tool.setuptools_scm]
root = ".."
version_file = "cuda/bindings/_version.py"
git_describe_command = ["git", "describe", "--dirty", "--tags", "--long", "--match", "v*[0-9]*"]
git_describe_command = ["git", "describe", "--dirty", "--tags", "--long", "--match", "v13.4.*"]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note to me: this is a UNIX glob pattern not a regex.

Do we need v13.4.* here? It kind of defeats the purpose of "simply" tagging to make a release, as we would now require a bump to this number at each minor release.

If it /must/ be this way, we need to update the release TODO instructions to add this extra step. By the time we tag the release, it will be too late to update this number, so it shouldn't be forgotten.

2 changes: 1 addition & 1 deletion cuda_python/setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
version = get_version(
root="..",
relative_to=__file__,
git_describe_command=["git", "describe", "--dirty", "--tags", "--long", "--match", "v*[0-9]*"],
git_describe_command=["git", "describe", "--dirty", "--tags", "--long", "--match", "v13.4.*"],

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto as above.

)


Expand Down
Loading