Use setuptools_scm versioning

[ewu63]

Purpose

Closes #446. I opted to using setuptools_scm for a few reasons

• It builds the version at package build time, and ships the version file with the package. versioneer vendors some Python code that does this logic at import time which seems way too fragile
• setuptools_scm is used >20x more than versioneer on conda-forge based on search hits
• bump-my-version is a lot more manual IMO - the tag no longer becomes a single source of truth

We will have to adjust some stuff with conda-forge once this is merged, but this should work as-is. A few things to note:

• The versioning scheme is the default guess-next-dev, feel free to suggest alternatives, I have used post-release before. I don't think this ultimately matters since 99% of people should be using a tagged release, but for those that want traceability in a dev env this may matter
• The GitHub archive by default does not generate tag information in the tarball, leading to a loss of version information. This can be remedied via .git_archival.txt file which instructs GitHub to store some extra metadata - this can then be used by e.g. conda-forge when packaging. However, I think it's cleaner to just grab the version from the environment variable - the version is known in the PR. If we think there are other people that may care about this information, we can add this small file, but I have omitted it for now since I don't think people typically care about the archival tarball.

We can use this PR to iterate on this design, and if we like it we can mirror the approach elsewhere in the org. Some extra work was needed here to interface it to meson-python, standard integration with setuptools is trivial since this tool is called....setuptools_scm so it works out of the box.

Expected time until merged

Not urgent.

Type of change

• Bugfix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (non-backwards-compatible fix or feature)
• Code style update (formatting, renaming)
• Refactoring (no functional changes, no API changes)
• Documentation update
• Maintenance update
• Other (please describe)

Testing

Checklist

• I have run ruff check and ruff format to make sure the Python code adheres to PEP-8 and is consistently formatted
• I have formatted the Fortran code with fprettify or C/C++ code with clang-format as applicable
• I have run unit and regression tests which pass locally with my changes
• I have added new tests that prove my fix is effective or that my feature works
• I have added necessary documentation

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 54.22%. Comparing base (43e2e55) to head (51bcbb8).

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #490   +/-   ##
=======================================
  Coverage   54.22%   54.22%           
=======================================
  Files           1        1           
  Lines         225      225           
=======================================
  Hits          122      122           
  Misses        103      103           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[marcomangano]

This looks cool, I have a few (naive?) questions for now, as I try to understand the process:

• The changes in the .yml are files necessary to prevent the build/test on GHA to break, right?
• Can you expand on the difference between the two (and others?) versioning approaches? My understanding from the original issue is that we aim to get the version from the Github release. Is this related to commits that fall in-between releases, including running the latest version of main?
• The _version.py file is generated in doc/conf.py is a fix for RTD because it does not go through the full meson build which normally generates the file, correct?
• I am not sure about which env variable you refer to when talking about the tarball issues, could you clarify? I agree that any issue related to the archival tarball is secondary anyway.

[ewu63]

• The changes in the .yml are files necessary to prevent the build/test on GHA to break, right?

Yes, it's to checkout the repo with tags under GHA using the checkout action.

• Can you expand on the difference between the two (and others?) versioning approaches? My understanding from the original issue is that we aim to get the version from the Github release. Is this related to commits that fall in-between releases, including running the latest version of main?

If you are referring to the versioning scheme, it is for guessing what the version should be if it is not a tagged commit. For releases this should not matter since we only build on tags, but for local development (editable install for whatever reason, and for CI builds in a PR branch or on main post-merge) the version needs to be generated, and there are different ways of generating a new temporary non-released version. setuptools_scm has docs here and you can also test this out locally by invoking python -m setuptools_scm, e.g. right now it gives 2.16.1.dev10+g15a2e0d8e.

• The _version.py file is generated in doc/conf.py is a fix for RTD because it does not go through the full meson build which normally generates the file, correct?

It is doing two things. Note that we do not install the package into RTD when rendering the docs. So, first it must figure out the version to embed into Sphinx, and it does this via the get_version call. Second, it also writes a _version.py file which stores the version. This makes the package importable (the init.py requires this file, and missing it indicates that something has gone wrong with installation so it fails loudly). The import is used for 1) autodoc which generates the API docs, and 2) using the optionstable to generate the options. So, we need both and it's done here in one call to get_version().

• I am not sure about which env variable you refer to when talking about the tarball issues, could you clarify? I agree that any issue related to the archival tarball is secondary anyway.

I mean that, in the conda-forge recipe here, the version is already available as a jinja variable so we do not need to re-derive it from the source tarball. Conveniently the GitHub tag triggers a PR on conda-forge and that tag metadata info travels with it, so it all works out.

This is fundamentally changing how we version/package so it's good to talk about it and iron out the details.

[A-CGray]

Thanks for looking into this @ewu63. I want to take the time to fully understand how this works because, assuming we merge this, we'll probably extend this approach to our other MDOLab repos.

What exactly is the process of bumping the version and making a new release with this new system? My current understanding is we don't manually specify the version anywhere in the repo anymore, when we want to bump the version, the process is:

• We make a new tag and release as before
• setuptools_scm detects the new tag and updates _version.py
• setuptools_scm pushes the new _version.py to the repo?

Is this correct? If so then does setuptools_scm also alter the tag to point to the new commit with the updated _version.py?

[ewu63]

Sorry I didn't explain this clearly in the PR description. The process as follows:

• We decide at some point that the current commit on main will be released with a given version
• We tag the repo (either manually, or typically via a GitHub release where we create a new tag at the same time)

That's it. The version is not tracked as a file in the repo (which require a commit to change), but will be derived at build time using setuptools_scm from the git tag. This has a few advantages:

• Each install will have a unique version based on git history, even intermediate commits. Right now software will always say e.g. 1.6.0 until we bump it, even though it is actually 1.6.0+something
• We don't have to put the version in two places, risking them get out of sync. There is a single source of truth for the version.
• An addendum to the above: currently we parse the version string via regex in a few places which is a little fragile. This will shift to using setuptools_scm to figure out the version string and then provide it where we need.

What this does mean is that, without the git checkout, the software (if you e.g. delete the .git folder or download the release tarball which does not have the version file) will not have the version explicitly stated. If you attempt to install this, the installation should error out because the metadata cannot be found. The only relevant use case for this is if someone downloads the tarball, so I have mentioned a setting we can enable (by adding a .git_archival.txt file in the repo) which tells GitHub to dump additional information (such as the git commit/tag) into the tarball, such that if people are relying on that artifact without git, it is standalone. That's something we can do (and as I write this out I feel we should do this).

Downstream package deployment mechanisms (at the moment we only have conda-forge, but we may add PyPI in the future) will have to adjust their script to derive this version information not from the __init__.py but from the git tag, which is usually available. They will then provide the package that includes the version metadata, but it's stored elsewhere and not in a bare python file.