Skip to content

Move to a src layout and ship type information - #40

Merged
lukeshingles merged 2 commits into
mainfrom
src-layout-type-stubs
Aug 9, 2026
Merged

Move to a src layout and ship type information#40
lukeshingles merged 2 commits into
mainfrom
src-layout-type-stubs

Conversation

@lukeshingles

Copy link
Copy Markdown
Member

The package was a single top-level module built from extinction.pyx at the repo root. Two consequences: it was impossible to ship PEP 561 type information, because the py.typed marker has to live inside a package directory, and the test suite could import from the working directory instead of from the installed extension.

extinction is now a package under src/, with the compiled module as extinction._extinction and a thin __init__.py re-exporting the public API. Imports, __version__ and the public API are unchanged for callersimport extinction; extinction.ccm89(...) works exactly as before.

Type information

py.typed plus a stub for the compiled module. This adopts the extinction.pyi that was already sitting untracked in the working tree rather than a stub written from scratch — it is more precise than what I would have written, typing unit as Literal["aa", "invum"] and wave as NDArray[np.float64] rather than a loose ArrayLike, which matches the implementation: the double[:] and np.ndarray parameters genuinely will not accept a plain list, and any unit outside those two raises ValueError.

Checked with mypy against the installed wheel:

note: Revealed type is "numpy.ndarray[tuple[Any, ...], numpy.dtype[numpy.float64]]"
note: Revealed type is "float"
error: Argument "unit" to "ccm89" has incompatible type "Literal['nonsense']"; expected "Literal['aa', 'invum']"
error: Argument 2 to "fm07" has incompatible type "str"; expected "float"

A wheel-bloat trap this introduced

Moving the sources inside the package directory meant MANIFEST.in started pulling them into the binary wheel — the generated _extinction.c alone is 1.4 MB, roughly four times the size of the compiled .so. Wheels went from 104 KB to 296 KB before I caught it. [tool.setuptools.exclude-package-data] keeps the .pyx and .c in the sdist, where they belong, and out of wheels.

Other mechanics

cythonize now gets an explicit include_path, because include "extern/bsplines.pxi" in the .pyx resolves against the project root, not the directory holding the .pyx — without it the build breaks once the source moves. extern/ deliberately stays at the repo root.

Verification

  • sdist and wheel both build; the sdist installs from source (--no-binary) and imports.
  • The wheel contains exactly __init__.py, the .so, _extinction.pyi and py.typed — no sources.
  • All 9 tests pass against the installed wheel from an unrelated working directory, with the module resolving to site-packages.
  • mypy resolves the stubs as above.
  • Docs still build, with only the pre-existing macOS-only autosummary filename-collision warning.

Note for review

test.py stays at the repo root rather than moving to tests/, to keep this diff to the layout change; the cibuildwheel test-command and MANIFEST.in reference it by path. Worth doing separately if you want it.

🤖 Generated with Claude Code

The package was a single top-level module built from a .pyx at the repo
root. That made it impossible to ship PEP 561 type information, since
the py.typed marker has to live inside a package directory, and it let
the test suite import from the working directory rather than from the
installed extension.

extinction is now a package under src/, with the compiled module as
extinction._extinction and a thin __init__.py re-exporting the public
API, so imports and __version__ are unchanged for callers.

- Add py.typed and the _extinction.pyi stub, so mypy resolves return
  types and rejects a bad unit string or a non-float a_v.
- Exclude the .pyx and generated .c from binary wheels. They now live
  inside the package directory, where MANIFEST.in would otherwise pull
  them into the wheel; the generated C alone is 1.4 MB, four times the
  size of the compiled module. They remain in the sdist.
- Pass include_path to cythonize, since the include of
  extern/bsplines.pxi resolves against the project root rather than
  the directory holding the .pyx.

Verified: sdist and wheel build, the sdist installs from source, the
wheel contains only __init__.py, the .so, the stub and py.typed, the
suite passes against the installed wheel from an unrelated directory,
mypy resolves the stubs, and the docs still build.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Moves extinction to a src package layout and adds PEP 561 typing support while preserving public imports.

Changes:

  • Packages the compiled extension as extinction._extinction.
  • Adds type stubs and a py.typed marker.
  • Updates build and distribution configuration.

Reviewed changes

Copilot reviewed 5 out of 7 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
src/extinction/py.typed Marks the package as typed.
src/extinction/_extinction.pyx Relocates the compiled implementation.
src/extinction/_extinction.pyi Defines public type information.
src/extinction/__init__.py Re-exports the public API.
setup.py Builds the relocated extension.
pyproject.toml Configures package discovery and package data.
MANIFEST.in Includes package sources in source distributions.
Suppressed comments (2)

src/extinction/_extinction.pyi:50

  • This signature incorrectly restricts apply to float64 arrays. Unlike the Cython wavelength APIs, apply is implemented with ordinary NumPy operations and valid calls with other floating dtypes (for example float32 extinction and flux arrays, including in-place use) are supported. The stub will reject those callers; use dtype-generic array types/overloads while preserving the in-place relationship to flux.
def apply(
    extinction: _Array, flux: _Array, inplace: bool = ...
) -> _Array: ...

src/extinction/_extinction.pyi:53

  • remove likewise accepts NumPy floating arrays other than float64, but _Array rejects every such valid call. Please give this convenience API dtype-generic array types/overloads rather than reusing the float64-only alias required by the compiled wavelength functions.
def remove(
    extinction: _Array, flux: _Array, inplace: bool = ...
) -> _Array: ...

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread src/extinction/_extinction.pyi Outdated
Two mismatches with runtime behaviour, both verified against the built
extension:

- Fitzpatrick99.r_v is cdef readonly, so assigning to it raises
  AttributeError. A plain attribute annotation told type checkers the
  assignment was fine; it is now a read-only property.
- apply() and remove() do plain numpy arithmetic rather than going
  through a typed memoryview, and accept a list, a float32 array or an
  int64 array. Annotating them as float64 arrays rejected valid calls.

The wavelength arguments keep NDArray[np.float64]: those go through
double[:] memoryviews and genuinely reject both lists (TypeError) and
float32 arrays (ValueError).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@lukeshingles
lukeshingles merged commit 4fe2caf into main Aug 9, 2026
18 checks passed
@lukeshingles
lukeshingles deleted the src-layout-type-stubs branch August 9, 2026 21:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants