Add xImage reconstruction to the shared cuPhoton CLI

.github/workflows/ci.yml

@@ -74,7 +74,7 @@ jobs:
       - run: uv python install "${PYTHON_VERSION}"
       - run: >-
           uv sync --locked --python "${PYTHON_VERSION}"
-          --extra dev --extra torch --extra viz --extra photometry
+          --extra dev --extra torch --extra viz --extra photometry --extra ximage --extra rubin
       - run: make ci-test-cpu
       - name: Build native xRay CPU extension
         if: matrix.name == 'linux-x64' && matrix.python == '3.12'
@@ -112,7 +112,7 @@ jobs:
       - name: Run synthetic CPU quickstarts
         if: matrix.name == 'linux-x64' && matrix.python == '3.12'
         run: >-
-          uv run --locked --extra dev --extra torch --extra viz --extra photometry
+          uv run --locked --extra dev --extra torch --extra viz --extra photometry --extra ximage --extra rubin
           python examples/run_quickstarts.py --profile cpu
           --output-dir "${RUNNER_TEMP}/cuphoton-quickstart"
 

CITATION.cff

@@ -3,7 +3,9 @@
 # SPDX-License-Identifier: Apache-2.0
 
 cff-version: 1.2.0
-message: "If you use cuPhoton, please cite this software release."
+message: >-
+  If you use cuPhoton, please cite this software release. For xImage
+  reconstruction, also cite ImageMM; see CITATION.md.
 title: "cuPhoton: GPU-accelerated astronomy and imaging reference workflows"
 type: software
 authors:
@@ -12,3 +14,21 @@ version: 0.1.3
 license: Apache-2.0
 repository-code: "https://github.com/NVIDIA/cuPhoton"
 url: "https://github.com/NVIDIA/cuPhoton"
+references:
+  - type: article
+    title: "ImageMM: Joint Multi-frame Image Restoration and Super-resolution"
+    authors:
+      - family-names: Sukurdeep
+        given-names: Yashil
+      - family-names: Budavári
+        given-names: Tamás
+      - family-names: Connolly
+        given-names: Andrew J.
+      - family-names: Navarro
+        given-names: Fausto
+    journal: The Astronomical Journal
+    year: 2025
+    volume: 170
+    issue: 4
+    notes: "Article 233."
+    doi: 10.3847/1538-3881/adfb72

CITATION.md

@@ -18,3 +18,18 @@ bibliography formats.
 The current citation metadata does not include an archived DOI. If the
 release you use publishes one, prefer its DOI while retaining the exact
 version or commit used.
+
+## xImage
+
+If you use xImage reconstruction in a publication, also cite the ImageMM
+method paper:
+
+> Yashil Sukurdeep, Tamás Budavári, Andrew J. Connolly, and Fausto Navarro.
+> “ImageMM: Joint Multi-frame Image Restoration and Super-resolution.”
+> *The Astronomical Journal*, 170(4), article 233, 2025.
+> [doi:10.3847/1538-3881/adfb72](https://doi.org/10.3847/1538-3881/adfb72)
+
+Cite the cuPhoton version and commit used for the implementation; the
+ImageMM citation credits the method. See the
+[xImage component guide](docs/components/ximage.md) for supported behavior
+and validation limits.

Makefile

@@ -2,9 +2,9 @@
 #
 # SPDX-License-Identifier: Apache-2.0
 
-.PHONY: sync sync-gpu sync-cutile lock lock-check lint typecheck format test test-cpu test-core test-xdr test-xfit test-xfit-real test-xpois test-xscan test-xrep test-xray test-gpu test-cutile clean-dist build package-check wheels conda release-check ci-lint ci-test-cpu hooks
+.PHONY: sync sync-gpu sync-cutile lock lock-check lint typecheck format test test-cpu test-core test-xdr test-xfit test-ximage test-xfit-real test-xpois test-xscan test-xrep test-xray test-gpu test-cutile clean-dist build package-check wheels conda release-check ci-lint ci-test-cpu hooks
 
-CPU_EXTRAS = --extra dev --extra torch --extra viz --extra photometry --extra xmonitor-inference
+CPU_EXTRAS = --extra dev --extra torch --extra viz --extra photometry --extra xmonitor-inference --extra ximage --extra rubin
 GPU_EXTRAS = --extra dev --extra gpu --extra viz
 CORE_EXTRAS = --extra dev --extra photometry
 VIZ_EXTRAS = --extra dev --extra viz --extra photometry
@@ -70,6 +70,9 @@ test-xfit:
 test-xfit-real:
 	$(UV_RUN) $(CORE_EXTRAS) pytest -rs -m real_data tests/xfit/test_real_data.py
 
+test-ximage:
+	$(UV_RUN) $(CPU_EXTRAS) pytest tests/ximage
+
 test-xpois:
 	$(UV_RUN) $(VIZ_EXTRAS) pytest tests/xpois
 

README.md

@@ -12,6 +12,7 @@ structured run artifacts provide integration points for those workflows.
 | --- | --- | --- |
 | FITS files whose pixels you need on a GPU | [xDataReader](docs/components/xdr.md) | Decoded and scaled device arrays |
 | Images with different pixel-to-sky mappings | [xRep](docs/components/xrep.md) | Images resampled onto a common sky grid |
+| Registered frames with known blur and variance | [xImage](docs/components/ximage.md) | A reconstructed common scene and separate PSF calibration |
 | An aligned reference image and a new observation | [xPois](docs/components/xpois.md) | A matched reference and a difference image |
 | Small cutouts containing positive/negative residuals | [xFit](docs/components/xfit.md) | Fitted positions, amplitudes, shapes, uncertainties, and fit status |
 | Candidate cutouts to score, or reviewed examples to train on | [xScan](docs/components/xscan.md) | Real/bogus scores, evaluation metrics, and review material |
@@ -100,6 +101,16 @@ pixel scale, rotation, and position; matching the images' blur is a separate
 operation. Outputs include the reprojected arrays, optional masks, and grid
 metadata.
 
+### xImage
+
+xImage (`cuphoton.ximage`) estimates a common nonnegative scene from
+registered frames, known per-frame PSFs, good-pixel masks, and Gaussian
+noise variances. Each frame uses a spatially fixed PSF. It provides L2 and
+Huber reconstruction, experimental robust-loss extensions, and separate
+PSF calibration with Adam or multiplicative updates. See the
+[xImage guide](docs/components/ximage.md) for input conventions, synthetic
+examples, and a worked Rubin adapter.
+
 ### xPois: subtract a matched reference to reveal changes
 
 xPois (`cuphoton.xpois`) starts with aligned reference and target images. It

THIRD_PARTY_NOTICES.md

@@ -78,6 +78,7 @@ more than one license.
 | `base` | `pandas>=2.2` | `3.0.3` | `BSD-3-Clause` | [pandas](https://github.com/pandas-dev/pandas) | `uv / PyPI` |
 | `photometry` | `photutils>=3.0` | `3.0.0` | `BSD-3-Clause` | [Photutils](https://github.com/astropy/photutils) | `uv / PyPI` |
 | `base` | `pyarrow>=23.0` | `24.0.0` | `Apache-2.0`; binary distributions include Arrow and third-party notices | [Apache Arrow](https://github.com/apache/arrow) | `uv / PyPI` |
+| `base` | `pydantic>=2.11,<3` | `2.14.1` | `MIT` | [Pydantic](https://github.com/pydantic/pydantic) | `uv / PyPI` |
 | `ximage` | `zarr>=3,<4` | `3.4.1` | `MIT` | [Zarr](https://github.com/zarr-developers/zarr-python) | `uv / PyPI` |
 | `rubin` | `lsst-images[https]==30.0.10` | `30.0.10` | `BSD-3-Clause`; optional dependency tree includes `starlink-pyast` under `LGPL-3.0-or-later` | [LSST Images](https://github.com/lsst/images) | `uv / PyPI` |
 | `base` | `PyYAML>=6.0` | `6.0.3` | `MIT` | [PyYAML](https://github.com/yaml/pyyaml) | `uv / PyPI` |

docs/README.md

@@ -38,6 +38,7 @@ requirements.
   Astropy/xDR reader interface, with [runtime compression controls](components/xdr.md#runtime-choices)
   for CLI, Python and manifests; native GDS depends on the storage setup.
 - [xFit](components/xfit.md): batched nonlinear least-squares dipole fitting.
+- [xImage](components/ximage.md): multi-exposure reconstruction and PSF calibration.
 - [xPois](components/xpois.md): kernel fitting and image subtraction.
 - [xScan](components/xscan.md): transient datasets, classification,
   evaluation, and review.

docs/architecture.md

@@ -62,6 +62,12 @@ artifact generation requires a GPU and fits row traces within tiles,
 repeating each result across that tile's columns. Interpret the maps at that
 granularity; CPU paths support individual trace and reference workflows.
 
+xImage can reconstruct a reference scene after registration with xRep. Its
+PSF calibration is a separate operation, and reconstruction requires supplied
+per-exposure kernels. It currently returns host arrays without propagated
+output variance or an effective output PSF; callers must qualify these
+quantities before using a reconstruction as an xPois reference.
+
 ## What an exposure contains
 
 An optical exposure includes an image and the information needed to interpret
@@ -105,7 +111,7 @@ metadata requirements.
 ## Package boundaries
 
 `cuphoton.core` owns shared CLI mechanics, application paths, logging, and
-invariant evaluation. The six science namespaces own their domain models,
+invariant evaluation. The science namespaces own their domain models,
 algorithms, adapters, workflow configuration, and workflows. They share
 Core's public CLI facade.
 

docs/cli.md

@@ -13,7 +13,7 @@ uv run cuphoton xpois --help
 uv run cuphoton xpois help fit-kernel
 ```
 
-Access each component through the fixed command groups: `xdr`, `xfit`, `xpois`,
+Access each component through the fixed command groups: `xdr`, `xfit`, `ximage`, `xpois`,
 `xscan`, `xrep`, `xray`, and `xmonitor`.
 
 `cuphoton xmonitor quickstart --output-dir xmonitor-demo` creates invented
@@ -65,6 +65,13 @@ execution remains the default. `--warmup-rounds` or `--measure-rounds` opts
 into persistent workers and a separate artifact directory for each round.
 See [xFit](components/xfit.md).
 
+## xImage: `cuphoton ximage`
+
+`data-inspect` and `data-validate` check image bundles. `psf-solve-adam` and `psf-solve-mm`
+calibrate kernels from supplied PSF targets; `reconstruct-l2` and
+`reconstruct-rho` estimate a common scene from aligned exposures and kernels.
+See [xImage](components/ximage.md) for configuration and offline FITS adapters.
+
 ## xPois: `cuphoton xpois`
 
 `data-inspect`, `fit-kernel`, `subtract`, `fit-batch`, `benchmark-backends`,

docs/components/ximage.md

@@ -0,0 +1,183 @@
+# xImage
+
+`cuphoton.ximage` reconstructs a common nonnegative scene from registered
+frames using supplied per-frame point-spread functions (PSFs), noise
+variances, and good-pixel masks. It provides weighted least-squares and
+robust-loss reconstruction with PyTorch on CPU or CUDA 13 GPUs.
+
+The reconstruction model assumes:
+
+- Frames are already registered to a common pixel grid. Use
+  [xRep](xrep.md) when resampling is needed before reconstruction.
+- Each frame has one known PSF, fixed across the field. Reconstructing
+  separate tiles can approximate field variation when each tile is supplied
+  an appropriate local PSF; the solver does not fit a spatial PSF field.
+- The reconstructed intensity is nonnegative, as required by the
+  multiplicative update.
+- Noise is Gaussian with known pixel variances. The L2 solver minimizes
+  the corresponding variance-weighted squared error.
+
+xImage implements a PyTorch adaptation of the
+[ImageMM reconstruction approach](https://doi.org/10.3847/1538-3881/adfb72)
+described by Sukurdeep et al. (2025). For publications using xImage
+reconstruction, cite both ImageMM and the cuPhoton version used; see the
+[citation guide](../../CITATION.md#ximage) and
+[scientific references](../references.md#image-restoration-and-super-resolution).
+The robust example uses the paper's Huber loss. The additional `cauchy`,
+`bisquare`, and `damped_rl` losses and the `scalar_per_exposure` background
+model are experimental extensions. Their use requires validation for the
+intended data and measurement.
+
+PSF calibration is a separate operation: Adam or multiplicative updates fit
+per-exposure kernels to supplied PSF stamps using a Gaussian reference.
+Reconstruction holds its supplied kernels fixed. It does not estimate scene
+and PSF jointly from the science images.
+
+## Synthetic example
+
+Install the component dependencies and create a small, deterministic bundle:
+
+```bash
+uv sync --locked --extra ximage
+uv run python examples/ximage/make_synthetic_bundle.py --output data
+uv run cuphoton ximage data-validate --base data
+uv run cuphoton ximage reconstruct-l2 \
+  --config examples/ximage/reconstruct-l2.example.yaml
+uv run cuphoton ximage reconstruct-rho \
+  --config examples/ximage/reconstruct-rho.example.yaml
+```
+
+The example generates three noisy exposures of synthetic point sources.
+Its configurations reconstruct a central 48-by-48 ROI within the 64-by-64
+images and disable masking. With `padding: VALID`, the input must include
+the surrounding pixels required by the PSF support: four pixels beyond each
+ROI edge for the example's native 9-by-9 PSFs. Outputs
+are written under `runs/`; an existing named run is preserved unless
+`output.overwrite` is explicitly enabled. Configuration paths resolve from
+the working directory.
+
+Set `device: cpu` or `device: cuda` to require that device. `device: auto`
+selects CUDA when available and otherwise uses CPU. Float32 and float64 are
+supported; each run records the resolved device, precision, and timings.
+
+## Inputs and reconstruction
+
+Bundles use NPY directories, HDF5, or Zarr stores. Images and variances have
+axes `(batch, y, x, frame)`; PSFs have axes
+`(psf_y, psf_x, channel, frame)`. Each PSF is the image of a point source,
+including when loaded from a PSF calibration run. The current solvers
+require one PSF channel; evaluation supports one image batch. Frames must
+describe the same registered scene.
+
+An optional boolean `masks` array has the same shape as the images, with
+`True` selecting a good pixel. `mask.mode: provided` uses these masks and
+`mask.mode: none` ignores them. Missing masks admit all pixels. Supply finite image values and strictly
+positive finite variances. Optional `exp_times` defaults to
+ones; supplied values divide image intensities and their squares divide
+variances. The auxiliary `sky` and `mcgauss` arrays are optional.
+Convert legacy twelve-plane masks explicitly through
+`cuphoton.ximage.adapters.legacy` before using the generic bundle.
+See [Data and artifact contracts](../data-artifacts.md#ximage-reconstruction-bundles)
+for all arrays, mask conventions, and output fields.
+
+```bash
+cuphoton ximage data-inspect --store data --artifacts images,psfs
+cuphoton ximage psf-solve-adam \
+  --config examples/ximage/psf-adam.example.yaml
+cuphoton ximage psf-solve-mm \
+  --config examples/ximage/psf-mm.example.yaml
+```
+
+PSF calibration supports L1 or L2 objectives with Adam and L2 with
+multiplicative updates. To reconstruct with calibrated kernels, set
+`psf_path` in the reconstruction configuration to the saved `psfs.npy`.
+With `psf_path: null`, reconstruction uses the bundle's PSFs.
+
+Each completed run contains `config.yaml`, `summary.json`, numeric result
+arrays under `artifacts/`, and numeric convergence traces under `trace/`.
+With `VALID` padding, the saved reconstruction includes PSF context around
+the selected ROI; evaluation projects and crops it to the observation ROI.
+
+Run directories and local Rubin manifests retain input file paths, exposure
+identifiers, sky coordinates, and caller-provided metadata needed to trace
+the computation. Benchmark reports also record hardware details. Review
+these generated files before sharing them; they are not anonymized exports.
+
+ROI selection, exposure selection, exposure batching, and optional tiling
+limit the work performed per run. Consult `cuphoton ximage --help` and the
+annotated example configurations for command options.
+
+## Evaluation and timing
+
+```bash
+cuphoton ximage evaluate-reconstruction \
+  --run-dir runs/reconstruct-l2-example
+cuphoton ximage benchmark-devices \
+  --config examples/ximage/reconstruct-l2.example.yaml \
+  --workflow reconstruct-l2 --devices cpu,cuda --warmup 1 --repeats 3
+```
+
+Generic image similarity and background-statistic functions are available
+from `cuphoton.ximage.metrics`, including `psnr`, `ssim`,
+`fourier_sharpness`, and `image_metrics`. Run evaluation also supports the
+Rubin adapter's aperture photometry and catalog comparisons. It compares
+against an equally weighted mean or median coadd of valid,
+exposure-time-normalized pixels. Masked or invalid-variance samples are
+excluded; pixels with no valid contributors are zero. Supply
+`--pipeline-coadd-fits`, `--catalog-parquet`, or `--reference-catalog-csv`
+for external comparisons.
+A coadd and reconstruction made from the same exposures share noise, and
+similarity or sharpness alone does not establish reconstruction accuracy.
+Catalog comparisons also depend on registration, flux calibration, source
+selection, and the reference catalog. Evaluate those assumptions for the
+intended scientific measurement; no automatic scientific acceptance decision
+is produced.
+
+The device benchmark times complete workflow calls, including input loading
+and artifact writing, after the requested warmup runs. It records individual
+timings and hardware metadata. `--evaluate` adds reconstruction metrics;
+elementwise numerical parity requires a separate array comparison.
+
+## Worked adapter: local Rubin products
+
+The `cuphoton.ximage.adapters.rubin` package converts local Rubin products
+to the generic frame, PSF, variance, and good-pixel-mask contract. Its
+`rubin-*` commands inspect caller-supplied FITS and catalog files, build
+manifests and assemble reconstruction bundles. `rubin-build-store` can align
+visit images using their WCS; inspect its mask and interpolation settings
+for the selected products. Inspect the
+per-exposure `psf_source` metadata as well: when no native PSF is available,
+the builder uses a Gaussian derived from exposure summary moments, or a
+unit impulse (`delta_fallback`) if those moments are unavailable. These
+fallback kernels are approximations; supply an appropriate PSF model for
+scientific reconstruction.
+The build result reports `psf_source_counts` and warns about unit-impulse
+placeholders. The generated `sky` plane is zero-filled, not a sky estimate.
+The optional `rubin` dependency extra enables the `lsst-images` readers.
+Its HTTPS support is needed to interpret archive schema identifiers even
+when reading local files.
+Data acquisition and authentication remain the caller's responsibility.
+
+Evaluating coadds or sky-coordinate catalogs against a reference-WCS-aligned
+bundle requires its local reference FITS file and valid WCS headers.
+
+Pipeline coadds carry resampling and combination effects. They can serve as
+comparison images; treating them as independent raw exposures requires an
+appropriate noise and PSF model. Coadd-derived reference catalogs inherit
+the same image noise and are useful for relative checks.
+
+## Python API
+
+```python
+from cuphoton.ximage import load_ximage_bundle, validate_bundle
+
+with load_ximage_bundle("data") as bundle:
+    issues = validate_bundle(bundle)
+    if issues:
+        raise ValueError(issues)
+    images = bundle.images
+```
+
+HDF5 and Zarr arrays remain lazy while the bundle is open. Materialize any
+arrays needed after closing it. Workflow functions `run_psf_solve` and
+`run_reconstruction` are available from `cuphoton.ximage.workflows`.