Repository navigation
Study configuration, schema-generated NeXus models, and BMD estimation - #106
Merged
Merged
Conversation
Same shape as the Raman path -- xrd2effect / configure_papp_xrd / xrd2ambit --
so a diffractogram reaches NeXus through the AMBIT data model the same way a
spectrum does.
Writes what the NXxrd application definition specifies (it extends NXmonopd):
definition = NXxrd, the angular axis as polar_angle in deg rather than a
free-form "2theta", intensity as the NXdata signal, and the beam described by
incident_wavelength plus a derived incident_energy (E = hc/lambda). That naming
is what makes the result readable by NXxrd-aware tools.
Bruker RAW/UXD vendor headers are recognised by their native key names
(_WL1, _ANODE, _KV, _MA, _GONIOMETER_RADIUS, slits, _STEPSIZE ...) and routed
to NXinstrument sub-groups: source target_material/voltage/current, detector
distance/voltage/gain/slit, collimator slits, monochromator. Explicit arguments
win over header values, and anything the definition does not name falls through
to /parameters/{key}, exactly as configure_papp does for Raman.
peaks2effects turns an indexed peak table into EffectRecords, one per
reflection, carrying hkl, polar_angle and the literature reference as
conditions -- so an assignment table travels with its diffractogram instead of
being flattened into free text.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two defects in the auxiliary_signals branch, both visible in a viewer.
long_name was built as "{primary_endpoint} ({aux_name})", so a DSC group
plotting five transition temperatures showed every curve as
"Tm1 (1st heating) (Tc (cooling))/degC" -- the primary's name wrapping each
auxiliary's. A viewer displays long_name, so that is what a user reads. An
auxiliary is now named for itself.
The unit came from the SIGNAL rather than the auxiliary, for the same reason a
plain ValueArray never reached the branch that reads its unit:
isinstance(item, MetaValueArray or isinstance(item, ValueArray))
collapses to isinstance(item, MetaValueArray), because `MetaValueArray or ...`
short-circuits on the truthy class. A ValueArray therefore fell through to the
ndarray case and silently borrowed the signal's unit, labelling a dimensionless
replicate count "g" and a 0-1 normalised ratio "counts". Fixed to a tuple
isinstance, and long_name no longer falls back to the signal's unit -- unit=None
on a ValueArray is a deliberate statement that the quantity is dimensionless.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… table Generates typed pydantic submodels for NXraman (and its NXoptical_spectroscopy base) directly from the vendored NXDL definitions via a dev-only pynxtools codegen script, so pyambit's Raman parameter vocabulary traces back to the real schema instead of a hand-maintained if/elif chain. NXRamanProtocolApplication carries the typed model and flattens it into ProtocolApplication.parameters for the existing to_nexus writer, which now also instantiates real NeXus group classes (NXbeam, NXdetector, ...) instead of generic NXgroup, and no longer loses an explicit Raman /definition to the AMBIT_DATAMODEL rewrite. Also fixes two pre-existing to_nexus bugs found along the way: a leading-slash path-parsing bug that created a stray empty-named group, and a relative-path NXlink that recursed indefinitely when resolved. configure_papp/spe2ambit preserve spectrastream's exact backward-compatible meta-key contract. A blueprint.py module derives eNanoMapper Template Designer schema rows from the same generated model, so pyambit parameters and blueprint fields can't drift from each other. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Backward-compat meta-key setters now parse "600 g/mm"-style strings into Value(loValue, unit) instead of dropping to plain float(), and report success/failure so an unroutable value (e.g. free-text "fibre") falls back to the generic parameters bucket instead of being silently lost.
NXcalibration and NXprocess were absent from DEFAULT_BASE_CLASS_MERGE_TARGETS, so both generated as near-empty stubs (NXCalibration had only device_path/ calibration_status; NXProcess was literally `pass`). Regenerated with --merge-base-class NXcalibration --merge-base-class NXprocess (the existing CLI escape hatch, no code change) for NXraman. NXCalibration now carries identifier_calibration_reference, calibration_object, original_axis/calibrated_axis, calibration_parameters, fit_formula_description, applied, physical_quantity, description, plus the DATA/PARAMETERS/NOTE dict fields - the vocabulary a calibration-workflow NeXus export needs. Two structural gaps found during verification, relevant to callers: - NXCalibration does NOT inherit NXProcess's program/version/date fields - extends-walking only resolves what generate_tree_from(appdef) itself references; merge-target base classes are not chained into each other. - original_axis/calibrated_axis generate as scalar Optional[Union[float, Value]], not arrays - codegen has no NX_FLOAT[rank] support yet. Real calibration curve arrays, and the free-form contents of calibration_object (NXnote) / calibration_parameters (NXparameters, itself still a near-empty base class even under --full), need to be written via the existing nexus_writer.py/nexusformat.nexus.tree machinery, not these pydantic models, which are appdef-scalar-field-shaped rather than general containers. Not promoted to DEFAULT_BASE_CLASS_MERGE_TARGETS yet - leaving that decision to whoever finishes wiring a full NXcalibration writer, since the array/ container gaps above mean the typed model alone isn't sufficient.
…me, and composition round-trip
- spe2ambit: nx_name fed nexus_writer's entry_id path unsanitized, so a
DOI-shaped provider (e.g. "10.17605/OSF.IO/7CQV4") raised
NeXusError("Invalid path") -- sanitize only the NeXus-facing name.
- SubstanceRecord.to_nexus wrote composition's chemical formula to a
"formula" field NXsample_component's NXDL doesn't define, while never
populating the real field name ("chemical_formula"); cas/einecs/
inchi/inchikey have no NXDL field on NXsample_component either, kept as
free attrs rather than dropped.
- Nexus2Ambit.substance_from_nexus hardcoded composition=None -- it was
never read back at all. Added composition_from_nexus so
SubstanceRecord.composition now round-trips through to_nexus/parse.
Found and fixed while wiring pipeline_nexus's RRUFF reader to populate
real chemistry composition instead of a CHEBI_59999 placeholder.
…type
An EffectArray axis or signal holding text values (e.g. Location:
"Norway"/"Utrecht", or a Marker name) arrives as a numpy object-dtype array
of Python strings. h5py has no native HDF5 equivalent for dtype('O') and
raises on write; the fix, already used for the textValue auxiliary field
below, is to pass an explicit h5py string_dtype whenever the array is
object-dtype.
Surfaced by nanodata-polyrisk's read_lorenzo.py, which needs a categorical
Location/analyte/polymer axis on several EffectArrays.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
investigation_uuid was a bare identifier with nowhere to attach a title or description -- and since one investigation can span many ProtocolApplications, many substances, and many separate writes, that label does not belong on any single one of them (Citation isn't it either: some project data has no publication to cite at all). ProtocolApplication.investigation_uuid now accepts Optional[Union[str, Investigation]]. A bare string works exactly as before. Passing an Investigation(uuid=..., title=..., description=..., image=...) additionally gets nexus_writer to write /investigation/<uuid> ONCE (mirroring the existing substance/<uuid> dedup pattern) and link every entry that shares the uuid to it, so the label isn't duplicated per entry and fills in correctly regardless of which entry is written first. Also fixes a pre-existing bug the new tests surfaced: entry_id can carry a redundant leading "/" whenever there is no hierarchy path in front of it (hierarchy=False, the common case), which NXgroup's path-splitting turns into a mangled entry instead of a normal NXentry. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
A base64-encoded string field shows as raw text ("iVBORw0KG...") to any
generic NeXus/HDF5 viewer -- only code that already knows to decode it would
ever see a picture. NXnote(type="image/png", data=<uint8 bytes>) is the
NeXus-native way to embed a picture, so nexus_writer now decodes the
model's base64 str (kept there for JSON-portability) into that shape.
Also fixes parse_studies, which iterated every nxroot top-level key as a
study entry: "investigation" (like "substance") is a shared group written
once and linked from real entries, not a study itself, and has no
"definition" field -- parse_entry crashed on it.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
NXnote defines a "file_name" field ("Name of original file name if note was
read from an external source"), so an embedded picture can say what it was
called. Optional and cosmetic -- an Investigation without it writes exactly
as before.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
entry.attrs["default"] was set inside "if _default is None", but _default
was never reassigned after the first iteration -- so that branch was true
on EVERY effect, and the attribute silently ended up naming whichever
group was written LAST, not first. For a study whose last-written effects
are categorical (e.g. CALIBRATION indexed by a "standard" name), a generic
NeXus viewer's default plot then fails ("Expected numeric type").
Also completes the chain one level further: NeXus expects entry/@default to
name a group whose OWN @default in turn names a plottable NXdata
(root -> entry -> group -> NXdata) -- entry/@default alone left a viewer to
guess which child of a mixed group (numeric AND categorical NXdata
side-by-side, as RESULT is here) to render. Both levels now prefer the
first numeric-axis NXdata found, only falling back to a categorical one if
no numeric one exists in the whole study.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Repository layout, environment/tooling notes (Poetry, not uv -- this repo has no PEP 621 [project] table so uv silently resolves nothing), and the sharp edges found and fixed while building the Investigation feature and the read_lorenzo.py / read_markus.py readers in nanodata-polyrisk: the entry_id leading-slash bug, the @default chain, object-dtype axis/signal handling, and convert_effectrecords2array's actual (non-pivoting) behavior. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Nexus2Ambit.substance_from_nexus hardcoded substanceType="CHEBI_59999" on read, discarding whatever was actually written to the NXentry attrs. Ambit2Solr checked the literal "E.method_s" key instead of the bare "E.method" key that nexus_writer.to_nexus and every real caller in this codebase actually set, so E.method_s never populated in the Solr index. Both found via a real end-to-end run: a new BLoP reference-library reader (nanodata/pipeline_nexus) producing NeXus, indexed through nanodata/import_pipeline's nexus_index into Solr JSON. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- entry2solr: build one effect document per loop iteration (previously
only the last effect survived, and zero effects raised UnboundLocalError);
skip the conditions child doc when an effect has no real conditions;
attach params once per study instead of once per effect.
- substancerecord2solr: return None when every study was filtered out,
instead of emitting an orphan substance document with no children.
- Add composition2solr: writes substance.composition as type_s:composition
child documents (ChemicalName_s/CASRN_s/InChIKey_s/InChI_s/formula_s),
matching the standard AMBIT composition Solr shape. Composition was
parsed into the NeXus model already but never reached the Solr index.
- spe2ambit: sanitize nx_name's provider value ("/" -> "_") before it
reaches nexus_writer's entry_id path, since NXgroup treats "/" as a
path separator and a DOI-shaped provider crashed with NeXusError.
Matches AMBIT Java's ambit2.base.data.substance.ExternalIdentifier (type/id pairs) so a substance can carry its source library and the item's id within that library. Written to NeXus as two parallel string-array attrs (NeXus/HDF5 attrs can't hold a list of structs directly) and read back the same way; also surfaced as externalIdentifierType_hss/externalIdentifierId_hss in Solr output. Guards against h5py collapsing a length-1 string-array attr back to a bare str on read, which a naive zip() would otherwise iterate character-by-character instead of element-by-element. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…Xcite configure_papp/spe2ambit gain group_investigation=True, so a caller (e.g. RRUFF: one .nxs file per sample) can skip linking into a shared investigation/<uuid> NeXus group while investigation= still feeds Citation.title/assay_uuid. nexus_writer.to_nexus now creates that shared node as NXcite instead of a bare NXgroup, matching how papp.citation is already written and giving title/description a real NeXus home. Also documents in AGENTS.md that ProtocolApplication.uuid has no UUID-format validation anywhere in this codebase -- a prior comment in pipeline_nexus/tasks/read_rruff.py claimed otherwise and hashed a readable identifier because of it; the real constraint is external (Java-side AMBIT/database schema), and a readable identifier belongs on nx_name instead. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Ports the benchmark-dose (BMD) construction from the now-superseded dose_response_embedding prototype into pyambit as bmd.py, generalised to work against arbitrary AMBIT records rather than one project's conventions: the dose axis, exposure time, assay method and control annotation all come from study_config per endpoint category. Adds units.py, pyambit's own eNanoMapper unit-conversion module (Concentration, ConcentrationMolar, ConcentrationSurface, Molar, Percent, Dose), ported from pynanomapper.units rather than imported from it -- pyambit is a foundational dependency for pynanomapper, so importing back would be circular. convert_units is deliberately not ported: it is broken on the current measurement release (drops its measures= argument). Fixes the property-identity key to stop treating every leftover ProtocolApplication parameter as part of what defines a property. other_parameters folded operator name, preparation date, input filename and free-text setup notes into PropertyKey verbatim, so two laboratories measuring the identical property (same material, method, cell type, exposure time) still got different keys the moment their bookkeeping fields differed -- which they always do between independent labs. On real AMBIT data this made every property come out with exactly one provider: consolidate_providers never saw two laboratories together. Cell type now gets its own PropertyKey field (resolved via study_config.cell_field, matching how method is already resolved); everything else from parameters is dropped from the key, matching the identity notebooks-ambit's curate_passing.py/curate_failing.py already use for the same records (material, category, method, cell type -- nothing else). Verified against notebooks-ambit's own grouping (see companion commit): under that identity, 98/530 real study groups already have >=2 laboratories, one with 8 labs across only 2 substance records -- all of which the old key would have split into single-provider properties. test_bookkeeping_parameters_do_not_split_the_property pins the fix. test_reading_a_record_does_not_mutate_it and test_a_record_still_writes_to_nexus_after_being_read_for_bmd confirm bmd.py is read-only: a per-well REPLICATE index stays a real array axis for NeXus; bmd.py only reads along the dose axis to gather wells into a bootstrap sample, never writes back to the model. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…orpus
- nexus_parser.parse_entry: read nxentry["reference"]["year"] unconditionally.
Citation.year is Optional and to_nexus writes nothing for a None year, so
any file with no publication year (every MOMENTUM Template Designer
workbook -- unpublished lab records, not a paper) raised
"NeXusError: Invalid path" on every read, making the whole corpus
unparseable. Now reads year/title/owner defensively, defaulting to
None/"" when the reference group or a field is absent.
- nexus_writer.to_nexus (ProtocolApplication and SubstanceRecord): built the
substance NeXus group name directly from the substance uuid/i5uuid. "/" is
the NeXus/HDF5 path separator, so a material name that legitimately
contains one (e.g. "PP/Talc leachate", seen on real MOMENTUM ELISA
workbooks) broke nx_root[f"substance/{uuid}"] = ... with the same
"Invalid path" error. New _nx_group_name() sanitizes only the group name;
the unsanitized value is kept verbatim in the group's own uuid/publicname
attrs, so nothing is lost.
Both covered by tests/pyambit/datamodel/nexus_parser_test.py, built on a
synthetic ProtocolApplication/SubstanceRecord round-tripped through
to_nexus -> a temp file -> Nexus2Ambit, no external fixture.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- process_pa's entry/@default and group-level @default selection only compared "is numeric", so an axis-less RAW_DATA scalar series (e.g. qpcr's "individual PCR efficiency", one value per row, no condition) could win the default race over an AGGREGATED array with a real declared axis (e.g. "normalization factor (genorm)" over concentration) purely by write order. Every such file's default plot showed the wrong thing (efficiency vs row index instead of the actual dose-response curve). Added _has_declared_axes/_default_quality and prefer axis-bearing numeric data over axis-less numeric data at both levels; removed the now-unused _has_numeric_axes. - interpretation was stamped "image" for any signal of rank >= 2, but NeXus only defines it for rank 0/1/2 (scalar/spectrum/image). A real rank-4 signal (wp5's "Concentration bacteria", grid-built over 4 conditions via share_conditions) got mislabeled "image", which made h5web try to render it as 2D and crash with "Expected numeric, boolean, enum or complex type". Now left unset above rank 2, so a generic viewer falls back to its own N-D array selector. - Aux-signal writing already used dtype-based (not name-based) string encoding for object-dtype arrays -- confirmed with the existing test suite, no change needed here. Two regression tests added covering both fixes directly.
ProtocolApplication.parameters's clean_parameters validator flattened every "/" in a parameter key to "_" unconditionally -- stale sanitize logic predating nexus_writer.to_nexus's parameter loop, which deliberately treats "/" as a path separator (splits it to build nested NeXus groups, falling back to param_lookup()'s heuristic only for a single-segment key). The flattening defeated that: a key like "CULTURE CONDITIONS/Exposure method" landed as one literal underscore-joined key both in the written NeXus file and in ProtocolApplication's own JSON dump, instead of a real nested "CULTURE CONDITIONS" group. Also adds a "calibration" -> NXcalibration() case to to_nexus's group-class dispatch, alongside its existing instrument/environment/ parameters cases, for pynanomapper's blueprint-group-name mapping (CALIBRATION -> calibration). Regression test added.
A material id like "PP/Talc leachate" flows into papp.nx_name and then into entry_id, where NXgroup.__setitem__ treats "/" (and "\") as a path separator -- nx_root[entry_id] then looks for a non-existent parent group "PP" and raises NeXusError: Invalid path, aborting the whole file. Replace "/" and "\" with "_" in the nx_name segment of entry_id only (same treatment `provider` already gets a few lines below). The readable id is untouched on the substance name/publicname and papp.owner, so nothing that needs the original loses it. nx_name is bound before the try so the except-branch fallback path uses the sanitized value too. Found on the MOMENTUM ldh corpus (UM_ldhsubmerged_IG_2's "PP/Talc leachate" material). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The first cut drew every resolved plot as a connected line with a marker
per point, which produced three wrong figures on the MOMENTUM corpus:
- **A replicate/experiment axis was used as x.** PCR efficiency plotted
against "Experiment 1..6", joined by a line -- run number is nominal, so
the line drew a trend that does not exist. Now a distinct "replicates"
form: points about a mean ± SD band, no connecting line.
- **A mixed dose axis crashed.** MOMENTUM cytokine lists its controls on
the concentration axis ("0.003, 1-propanol, 0.017, 0.1, IC"), so
_looks_log_scale did float(b' 1-propanol') and raised. The axis is now
split: the numeric doses are the curve, the named controls become
labelled reference lines on the response scale, where they belong.
- **Markers swamped dense data.** An 863-point IR spectrum drew 863 dots
on the line they sit on. Markers now only below ~40 points, sized by
count; a bare positional "index" axis (unrelated samples) draws points
with no line at all, since the order is not a sequence the data claims.
Also: wavenumber axes run high->low per convention; `labeled` no longer
appends a unit the endpoint name already carries ("Wavenumber (cm-1)
(cm-1)"); the legend sits below the axes instead of loc="best", which
collided with the right-margin reference labels.
Chrome follows one validated palette (blue/orange/aqua -- worst adjacent
CVD ΔE 9.2, normal-vision 27.6 on the light surface): 2px lines, markers
ringed in the surface color, hairline solid horizontal-only grid, muted
axis ink, individual runs in a de-emphasis gray so the mean reads as the
subject.
resolve_plot now returns a dict payload rather than a positional tuple --
callers read fields by name, and the three forms no longer share one
tuple shape. Every existing consumer passes (kind, payload) straight to
draw(), so none needed changing.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
figure_png() renders one entry's default plot to PNG bytes; embed_investigation_image() stores it at investigation/<uuid>/image as an NXnote with type="image/png" and the bytes as a uint8 array -- exactly the shape nexus_writer writes for Investigation.image, so H5Web/HDFView render it directly (a base64 string field would show as "iVBORw0KG..." text). Deliberately a second pass over the written file rather than a step before to_nexus: the figure is rendered FROM the .nxs, so what gets embedded is what the file actually contains, not what the converter believed it was writing. A file with nothing plottable simply gets no image, and an existing image is left alone unless overwrite=True. The description was already being written (nexus_writer stores Investigation.description as an attr on the NXcite group); this completes the pair, so the picture and the prose both travel with the data. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Reverts the automatic embedding of a rendered PNG into every .nxs at investigation/<uuid>/image. That was wrong: it puts ~40KB of DERIVED data per entry into archival measurement files, it goes stale the moment the plotting code changes with nothing to detect it, and it is paid for on every HSDS upload. figure_png() stays -- a viewer or thumbnail service calls it to render on demand and gets the same picture as PNG bytes, with nothing written back. embed_investigation_image() and its two tests are removed; a figure_png-renders-on-demand test replaces them and asserts the file size does not change. Investigation.image and the base64->NXnote writer path stay -- those are pre-existing and remain the supported way for a converter to pass a deliberately-chosen image through the model. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
… the model
nexus_writer stored Investigation.description as an ATTRIBUTE on the
investigation NXcite group while title was a field. NXcite's NXDL defines
description as a field (like title), and a NeXus viewer lists fields in the
tree but only shows attributes once a node is expanded -- so the authored
prose was effectively invisible. And Nexus2Ambit skipped the whole
investigation group (nexus_parser.py's ("substance","investigation") skip
tuple), reading investigation_uuid back as a bare string, so the prose was
written to every .nxs and never read by anything.
- nexus_writer: write description as `group.description` (a field). Both
spellings are still accepted on read, so files written by the earlier
code resolve.
- nexus_parser: `parse` now reads the investigation group
(investigation_from_nexus / parse_investigations), collecting each
uuid's title+description; parse_entry hands the papp the labelled
Investigation object (ProtocolApplication.investigation_uuid is already
Union[str, Investigation]) instead of a bare uuid.
- solr_writer: unwrap the Investigation (entry2solr previously had no
unwrap and would have serialised the object), keep investigation_uuid_s
as a plain string, and index investigation_title_s /
investigation_description_s -- that prose is the most searchable text in
a study record.
A round-trip test (write -> Nexus2Ambit -> Ambit2Solr) covers it, plus a
real MOMENTUM file confirms the attr-form fallback works for
already-converted corpora.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
entry_metadata only read investigation `description` from an attribute, but nexus_writer (8865f0d) writes it as a FIELD on investigation/<uuid> -- so it came back empty for every file written since. Read field-or-attr (as `title` already was), and when an NXentry has no `investigation` link, resolve the shared investigation/<uuid> group by collection_identifier. Add iter_investigations(), summarize_investigations() (fold entry metadata into one record per investigation) and substance_names() (every material a file carries). Regression tests. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Six strands of work, most of it driven by running pyambit against real corpora and fixing what broke.
Study configuration — study_config.json / study_config.py, the same configuration jToxKit uses. Per endpoint category it names the dose axis, exposure time, assay method, control annotation and cell field, so downstream code stops hardcoding one project's conventions.
Schema-generated NeXus models — nexus_models/ is now generated from the vendored NXDL definitions by a dev-only pynxtools codegen script (dev-tools/gen_nexus_models.py), replacing a hand-maintained if/elif parameter table. Raman's vocabulary traces back to the real schema; blueprint.py derives eNanoMapper Template Designer rows from the same model so the two can't drift.
BMD estimation — bmd.py, ported from the superseded dose_response_embedding prototype and generalised to arbitrary AMBIT records via study_config. units.py carries eNanoMapper unit conversion, ported rather than imported from pynanomapper, which depends on pyambit and would be circular. Includes a significant fix to the property-identity key: bookkeeping fields (operator, prep date, input filename, setup notes) were folded into PropertyKey, so two labs measuring the identical property never grouped together
Investigation — a shared, human-readable label for investigation_uuid, written once as NXcite and linked from every entry that shares it, with the image stored as a real NXnote rather than a base64 string field.
Plotting and XRD — nexus_plot.py renders summary figures on demand, choosing the chart form from what the data actually is (replicate axes as points about a mean, mixed dose axes split into curve plus labelled controls, markers suppressed on dense spectra). nexus_xrd.py mirrors nexus_spectra for diffractograms, writing what NXxrd specifies.
Round-trip and data-loss fixes — a long tail found on real files: nx_name/provider sanitization where a material name or DOI contains /; "/" preserved as a genuine NeXus path separator in parameter keys; the @default plot chain corrected at both levels; object-dtype categorical axes written with an explicit string dtype; composition, substanceType, E.method_s and ExternalIdentifier round-tripping; and several Solr child-document bugs where only the last effect survived, or an orphan substance document was emitted.
AGENTS.md records the repository layout and the sharp edges found along the way.