Add EMAPA as an anatomy source

[gaurav]

Summary

Adds EMAPA (Mouse Developmental Anatomy Ontology) as an anatomy identifier source (#733), contributing 8,078 identifiers to the anatomy compendia. Source documentation, the generated impact report and a full build-vs-build clique diff are under docs/sources/EMAPA/.

Two things for an SME to weigh in on are at the bottom; everything above them is evidence for those two decisions.

What this does to existing cliques: nothing

docs/sources/EMAPA/clique-diff.md records a full babel-clique-diff between anatomy built without and with EMAPA, from identical cached intermediates on the same commit. It closes the gap the impact report cannot see (#895): before-cliques that split, lose members, or disappear.

No pre-existing clique changes in any way. Zero change rows: nothing splits, nothing is retyped, nothing loses a member, and no identifier is dropped from any of the four anatomy compendia. Every difference is a brand-new EMAPA-only clique.

compendium	before	after	diff
AnatomicalEntity.txt	145,743	147,565	+1,822
Cell.txt	9,197	9,197	0
CellularComponent.txt	9,469	9,469	0
GrossAnatomicalStructure.txt	10,708	12,639	+1,931

Totals go 175,117 → 178,870; the +3,753 is exactly the pure-new clique count the impact report computes independently.

How EMAPA is represented in Babel

Every EMAPA term is typed at extraction: terms at or below EMAPA:35949 "organ" or EMAPA:35868 "tissue" as biolink:GrossAnatomicalStructure, everything else as biolink:AnatomicalEntity.

That declared type is only a starting point — the clique's type wins, and EMAPA is placed last in the type-precedence order, behind GO, CL and UBERON. So EMAPA can add typing to a clique that had none, but can never override an existing ontology's judgement:

	declared by EMAPA	final, in the compendia
biolink:AnatomicalEntity	4,090	2,787
biolink:GrossAnatomicalStructure	3,988	5,291

The totals match at 8,078, so the ~1,300-term shift is retyping — an EMAPA term called an anatomical entity landing in GrossAnatomicalStructure.txt because a UBERON member of its clique is typed that way — not loss. No EMAPA identifier is dropped.

How EMAPA joins to the rest of anatomy. EMAPA asserts no outgoing mapping triples at all in UberGraph — no hasDbXref, no skos:exactMatch/closeMatch/broadMatch/narrowMatch. Every EMAPA↔UBERON equivalence comes from the other direction: UBERON asserts 4,356 hasDbXref triples pointing at EMAPA. So the identifiers EMAPA contributes are the substance of this ingest, and the mappings that place them into cliques are UBERON's.

Fixes two cliques that merged a gross structure with a cell type

Found by scanning for identifiers in a compendium they had no business being in — an unexpected Biolink type is the cheapest signal that a merge went wrong. Both conflations predate this PR and ship on main today.

• CL:0000166 "chromaffin cell" shared a biolink:Cell clique with UBERON:0001236 "adrenal medulla" — the cell type merged with the structure it sits in. Cause: UBERON:0001236 hasDbXref MESH:D019439, and MESH:D019439 is "Chromaffin Cells".
• UBERON:0003124 "chorion membrane" sat in a biolink:CellularComponent clique led by GO:0042600 "egg chorion" — a mammalian extraembryonic membrane merged with the acellular envelope of an insect egg. Cause: UMLS:C0008503 eq GO:0042600 in the UMLS concord.

Neither is fixable at the prefix level (MESH and GO are legitimate anatomy xref targets), so both pairs are now listed individually in input_data/anatomy_badxrefs.txt.

This also rescues two EMAPA terms the branch was silently losing. EMAPA:18428 "adrenal medulla" and EMAPA:16112 "chorion" were dragged into those mistyped cliques and then dropped entirely, because EMAPA is not a permitted prefix for biolink:Cell or biolink:CellularComponent. A clique diff cannot see this class of loss — a dropped identifier appears on neither side.

SME sign-off required: two judgement calls

1. EMAPA identifiers are not forced to be unique within a clique

122 UBERON terms cross-reference more than one EMAPA term (263 EMAPA CURIEs); for 116 of them, more than one target is a live, labelled EMAPA term. Full list, one row per mapping, in docs/sources/EMAPA/multi-emapa-xrefs.csv.

Reading that list, these are overwhelmingly genuine 1:n mappings — a mouse developmental partonomy naming a structure at finer or stage-specific granularity than the adult UBERON term:

UBERON term	EMAPA terms it xrefs
UBERON:0000044 "dorsal root ganglion"	EMAPA:16667 "spinal ganglion", EMAPA:16668 "dorsal root ganglion", EMAPA:18372 "tail dorsal root ganglion"
UBERON:0001017 "central nervous system"	EMAPA:16470 "central nervous system", EMAPA:16754 "tail central nervous system"
UBERON:0001049 "neural tube"	EMAPA:16164 "neural tube", EMAPA:16757 "tail neural tube"

Restricting EMAPA to one identifier per clique would block all 122 merges, and would have silently withdrawn three already-published identifiers from the compendia.

• Confirm that collapsing e.g. "spinal ganglion", "dorsal root ganglion" and "tail dorsal root ganglion" into one clique with UBERON:0000044 is acceptable. Restricting EMAPA does not produce a better clique — it produces an arbitrary winner plus singletons, and can silently delete identifiers. If these mappings should not be equivalences at all, the fix is to filter the offending UBERON xrefs, not to restrict the prefix.

2. EMAPA terms carry no taxon

We considered attaching NCBITaxon:10090 "Mus musculus" to every EMAPA term, since EMAPA is by definition a mouse ontology, and decided against it. The tempting precedent is MP, where the taxon cleanly separates MP (mammal) from HP (human). EMAPA is not analogous: of the 8,078 EMAPA CURIEs in the built compendia, 4,345 share a clique with a UBERON term, and UBERON is a multi-species ontology. Marking those cliques mouse-specific would be incorrect — UBERON:0001049 "neural tube" is not a mouse structure just because EMAPA:16164 is one of its cross-references. A taxon attaches to the clique, not to the EMAPA identifier within it, so there is no way to say "mouse, but only for the EMAPA member".

• Confirm that no taxon is preferable to a mouse taxon here.

Both decisions are recorded in docs/sources/EMAPA/README.md.

Implementation notes

For code review; none of this changes the answers above.

• EMAPA is a part_of partonomy rather than an is_a hierarchy, so build_sets() and get_subclasses_of() now take a hierarchy_predicate (defaulting to rdfs:subClassOf). A subClassOf walk from EMAPA:0 reaches only two terms.
• build_sets() now sorts its output. It previously iterated a set of strings, so identical code and data produced different concord row orders — and therefore different cliques — between runs (unique_prefixes tie-break picks the lexicographically smallest CURIE, not the valid one #945). EMAPA ended up unrestricted so this decides no EMAPA clique, but UBERON and GO are restricted and the guarantee makes any such tie-break reproducible.
• input_data/anatomy_badxrefs.txt is applied through the concord_pair_filter hook glom_from_files() already exposes, so it needed no new plumbing.

Test plan

• ruff check / ruff format / snakefmt / rumdl — all clean
• uv run pytest -m unit -q — 421 passed
• uv run pytest tests/pipeline -q --all — 56 passed, 3 skipped, 2 xfailed. Both xfails are pre-existing KNOWN_DUPLICATES entries (MESH → Pipeline tests reveal MeSH duplication between different compendia #730, UMLS → Pipeline tests reveal UMLS duplication between different compendia #729); EMAPA passes.
• Impact report regenerated from a local anatomy build, after deleting the previous report entirely so every file provably comes from the new build
• Clique diff regenerated from a deterministic two-sided build — 0 change rows
• SME review of the two decisions above

🤖 Generated with Claude Code

[gaurav]

Heads-up after rebasing onto the updated add-emapa (#742): the shared clique-building skeleton was refactored out of anatomy into a new module src/createcompendia/cliques.py (commit 154f3788).

anatomy.compute_cliques_for_impact_report is now a thin wrapper over cliques.glom_from_files(...), which runs the common load ids → glom, load concords → filter → drop overused xrefs → glom loop parameterized by three hooks:

• concord_pair_filter(parts, infile, dicts) — per-pair keep/drop, with access to the clique state built so far
• overused_xref_remover(pairs, infile) — per-file remove_overused_xrefs variant
• glom_kwargs — extra kwargs forwarded to every glom call (e.g. disease's close={MONDO: ...})

build_compendia still calls the wrapper, so the impact report's reglom stays in lockstep with the real build (no drift).

This AddingNewSources.md was written before that refactor, so two sections describe the compute hook as anatomy-only logic and will be slightly stale once rebased:

• §3 "Add a compendium-building hook" — should describe a new type's compute_cliques_for_impact_report as a thin wrapper supplying its hooks to cliques.glom_from_files, rather than reimplementing the glom loop.
• §Future "Roll out the compute hook to other semantic types" — can note that the skeleton already exists in cliques.py, so rolling out to disease/taxon/process is now adding a thin wrapper + a SEMANTIC_TYPE_CONFIG entry (and, ideally, routing each type's real build_compendia through the same wrapper, or adding a sync test).

The CLAUDE.md update in #742 (commit 25174094, "extending the report to a new semantic type" section) has ready-to-lift wording for both.

[cbizon]

Can this be merged?

[SkyeAv]

Summary

Good overall PR. Love some ideas like the new impact reports and bad xrefs. I think before we merge it's worth looking into some of the architectural patterns and standards this sets because I'm not sure all of them yet scale to all sources (i.e., the new-cliques/xref files).

• see comments for specifics

Recommendation

Merge with minor revisions.