chore(library): make behavior presets atomic, close a 20-behavior gap, and enforce both in CI

[changliu2]

Summary

Makes every behavior preset atomic, closes a 20-behavior coverage gap in the shipped library, and adds a CI guard so neither regresses.

Found while diffing examples/behavior_specs/ against assert_ai/library/behaviors/ — they are supposed to be one source of truth.

Three problems, none of which anything checked for

1. Presets bundled multiple behaviors

travel_planner covered six mechanisms across "Quality failures" and "Safety failures" — three of which (stereotyping, prompt_injection, sycophancy) already exist as their own atomic presets. travel_planner_benchmark bundled roughly six more. telecom_customer_service wasn't a behavior at all — it's an application spec (Role, Domain Basics, Operational Procedures) wearing kind: behavior.

Evaluating a bundle as one behavior produces a dataset mixing several mechanisms and a metric nobody can act on: you learn that something failed, never which mechanism. That's precisely what best practices §8.D ("use atomic behaviors") exists to prevent.

These three are application scenarios, so they move to a new scenario kind under assert_ai/library/scenarios/. A scenario is the context an eval runs against, not the behavior it measures — pair it with atomic behaviors via context:.

2. Twenty behaviors shipped to nobody

	count
specs in examples/behavior_specs/*.md	38
presets in assert_ai/library/behaviors/*.yaml	18
reachable by a pip install assert-ai user	18

Only the YAML ships in the wheel. Every agentic failure mode — goal drift, premature termination, repeated action loops, stale state, poor retrieval, tool-call error recovery, and 14 more — was invisible to installed users.

The 18 that existed in both places were byte-identical, so this was pure coverage loss, not divergence. The 20 new presets were generated from the existing markdown plus the category metadata already in that directory's README. No prose was invented.

3. Nothing detected either problem

scripts/check_behavior_library.py now fails CI when:

• a preset names another preset's behavior (provable bundling, not a judgement call)
• one preset carries several ## <category> failures sections
• a description reads as an application spec (## Role, ## Domain Basics, ## Operational Procedures)
• a spec markdown drifts from its YAML, or has no preset at all

Wired into Tier 1. It reports zero false positives on the 18 pre-existing good presets.

Note: the drift check compares word streams, not lines. The .md files are unwrapped while the YAML description: blocks hard-wrap at ~65 chars, so a line-based diff reports identical prose as ~5% similar. That bit me while investigating.

Not a breaking change

preset: resolution is wired into config loading (the library README's "not yet implemented" note is stale — tests/test_library_e2e.py exercises behavior: {preset: travel_planner} end to end). So moving those files would have broken real configs.

Instead, resolve_preset("behavior", "travel_planner") still resolves, via a shim that emits a DeprecationWarning pointing at the scenario kind. Config authors get told, not broken.

Also

• scenario added to VALID_KINDS, KIND_TO_SUBDIR, and the library --kind CLI choices
• assert_ai.library.scenarios added to package-data so scenarios actually ship
• READMEs updated in all three directories; examples/behavior_specs/README.md now names the library as the source of truth and says "edit the YAML, mirror it here"

Testing

• pytest tests/test_library_loader.py tests/test_library_e2e.py tests/test_import_smoke.py → 86 passed, incl. new coverage for the scenario kind and the deprecation path
• python scripts/check_behavior_library.py → 38 presets, atomic, in parity
• Full suite: 1146 passed. The 12 test_viewer_* failures are pre-existing on clean main (ERR_MODULE_NOT_FOUND: yaml — needs npm install in viewer/, which CI does); verified by stashing this branch and re-running.
• The 2 remaining ruff F401s in test_library_e2e.py also pre-exist on main; left alone rather than widening this diff.

Review notes

The 20 generated presets are mechanical and near-identical in shape — the metadata mapping in the "Agentic failure modes" README table is the part worth a careful look. Tag/applicable_to assignments were derived from the spec README's existing category headings; happy to re-cut any of them.