Skip to content

feat(observability): /sprint --health per-sprint outcome aggregator - #26

Merged
hx2ryu merged 1 commit into
mainfrom
feat/v15-sprint-health
May 15, 2026
Merged

hx2ryu merged 1 commit into
mainfrom
feat/v15-sprint-health

Conversation

@hx2ryu

@hx2ryu hx2ryu commented May 15, 2026

Copy link
Copy Markdown
Owner

Summary

  • Adds scripts/lib/sprint-health.py — read-only deep-dive on a single sprint's outcomes (verdicts, fix-loop counts, failure-mode guard activity, contributed patterns + curator transitions)
  • Workflow integration: /sprint <sprint-id> --health mirrors the existing /sprint <sprint-id> --status shape — --status shows progress, --health shows outcomes
  • Companion to sprint-fleet-status.py (cross-sprint summary). With this PR, the three observability surfaces line up:
Surface Question Tool
Per-sprint progress Where is the sprint right now? /sprint <id> --status
Per-sprint outcomes How is the sprint going? /sprint <id> --health (new)
Cross-sprint summary What's running across my sprints? /sprint --status (fleet)

Data sources

The aggregator reads five sources, all read-only:

  1. runs/sprint/<id>/contracts/group-*.md — group count
  2. runs/sprint/<id>/evaluations/group-*.md and group-*.adversarial.md — standard + adversarial verdicts
  3. runs/sprint/<id>/logs/evaluator.jsonl — eval rounds per group (proxy for fix-loop count)
  4. Project-scoped logs/guards.jsonl filtered by sprint_id — drift / self_deception / context / regression buckets
  5. .zachflow/kb/learning/patterns/*.yaml filtered by source_sprint + project-scoped logs/curator.jsonl for transitions on those patterns

Output formats: markdown (default, post-to-PR friendly) or JSON (for piping into external trend tracking).

Files

Type Path
New scripts/lib/sprint-health.py (315 LOC)
New tests/sprint-health-test.sh (6 cases — empty / mixed verdicts / eval rounds / guards filter / patterns filter / JSON parse)
Mod workflows/sprint/SKILL.md (CLI reference table + --health)
Mod workflows/sprint/phase-modes.md (new "## --health Mode" section)
Mod docs/roadmap.md (v1.x entry marked shipped)

Test plan

  • New sprint-health-test.sh passes (6 cases)
  • All existing tests still pass (11 suites: 4 guards + curator + migrate + schema + kb-smoke + jsonl-chain + bump-rubric + fleet-status)
  • 3-OS CI matrix green (cygpath compat already in place for jsonl/yaml reads)
  • First real sprint run that hits multiple groups — eyeball that the rendered markdown matches the actual run state

Design notes

  • No snapshots persisted by default — the script computes the snapshot on demand. Trend lines (last N snapshots, alerts) deliberately deferred so this PR stays tight; the JSON output is the contract for any future trend-tracking layer
  • sprint_id is the filter key everywhere — guards.jsonl events tag their sprint; curator.jsonl events don't (curator is kb-scoped) so we filter via the pattern's source_sprint instead. Both filters trivially exclude other sprints' data
  • Fix-loop count is a proxy, not a count — derived from evaluator.jsonl rounds (= rounds - 1). If the workflow caller adds a dedicated fix_loop_started event later, the script can switch sources without changing the API

🤖 Generated with Claude Code

Adds scripts/lib/sprint-health.py, a read-only deep-dive on a single
sprint's outcomes — complements the existing /sprint --status (progress)
and sprint-fleet-status.py (cross-sprint summary) by answering the
question "how is this sprint going?" from the data that has been
accumulating in jsonl logs since v1.2+.

Aggregates five sources:
  - contracts/group-*.md (group count)
  - evaluations/group-*.md and *.adversarial.md (standard + adversarial
    verdicts per group)
  - logs/evaluator.jsonl (evaluation rounds, used as a fix-loop proxy)
  - project-scoped logs/guards.jsonl filtered by sprint_id (drift /
    self_deception / context / regression buckets)
  - .zachflow/kb/learning/patterns/*.yaml filtered by source_sprint,
    plus project-scoped logs/curator.jsonl for transitions on those

Outputs markdown by default (suitable for posting verbatim into a Retro
note or PR body) or JSON when --format json is passed (for piping into
external trend tracking; no snapshots are persisted by default).

Workflow integration: phase-modes.md gains "## --health Mode" mirroring
the existing --status / --continue / --follow-up shape. /sprint
{sprint-id} --health is the user-facing invocation; SKILL.md's CLI
reference table is updated.

Unlocks the next-tier work (cross-sprint trend lines, alerts on guard
signal-to-noise ratios) by giving every later analyzer a single
well-typed snapshot to consume.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@hx2ryu
hx2ryu merged commit 7b19c70 into main May 15, 2026
3 checks passed
@hx2ryu
hx2ryu deleted the feat/v15-sprint-health branch May 15, 2026 05:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant