feat(observability): /sprint --health per-sprint outcome aggregator - #26
Merged
Merged
Conversation
Adds scripts/lib/sprint-health.py, a read-only deep-dive on a single
sprint's outcomes — complements the existing /sprint --status (progress)
and sprint-fleet-status.py (cross-sprint summary) by answering the
question "how is this sprint going?" from the data that has been
accumulating in jsonl logs since v1.2+.
Aggregates five sources:
- contracts/group-*.md (group count)
- evaluations/group-*.md and *.adversarial.md (standard + adversarial
verdicts per group)
- logs/evaluator.jsonl (evaluation rounds, used as a fix-loop proxy)
- project-scoped logs/guards.jsonl filtered by sprint_id (drift /
self_deception / context / regression buckets)
- .zachflow/kb/learning/patterns/*.yaml filtered by source_sprint,
plus project-scoped logs/curator.jsonl for transitions on those
Outputs markdown by default (suitable for posting verbatim into a Retro
note or PR body) or JSON when --format json is passed (for piping into
external trend tracking; no snapshots are persisted by default).
Workflow integration: phase-modes.md gains "## --health Mode" mirroring
the existing --status / --continue / --follow-up shape. /sprint
{sprint-id} --health is the user-facing invocation; SKILL.md's CLI
reference table is updated.
Unlocks the next-tier work (cross-sprint trend lines, alerts on guard
signal-to-noise ratios) by giving every later analyzer a single
well-typed snapshot to consume.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
scripts/lib/sprint-health.py— read-only deep-dive on a single sprint's outcomes (verdicts, fix-loop counts, failure-mode guard activity, contributed patterns + curator transitions)/sprint <sprint-id> --healthmirrors the existing/sprint <sprint-id> --statusshape —--statusshows progress,--healthshows outcomessprint-fleet-status.py(cross-sprint summary). With this PR, the three observability surfaces line up:/sprint <id> --status/sprint <id> --health(new)/sprint --status(fleet)Data sources
The aggregator reads five sources, all read-only:
runs/sprint/<id>/contracts/group-*.md— group countruns/sprint/<id>/evaluations/group-*.mdandgroup-*.adversarial.md— standard + adversarial verdictsruns/sprint/<id>/logs/evaluator.jsonl— eval rounds per group (proxy for fix-loop count)logs/guards.jsonlfiltered bysprint_id— drift / self_deception / context / regression buckets.zachflow/kb/learning/patterns/*.yamlfiltered bysource_sprint+ project-scopedlogs/curator.jsonlfor transitions on those patternsOutput formats: markdown (default, post-to-PR friendly) or JSON (for piping into external trend tracking).
Files
scripts/lib/sprint-health.py(315 LOC)tests/sprint-health-test.sh(6 cases — empty / mixed verdicts / eval rounds / guards filter / patterns filter / JSON parse)workflows/sprint/SKILL.md(CLI reference table +--health)workflows/sprint/phase-modes.md(new "## --health Mode" section)docs/roadmap.md(v1.x entry marked shipped)Test plan
sprint-health-test.shpasses (6 cases)Design notes
sprint_idis the filter key everywhere — guards.jsonl events tag their sprint; curator.jsonl events don't (curator is kb-scoped) so we filter via the pattern'ssource_sprintinstead. Both filters trivially exclude other sprints' dataevaluator.jsonlrounds (=rounds - 1). If the workflow caller adds a dedicatedfix_loop_startedevent later, the script can switch sources without changing the API🤖 Generated with Claude Code