Skip to content

dispatch: record per-worker token usage - #194

Merged
vu1n merged 1 commit into
mainfrom
feat/dispatch-worker-usage
Oct 13, 2026
Merged

vu1n merged 1 commit into
mainfrom
feat/dispatch-worker-usage

Conversation

@vu1n

@vu1n vu1n commented Oct 13, 2026 •

Copy link
Copy Markdown
Owner

Dispatch verdicts lack worker token accounting; recorded per-worker usage lets fork-k and cross-model H5 validation relate reward to cumulative token budgets.

Each worker row and dispatch.worker_summary body now has usage: null or four nullable counters: input_tokens, output_tokens, cache_read_input_tokens, and cache_creation_input_tokens. Dispatch snapshots each worker's normalized session events once after selection and pull, including failed, errored, and unverified workers. The summary artifact copies that snapshot.

The fold shares cost accounting's message deduplication and native-over-wire preference. A counter is null when any selected observation omits it or addition overflows; reported zero remains zero. Raw cumulative snapshots and legacy cost envelopes are excluded. Grades, winner selection, retry policy, critic accounting, and legacy cost JSON stay unchanged.

Recorded events cannot certify complete provider billing: producer lag, unreported calls, skipped malformed log records, and later spend by live workers remain possible. Documentation explains partial/full-budget checkpoints, all-worker totals, unknown coordinates, and avoiding repeated cumulative snapshots. No benchmark, paid experiment, merge, or deployment is included.

Before / after contract example (JSON output is tested; not visually verified):

 { "session": "w0", "score": 1.0, "passed": true,
-  "retries_used": 1, "status": "scored" }
+  "retries_used": 1, "status": "scored",
+  "usage": { "input_tokens": 30, "output_tokens": 0,
+             "cache_read_input_tokens": 0,
+             "cache_creation_input_tokens": null } }

Verification at 0a0044c4d6d24ec92dfc05b00ecd843deb5c7f20:

  • cargo test --locked --no-default-features --bin pillbox usage: 48 passed.
  • cargo test --locked --no-default-features --all-targets: 981 passed, 6 ignored across unit and integration targets; no failures.
  • cargo fmt --check, cargo clippy --locked --no-default-features --all-targets -- -D warnings, and git diff --check: passed.
  • Brief v0.4.2: check passed; pin changed 0 refs; doctor found 0 warnings and one pre-existing advisory about the draft workspace-cow-fork decision.
  • Independent structural review: no findings. Fresh-context adversarial correctness review: no findings, risk low. Separate static vuln-triage: 0 candidates, 0 confirmed, 0 unresolved.
  • Exact-head CI: all 5 checks succeeded (CI, Brief). PR head, remote branch, and reviewed SHA match. The merge-readiness helper intentionally refuses draft PRs; checks were verified directly on this head with a stable check set. This PR stays draft; no merge was attempted.
Mutant Target test Result Severity
Missing field becomes zero observed_usage_nulls_incomplete_counters_without_losing_other_fields killed error
Absent events become zero observed_usage_distinguishes_missing_from_reported_zero killed error
Wire overrides native observed_usage_reuses_cost_identity_and_source_preference killed error
Overflow saturates observed_usage_overflow_is_unknown_per_counter killed error
Retries multiply usage usage_is_snapshotted_once_after_all_retries_and_segments killed error
Errored/unverified usage omitted failed_errored_and_unverified_workers_retain_usage killed error
Read failure changes status usage_read_failure_leaves_selection_and_retry_behavior_unchanged killed error
Artifact drops snapshot record_worker_summaries_writes_an_artifact_to_each_worker_log killed error
Verdict omits nullable key verdict_json_is_the_documented_envelope killed error

All mutants ran only in a scratch worktree; no mutant changes remain.

@vu1n
vu1n marked this pull request as ready for review October 13, 2026 02:59
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 13, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-13T03:02:40.335854Z 0a0044c Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@vu1n
vu1n merged commit a5bebc3 into main Oct 13, 2026
5 checks passed
@vu1n
vu1n deleted the feat/dispatch-worker-usage branch October 13, 2026 03:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant