Skip to content

Let submissions create supply system_power.json, and warn when runs lack it - #147

Merged
arav-agarwal2 merged 1 commit into
mainfrom
fix/system-power-on-create
Oct 29, 2026
Merged

arav-agarwal2 merged 1 commit into
mainfrom
fix/system-power-on-create

Conversation

@arav-agarwal2

Copy link
Copy Markdown
Contributor

Problem

submissions create can fail with Missing system_power.json for results/<system> even though check-submission on the submitter's local tree passes.

submissions create builds the bundle from the uploaded run archives only. The builder takes system_power.json from beside point.yaml in each archive. A descriptor kept only in a local results/<system>/ folder, where a finished submission holds it, never reaches the bundle. Nothing along the way flags this:

  • runs create treats the file as optional.
  • The builder skips it silently.
  • The checker's message describes the finished tree, not where the build reads from.

Changes

  • submissions create --system-power PATH: supplies the descriptor for runs registered without one, so runs don't have to be re-registered. It overrides the runs' copies and is refused when the runs span more than one system. It is also available through the MCP create_submission tool.
  • Hint on checker failure: when a system lacks system_power.json, create explains that the build reads only from the run archives and names both fixes.
  • runs create warning: printed on stderr, so --dry-run JSON stays clean, when the run folder has no system_power.json. When a copy sits one level up, the warning points to it. This can't be an error, because the division isn't known until submissions create.
  • Docs: usage/submissions.md, usage/runs.md.

Testing

  • Full suite passes (1798). New tests cover the builder override (fills, overrides, refused across systems, rejects non-object), the create hint, the runs warnings, and MCP option coverage.
  • Reproduced the reported failure from a real affected submission's run archives. Without the flag, the build gives the same single power-descriptor error plus the new hint. With --system-power, the descriptor lands at results/<system>/ and the checker reports no errors.

🤖 Generated with Claude Code

…ack it

submissions create builds the bundle from the uploaded run archives only. A
system_power.json kept in the submitter's local results/<system>/ tree, where a
finished submission holds it, never reaches the bundle, so the checker reports
it missing while check-submission passes locally.

- submissions create --system-power PATH supplies the descriptor for runs
  registered without one (single-system submissions only); also exposed
  through the MCP create_submission tool.
- When the checker fails with a system lacking the descriptor, create explains
  that it reads only from the run archives and names the two fixes.
- runs create warns on stderr when the run folder has no system_power.json,
  and points out a copy one level up that would not be uploaded.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅

@arav-agarwal2
arav-agarwal2 merged commit 8d45cef into main Oct 29, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant