agent-ready: template verifies green, --json reports, headless remote push, Config schema - #23
Open
HamzaHassanain wants to merge 3 commits into
Open
HamzaHassanain wants to merge 3 commits into
HamzaHassanain wants to merge 3 commits into
Conversation
A fresh `polyman new` failed `polyman verify` three times over: - validator_tests.json fed "5 10" on line 1 while val.cpp reads one int then readEoln; fixture rewritten to match (2 VALID, 4 INVALID incl. missing trailing newline) - gen.cpp omitted the trailing newline, so generated tests failed "Expected EOLN" - acc2.java declared `package solutions;`, which `java -cp <dir> acc2` can never load Add template/Config.schema.json (draft-07, mirrors ConfigFile in types.d.ts, descriptions on every field) and point Config.json at it via "$schema". Every reader tolerates the key and it is never pushed to Polygon. tests/config-schema.test.ts keeps schema and template in sync. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RhXUgi99BpnzBNT2pymkDL
Headless agents could not drive polyman: verdicts had to be scraped from
box-drawing text and `remote push` of a new problem blocked on readline.
- `run <sol> --json` / `verify --json`: exactly one JSON document on
stdout (schemaVersion, polymanVersion, command, ok, steps, per-solution
matchesTag and per-test verdicts); human output moves to stderr; exit
codes unchanged. Emitted on every failure path via a process exit hook,
so aborts before solutions run still yield valid JSON with failedStep.
- `remote push -y/--yes -n/--name <slug>`: skip the create-problem
confirmation and supply the name. Without a TTY and without the flags,
push exits 1 naming the flags instead of hanging. An invalid or taken
--name errors instead of re-prompting.
- New src/report.ts collector; formatter gains an output-stream switch
and error sink; three direct console.* calls in executor.ts routed
through fmt.
- Docs: README, GUIDE, template/instructions/commands/{run,verify,remote},
working-rules.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RhXUgi99BpnzBNT2pymkDL
- `npm test` runs `vitest run` (no watch-mode hang for agents); `test:watch` keeps the old behaviour - eslint and git ignore .claude/worktrees so a parallel agent's worktree cannot break `npm run lint` - `polyman new` hints said `generate all` / `validate all`; the flags are `--all` - CLAUDE.md: three runtime deps (execa), formatter has no spinners Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RhXUgi99BpnzBNT2pymkDL
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Makes polyman workable for headless AI agents, both as a tool and as a codebase.
Template verifies green out of the box (it did not: three bugs)
validator_tests.jsonfixture did not matchval.cpp("5 10" on line 1 vs one int + EOLN)gen.cppomitted the trailing newline, so generated tests failedExpected EOLNacc2.javadeclaredpackage solutions;, whichjava -cp <dir> acc2cannot loadHeadless CLI
polyman run <sol> --jsonandpolyman verify --json: one JSON document on stdout, human output on stderr, exit codes unchanged. Emitted on every failure path, so an abort before solutions run still yields valid JSON withfailedStepset.polyman remote push --yes --name <slug>: create a Polygon problem without prompts. With no TTY and no flags, push exits 1 naming the flags instead of hanging.Config schema
template/Config.schema.json(draft-07, mirrorsConfigFile, descriptions on every field), referenced from the template'sConfig.jsonvia$schema. Every reader tolerates the key; it is never pushed to Polygon.Tooling / doc drift
npm test=vitest run(no watch-mode hang);test:watchadded.claude/worktreesso a parallel agent's worktree cannot break lintpolyman newhints saidgenerate all; the flag is--allSample
verify --json(trimmed):{ "schemaVersion": 1, "polymanVersion": "2.3.3", "command": "verify", "ok": true, "failedStep": null, "steps": [ { "name": "compile-generators", "ok": true, "errors": [] } ], "solutions": [ { "name": "tle_solution", "tag": "TL", "matchesTag": true, "tests": [ { "testset": "tests", "index": 1, "verdict": "TLE", "timeMs": 1003 } ] } ] }Test plan
npm run build,npm run lint,npm run format:checkcleannpm test: 822 tests, 24 filespolyman new→download-testlib→verifyexits 0, all three solutions match their tagsverify --jsonexit 0, valid JSON, no ANSI on stdout; broken copy exits 1 with valid JSON andfailedStep: "validate-tests"run main --all --jsonexit 0remote push . < /dev/nullexits 1 in under a second with the flag hintNotes
run --group/--indexexits 1 on a failing test while--allexits 0;.claude/scheduled_tasks.lockis tracked;assets/checkers/has strayncmp.exeandnyesnobinaries.🤖 Generated with Claude Code
https://claude.ai/code/session_01RhXUgi99BpnzBNT2pymkDL