[anandhu-eng]

The checker's first version numbered rules by their row in the Rules §9.1 table, and those numbers were later written as section numbers. 26 of the 84 rules in messages.yaml cite one of them: §10, §12, §14 and §15 don't exist, and §1, §§3–6, §7 and §9 exist but are the wrong sections.

This PR points each rule at the section that defines what it checks, checked against endpoints_policies main (4ba8224):

Rules	Was	Now
Directory and file presence (path-exists, required-dir, point-dirs, …)	§1	§8.1
src-dir, src-readme	§1	§8.1
point-count / point-cap	§2, §8	§5.3 / §5.6
Low/Med/High coverage	§§3–6	§5.3, §5.4
max-concurrency-declared	§7	§5.4
concurrency-in-range	§9	§5.5
load-pattern	§10	§6.1
min-query-count	§12	§6.4
metric-consistency-{duration,output-tokens} / -accounting	§14	§4.1 / §6.4
accuracy-gate / accuracy-present / accuracy-sample-count	§15	§4.3 / §6.6 / §4.3, §6.5

Seed rotation (§4.6) and cohorts (§4.2) are Submission Rules sections but were cited bare, which reads as the Rules (where §4.6 doesn't exist). They now say "Submission Rules §4.6", matching how the CLI already cites that document elsewhere.

The same fixes are applied to the README check tables and to code comments. A new test in test_messages.py fails if a spec names a section outside the Rules' §1–§9 or the Submission Rules' §1–§11. It fails on the old catalog.

No check behavior changes; only the cited section in output and docs.

🤖 Generated with Claude Code

[github-actions]

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅

[Copilot]

🟢 Approval recommended

The changes are limited to citations and supporting validation, with no identified blocking issues or changes to compliance-check logic.

No open issues 🎉

What changed in this PR

Corrects policy citations in the submission checker’s message registry, documentation, and comments without changing compliance-check behavior.

Changes:

• Replaces obsolete rule-row numbers with relevant policy sections.
• Explicitly labels citations belonging to Submission Rules.
• Adds citation-format and top-level section-range validation, and updates the annotation expectation.

File	Description
tests/​tools/​regenerate_fixtures.py	Clarifies seed-policy citations in comments and docstrings.
tests/​submission_checker/​test_seed_sets.py	Qualifies seed-policy citations in docstrings.
tests/​submission_checker/​test_messages.py	Adds citation validation and accepts Submission Rules prefixes.
tests/​submission_checker/​test_cohorts.py	Qualifies cohort-policy citations in docstrings.
tests/​submission_checker/​test_checks_seed_binding.py	Clarifies seed-binding policy references in docstrings.
tests/​submission_checker/​test_checks_agentic_accuracy.py	Removes an obsolete section reference from a docstring.
tests/​endpoints_submission_cli/​submissions/​test_builder.py	Qualifies the seed-policy citation in a comment.
tests/​endpoints_submission_cli/​conftest.py	Qualifies the fixture’s seed-policy comment.
tests/​endpoints_submission_cli/​commands/​test_submissions_commands.py	Corrects withdrawal-policy references in comments and docstrings.
tests/​endpoints_submission_cli/​commands/​test_check_submission_command.py	Updates the expected annotation citation.
src/​submission_checker/​seed_sets.py	Clarifies seed-policy references in documentation.
src/​submission_checker/​models/​file/​point_config.py	Qualifies seed and cohort citations in comments and docstrings.
src/​submission_checker/​models/​aggregate/​seed_binding.py	Qualifies seed-binding policy citations in docstrings.
src/​submission_checker/​models/​aggregate/​point_result.py	Corrects query-count and metric-check docstrings.
src/​submission_checker/​models/​aggregate/​context.py	Corrects coverage and accuracy references in documentation.
src/​submission_checker/​data/​seed_sets.yaml	Qualifies seed-policy citations in comments only.
src/​submission_checker/​data/​messages.yaml	Corrects registry citations and seed-message references.
src/​submission_checker/​cohorts.py	Clarifies cohort-policy references in documentation.
src/​submission_checker/​cli.py	Qualifies the seed-policy citation in CLI help.
src/​submission_checker/​checker.py	Corrects the accuracy-presence policy comment.
src/​endpoints_submission_cli/​commands/​submissions/​remove_run.py	Clarifies the rollback docstring’s policy reference.
README.md	Updates check tables, headings, and policy references.
pyproject.toml	Qualifies the package-data comment’s seed-policy citation.
docs/​endpoints-submission-cli.md	Qualifies seed-policy references in CLI documentation.

_{Review effort: Balanced}