The module digest and the discover tool: does the stdlib already do this?

[borisbat]

Why. An agent rolled its own uri_to_file_name because nothing told it uriparser already has one. Name-based search only finds what you already know the name of, and the manual's 3 MB of RST does not fit any context.

What changes.

• das2rst writes skills/daslang/references/everything.md beside the RST: one section per module, every public symbol with the first sentence of its description, overloads collapsed to one row.
• A detail/ topic with no extracted file reads its handmade/ twin, so the promoted builtin modules (fio, ast, rtti) show both halves on their RST pages too.
• The MCP server gains discover: intent words ranked by coverage, then by where they hit (name, group, description) with a small synonym list and a query-order tiebreak; FTS5 syntax filters through daslib/fts5_query; every call logs its query, hit count and top answer.
• The dupe-auditor and dupe-sweeper read the digest whole before the first function; CLAUDE.md's MCP-first table sends a writer to discover before any helper.
• The shipped-skills gate skips its purity bans for the generated file only; every other check still applies.

Observable behavior.

• daslang -documentation doc/reflections/das2rst.das -> also writes the digest (-- --digest <path> redirects it).
• fio, ast, rtti RST pages: 95 symbols printed a bare signature -> they print their handmade description.
• mcp__daslang__discover("convert uri to file name") -> uriparser::uri_to_file_name first; twelve intents measured, nine land the right symbol in the top three, the misses are description gaps.
• logs/mcp_server.log -> one mcp.discover line per call.

Where to look. daslib/rst.das (md_row, topic_path, write_digest), utils/mcp/tools/discover.das (score_row and the sort), and the two agent charters.

Validation, claims, ledger

Validation

• Green locally: sync, stamp-reach, jit-smoke, lint, format, review-md, ast-verify, hash-refs, untracked, ci-das, md-ascii, docs (Sphinx -W, no // stub, no Uncategorized), tests-aot (full test_aot).
• Four lanes red on this box for reasons outside the diff, left for CI to prove: utils-tests exits -1 at the MSBuild target while all 13 suites print green and each exits 0 run alone; tests-interp has zero assertion failures and one file, tests/watchdog/test_watchdog.das, over the 60 s per-file cap (83 s) on a loaded box; tests-jit has the same cap hit plus tests/jit_tests/jit_lib.das (7 cases that build a shared library through daslang -lib); tests-cpp fails standalone_capi_dasbind on a fixture DLL this build dir never produced. No C++, interpreter or JIT surface changes here.
• compile-sweep red was three imgui examples resolving a stale May copy of the module under the gitignored examples/graphics/modules/; with it set aside all three compile against the in-tree modules/dasImgui.
• tests/daslib/test_rst_digest.das (7 cases) and test_discover in utils/mcp/test_tools.das (9 rank-asserting cases) pass; ci/test_check_shipped_skills.py (16) and ci/test_fix_md_ascii.py pass.
• The digest joins ci/fix_md_ascii.py's shipped set: its typography is the docs' own (257 em dashes from //! comments), a rewrite would be undone by the next regeneration; mojibake is still policed. A byte-order mark found in one handmade page is stripped both at the source and by the generator.
• The daslang MCP server was down for the second half of the arc (its .mcp.json entry pointed at the deleted supervisor); lint and format ran through the CLI on the changed set.
• Woodpecker round ran against the tip before the audit batch; its one P2 (the digest lookup ignored the .claude/skills/daslang install copy) is fixed.

Claims - stated, not tested

• discover falls back to .claude/skills/daslang/references/everything.md when DAS_INSTALL_DOCDIR relocates the docs. Verified by reading the two install(DIRECTORY ...) rules; a break shows as "module digest not found" in a relocated SDK.
• digest_path (the var public in daslib/rst.das the new handmade doc names) exists and write_digest is a no-op while it is empty; checked by running the generator with and without it.
• Every name on the digest comes from module reflection through the same document_* writers as the RST, so it resolves by construction; spot-checked uri_to_file_name, fread, mkdir_rec, base_name.
• The discover ToolDef's arg_names order (query, limit, module) matches the handler; the tests call do_discover directly.
• The ci/check_shipped_skills.py skip names a file this diff adds. It cannot be fixed instead: its lines are the stdlib's own descriptions, and a lint module's description names lint rule IDs.

Not done

• 331 digest rows have no description because the symbol has none anywhere; largest gaps: result 29, option 25, lint_config 25. A //! backlog; the digest is the list.
• daslib/clargs and daslib/fts5_query have no document_module_* entry in das2rst, so they are absent from the manual and from the digest.
• The digest row for an overloaded name carries the first documented overload; fread shows the binary overload's sentence because the string overload has no doc file.
• Lint candidates surfaced by the audits, none implemented here: a CI check that the committed digest matches a fresh regeneration; a check that every backticked path in .claude/agents/*.md resolves; a REVIEW.das gate that every mcp ToolDef has a README row and a test case; the key-spelling gate for daslib/rst.das (every md_row topic equals the document_topic topic beside it).
• Two checklist findings need a ruling, not a forced edit: daslib/REVIEW.md's 60-word "daslib fact decided outside the folder" rule (split trigger from duty?) and utils/REVIEW.md's 200-word definitions preamble.
• bin/daslang.exe in this tree is a stale Sep 4 copy beside the real bin/Release/ binary; two audits and the old MCP entry picked it up. Deleting it is a local cleanup, not part of the change.

[Copilot]

🟡 Changes recommended

There are a few concrete correctness/quality issues in the new discover tool and digest topic fallback handling that should be addressed before approval.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

This PR adds a generated “module digest” for the daslang stdlib and a new MCP discover tool that intent-searches that digest, making it much easier to answer “does the stdlib already do this?” without knowing the exact symbol name.

Changes:

• Generate skills/daslang/references/everything.md as part of das2rst docs generation (with optional --digest redirect) and improve topic fallback so promoted builtins can surface both extracted and handmade descriptions.
• Add MCP discover tool (ranking + optional FTS5 filtering), wire it into the MCP registry/README, and add ranking/behavior tests.
• Update CI/shipped-skill gates and agent guidance to treat the digest as generated output while keeping other checks strict.

File summaries

File	Description
utils/mcp/tools/discover.das	New MCP tool implementing digest parsing, ranking, and optional FTS5 filtering.
utils/mcp/test_tools.das	Adds discover tool tests (ranking, synonyms, filtering, limit/error cases).
utils/mcp/REVIEW.md	Updates review checklist wording for SDK install rules and comment-keeper tests.
utils/mcp/registry_das.das	Registers discover ToolDef (schema + handler wiring).
utils/mcp/README.md	Documents discover in the MCP tool list.
tests/daslib/test_rst_digest.das	New tests for digest generation behavior (first sentence, overload collapse, BOM stripping, grouping).
tests/daslib/_rst_digest_mod.das	New fixture module used by digest tests.
skills/mcp_tools.md	Adds discover to the MCP-first tool table.
skills/internal/documentation_rst.md	Documents digest generation and --digest flag behavior.
skills/dupe_audit.md	Updates dupe-audit procedure to read the digest and use discover.
skills/daslang/SKILL.md	References the new generated everything.md digest in the language skill index.
skills/daslang/references/modules-and-stdlib.md	Points readers to everything.md for intent-based stdlib lookup.
skills/daslang/README.md	Declares references/everything.md as generated and describes regeneration rules.
skills/comment_style_hygiene.md	Clarifies //! attachment rules and when to respell to //.
doc/source/stdlib/handmade/structure_annotation-rtti-CodeOfPolicies.rst	Removes stray BOM/formatting artifact in handmade RST content.
doc/source/stdlib/handmade/function-rst-write_digest-0x29996dc1390e7fee.rst	Adds handmade description for rst::write_digest.
doc/reflections/das2rst.das	Adds clargs support for --digest, documents write_digest, and calls it at end of generation.
daslib/rst.das	Implements digest generation (md_row, md_begin, write_digest) and handmade fallback via topic_path.
daslib/REVIEW.md	Clarifies the “topic key computation must match” rule and updates architecture doc list.
CLAUDE.md	Adds discover to the “ask before writing a helper” table and streamlines language-skill reference text.
ci/test_fix_md_ascii.py	Tests that the generated digest is treated as shipped output.
ci/test_check_shipped_skills.py	Tests that the generated digest skips purity bans but still enforces other checks.
ci/REVIEW.md	Tightens policy on when bundle-gate skips are allowed (must be justified for newly added file).
ci/fix_md_ascii.py	Exempts the digest by full path in shipped set so typography isn’t rewritten.
ci/check_shipped_skills.py	Adds GENERATED_DIGEST and exempts it from purity bans while keeping other validations.
.claude/agents/dupe-sweeper.md	Updates dupe-sweeper agent charter to read the digest before reviewing functions.
.claude/agents/dupe-auditor.md	Updates dupe-auditor agent charter to read the digest before verdicts.
.claude/agents/dragon.md	Clarifies that generated .md outputs (including the digest) aren’t audited as rule documents.

Review details

• Files reviewed: 28/29 changed files
• Comments generated: 4
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

🟡 Changes recommended

The new discover tool has verified query-handling issues (case sensitivity and FTS operator detection) that can cause expected searches/filters to fail.

Get a fresh assessment by requesting another Copilot review.

Review details

• Files reviewed: 28/29 changed files
• Comments generated: 3
• Review effort level: Lite

[Copilot]

🟡 Changes recommended

Digest row capture in daslib/rst.das currently reads function topic files before they’re written, which can produce stale or missing digest descriptions.

Get a fresh assessment by requesting another Copilot review.

Review details

• Files reviewed: 28/29 changed files
• Comments generated: 1
• Review effort level: Lite

[Copilot]

🟢 Approval recommended

The changes appear coherent and covered by targeted new tests, with only a minor test-helper robustness improvement suggested.

Review details

• Files reviewed: 28/29 changed files
• Comments generated: 1
• Review effort level: Lite

[Copilot]

🟢 Approval recommended

The changes are cohesive, well-tested (new digest and discover tests), and the remaining feedback is a minor performance optimization.

Review details

• Files reviewed: 28/29 changed files
• Comments generated: 1
• Review effort level: Lite