diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index ef44697e..48334684 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -63,8 +63,18 @@ jobs: - 'mise.toml' - 'mise-tasks/**' - '.github/workflows/ci.yml' + # Every artifact the docs build reads, not just `capsule-docs/**` — the + # generator turns two committed description artifacts into the `/reference/` + # pages, so a filter that names only the site lets a stale reference page + # publish on a change to the surface it describes. See + # design/developer-docs.md, "Artifacts cross the boundary, not toolchains". docs: - 'capsule-docs/**' + - 'capsule-cli/cli-surface.json' + - 'capsule-server/openapi.json' + # `capsule-docs/biome.jsonc` extends the root one, so a change there can fail + # this job's format and lint steps from outside `capsule-docs/**`. + - 'biome.jsonc' - 'mise.toml' - 'mise-tasks/**' - '.github/workflows/ci.yml' @@ -81,6 +91,10 @@ jobs: - 'capsule-docs/endpoint-census-allowlist.txt' - 'capsule-docs/planned-modules.txt' - 'capsule-server/openapi.json' + # No check reads this one today; it is here because the cross-links check + # resolves any repo-relative path a document names, and the reference prose + # now names this artifact. Cheap: the job installs nothing. + - 'capsule-cli/cli-surface.json' - 'capsule-*/src/**' - '**/*.md' - '**/*.mdx' diff --git a/.markdownlint-cli2.jsonc b/.markdownlint-cli2.jsonc index d8c96d94..9e285c96 100644 --- a/.markdownlint-cli2.jsonc +++ b/.markdownlint-cli2.jsonc @@ -26,6 +26,16 @@ // submodules, so without this the gate passes there and fails on any dev machine // that has run `git submodule update`. "rawshift/**", + // Generated reference pages (capsule-docs/scripts/gen-reference.mjs). Gitignored + // build output whose formatting comes from the generator, not from a contributor: + // a finding here is fixed in the generator, and the file it points at may not + // exist on the machine reading the report. + "capsule-docs/src/content/docs/reference/cli/**", + "capsule-docs/src/content/docs/reference/api/**", + // The same two directories reached through the repo-root `docs` symlink, which + // this glob walker follows (unlike the docs-truth walk, which skips symlinks). + "docs/reference/cli/**", + "docs/reference/api/**", "CLAUDE.md", // symlink to AGENTS.md "CHANGELOG.md" // generated by convco; not hand-formatted ] diff --git a/SLICES.md b/SLICES.md index ce7d3266..250f7ccc 100644 --- a/SLICES.md +++ b/SLICES.md @@ -400,9 +400,9 @@ row's remainder now lives. | S-Z5 | Dead-code removal (exports stub, CLI import planner) | design/docs | — | S | MIXED | done | | | S-Z6 | Developer-docs parity pass | design/docs | — | M | MIXED | done | | | S-Z7 | Developer reference architecture (design) | design/docs | — | S | ACTIVE | done | | -| S-Z8 | Reference shell + CLI reference | design/docs | S-Z7 | M | ACTIVE | ready | | -| S-Z9 | REST reference from the Kynos document | design/docs | S-Z8, S-D8 | M | ACTIVE | blocked | Kynos document → `S-C27`/`S-D8` | -| S-Z10 | SDK / FFI / WASM reference | design/docs | S-Z8 | M | ACTIVE | ready | | +| S-Z8 | Reference shell + CLI reference | design/docs | S-Z7 | M | ACTIVE | done | man pages/completions scoped out | +| S-Z9 | REST reference from the Kynos document | design/docs | S-Z8, S-D8 | M | ACTIVE | done | | +| S-Z10 | SDK / FFI / WASM reference | design/docs | S-Z8 | M | ACTIVE | ready | uniffi has no stable dump — own issue | | S-Z11 | Notification architecture (design) | design/docs | — | S | ACTIVE | done | | **Row counts.** 205 rows — the 129 from the v1 campaign and wave 2, the 51 the @@ -413,12 +413,11 @@ area: **87 ACTIVE / 80 RETIRED / 38 MIXED**. By status: and `part 1 done`; they are counted together). Lanes are independent by construction; within a lane, "Depends on" is the only -ordering. Seven rows read `blocked`, and only two of them are waiting on code: +ordering. Six rows read `blocked`, and only two of them are waiting on code: `S-N2` behind `S-N1`, and `S-P4` behind `S-P2`/`S-P3`. The rest are waiting on a decision rather than on an implementation — `S-C47` is a legal question, `S-C49` -and `S-C51` each need a fact the slice that found them could not settle, `S-D24` -needs a design decision, and `S-Z9` needs the Kynos document -(`S-C27`/`S-D8`). `S-P1` landing freed the rest of lane P and `S-U19` with it; +and `S-C51` each need a fact the slice that found them could not settle, and +`S-D24` needs a design decision. `S-P1` landing freed the rest of lane P and `S-U19` with it; lane U was built so the other twenty-two Apple-client slices never waited on that chain in the first place. Everything else that once read `blocked` is startable: `S-A10` and `S-P7` are done (freeing `S-B10`, `S-D16`, `S-P1`, `S-Q5` — of @@ -5631,17 +5630,26 @@ and all three slices are `done` in `capsule-core`. ### S-Z8 — Reference shell + CLI reference - **Gap:** `/reference/` has an index and nothing under it, and `capsule-cli/README.md` - still defers entirely to `capsule --help`. No `clap_mangen` or `clap_complete` exists - anywhere in the workspace, so there is no man page and no shell completion either. + still defers entirely to `capsule --help`. - **Deliverable:** the reference shell (overview page per section) plus the first real - generated surface. `capsule-cli` gains a command-tree dump with a `--check` mode and - `clap_mangen`/`clap_complete` output; the docs build renders the committed dump into - `/reference/cli/`. The CI `docs` path filter widens to name every artifact the docs - build now reads — without that, a CLI change publishes a stale page without failing - anything. + generated surface. `capsule-cli` gains a command-tree dump with a `--check` mode; the + docs build renders the committed dump into `/reference/cli/`. The CI `docs` path filter + widens to name every artifact the docs build now reads — without that, a CLI change + publishes a stale page without failing anything. +- **Scoped out: man pages and shell completions.** The slice originally named + `clap_mangen` and `clap_complete`. Neither crate appears anywhere in `Cargo.lock`, so + both would owe a row in `design/dependencies.md`, and neither produces the description + artifact the docs build reads — they are *install* artifacts, emitted for a packager, + not a description of the surface. They are a packaging slice, not this one. - **Done when:** `/reference/cli/` renders the full command tree from the committed dump, the `--check` mode fails on a hand-edited dump, and the `docs` path filter names the dump. **Tier:** docs build + the new drift gate. **Depends on:** S-Z7. +- **Landed — verified 2026-09-02.** `capsule_cli::cli::command_tree()` emits the tree, + `capsule-cli/src/bin/gen_cli_surface.rs` writes and `--check`s + `capsule-cli/cli-surface.json`, and `cli-surface-check` runs in `check-rust` beside + `openapi-check-kynos`. `capsule-docs/scripts/gen-reference.mjs` renders it into + `/reference/cli/commands/` as gitignored build output, with the hand-written overview + at `/reference/cli/`. The `docs` filter names both committed artifacts. ### S-Z9 — REST reference from the Kynos document @@ -5654,8 +5662,16 @@ and all three slices are `done` in `capsule-core`. forfeit the search index, the link validator, and the site palette. - **Done when:** the committed contract is Kynos-emitted, `openapi-check` gates it, and `/reference/api/` renders every path in it as Starlight pages that Pagefind indexes. - **Tier:** docs build + `openapi-check`. **Depends on:** S-Z8, S-D8 (**live block** — - the schema must come from Kynos, which needs `S-C27`). + **Tier:** docs build + `openapi-check`. **Depends on:** S-Z8, S-D8 (the block cleared + when `S-C34` landed the Kynos emitter and `openapi-check-kynos`). +- **Landed — verified 2026-09-02.** All 51 paths and 59 operations of + `capsule-server/openapi.json` render across eleven group pages under `/reference/api/`, + from the ordered table in `capsule-docs/scripts/reference-groups.mjs` that + `astro.config.mjs` also builds the sidebar from. The document carries no `tags` on any + operation, so the grouping is hand-curated by path prefix and the generator **fails on + an operation no group claims** — a new endpoint family cannot publish under a heading + nobody chose, and cannot silently fail to publish. A test asserts the bucketed count + equals the declared one. ### S-Z10 — SDK / FFI / WASM reference @@ -5672,6 +5688,24 @@ and all three slices are `done` in `capsule-core`. - **Done when:** each dump has a `--check` in the Rust gate, the three binding pages render from committed dumps, and `/reference/crates/` resolves. **Tier:** docs build + the new drift gates. **Depends on:** S-Z8. +- **Still `ready`, and split out of the S-Z8/S-Z9 delivery.** Neither of its two artifacts + can be produced the way the slice assumes, and the evidence is recorded here so the next + attempt starts from it: + - **uniffi exposes no stable machine-readable dump.** `uniffi_bindgen 0.31.1` — the + pinned version — offers only `generate`, `scaffolding`, and `pipeline`, and `pipeline` + documents itself as inspecting the render pipeline. The one thing resembling a dump, + `print_repr`, prints Rust `{:#?}` `Debug` of `uniffi_meta::Metadata`, which carries no + `serde` derive and no `serde` dependency. A committed artifact today is therefore + either a `Debug` blob with no compatibility promise that churns on every uniffi bump, + or a hand-written mapper over a private IR — the second parser `AGENTS.md` forbids. + The symbol-presence assertions already in `mise-tasks/gen-bindings` enumerate the verbs + and are the honest seed: they assert, they do not emit. + - **The wasm `.d.ts` gate cannot live in `check-rust`.** That gate runs + `build-check-wasm`, two `cargo check`s; the artifact comes from `build-wasm`, which + needs `wasm-bindgen-cli`, installed only by the `web` CI job. Gating it is a + `check-web` change and a web-side artifact, not a fourth entry in the Rust gate. + Filed as its own issue with this evidence. The extension point is the group table in + `capsule-docs/scripts/reference-groups.mjs`: adding a surface is a group plus a renderer. ## Deferred Migrations Register diff --git a/capsule-cli/Cargo.toml b/capsule-cli/Cargo.toml index 613286b2..e8bc024b 100644 --- a/capsule-cli/Cargo.toml +++ b/capsule-cli/Cargo.toml @@ -9,6 +9,14 @@ publish.workspace = true name = "capsule" path = "src/main.rs" +# The command-tree description artifact the documentation site is generated from (slice +# `S-Z8`). Declared explicitly rather than left to autodiscovery so the crate's two binaries +# are visible in one place: `capsule` ships, this one only ever runs from `mise run +# cli-surface` / `cli-surface-check`. +[[bin]] +name = "gen_cli_surface" +path = "src/bin/gen_cli_surface.rs" + [lib] name = "capsule_cli" path = "src/lib.rs" diff --git a/capsule-cli/cli-surface.json b/capsule-cli/cli-surface.json new file mode 100644 index 00000000..2a959b2e --- /dev/null +++ b/capsule-cli/cli-surface.json @@ -0,0 +1,539 @@ +{ + "about": "A command line interface for Capsule - the photo management platform", + "long_about": "Capsule CLI provides tools for managing your photos and albums:\n\n- Authentication management\n- Sync local and remote data\n- Check status and list files\n- Manage albums and collections", + "name": "capsule", + "schema": 1, + "subcommands": [ + { + "about": "Authentication commands", + "name": "auth", + "subcommands": [ + { + "about": "Login to Capsule", + "args": [ + { + "help": "Account email (prompted when omitted)", + "id": "email", + "long": "email", + "positional": false, + "repeatable": false, + "required": false, + "takes_value": true, + "value_names": [ + "EMAIL" + ] + }, + { + "help": "Read the password from stdin instead of prompting, so the command works in scripts and CI where there is no terminal", + "id": "password_stdin", + "long": "password-stdin", + "positional": false, + "repeatable": false, + "required": false, + "takes_value": false + } + ], + "name": "login" + }, + { + "about": "Logout from Capsule", + "name": "logout" + }, + { + "about": "Create a Capsule account and sign in", + "args": [ + { + "help": "Account email (prompted when omitted)", + "id": "email", + "long": "email", + "positional": false, + "repeatable": false, + "required": false, + "takes_value": true, + "value_names": [ + "EMAIL" + ] + }, + { + "help": "Read the password from stdin instead of prompting, so the command works in scripts and CI where there is no terminal", + "id": "password_stdin", + "long": "password-stdin", + "positional": false, + "repeatable": false, + "required": false, + "takes_value": false + } + ], + "name": "register" + }, + { + "about": "Show authentication status", + "name": "status" + } + ] + }, + { + "about": "Review a local library: flag assets, filter by flag, sweep rejects to trash", + "args": [ + { + "help": "Path to the Capsule library", + "id": "library", + "long": "library", + "positional": false, + "repeatable": false, + "required": true, + "takes_value": true, + "value_names": [ + "PATH" + ] + }, + { + "help": "Read the library passphrase from stdin instead of prompting, so culling works in scripts and CI where there is no terminal", + "id": "passphrase_stdin", + "long": "passphrase-stdin", + "positional": false, + "repeatable": false, + "required": false, + "takes_value": false + }, + { + "help": "Flag an asset as a keeper (repeatable)", + "id": "pick", + "long": "pick", + "positional": false, + "repeatable": true, + "required": false, + "takes_value": true, + "value_names": [ + "ASSET_ID" + ] + }, + { + "help": "Clear an asset's flag back to the never-flagged default (repeatable)", + "id": "neutral", + "long": "neutral", + "positional": false, + "repeatable": true, + "required": false, + "takes_value": true, + "value_names": [ + "ASSET_ID" + ] + }, + { + "help": "Flag an asset for rejection (repeatable)", + "id": "reject", + "long": "reject", + "positional": false, + "repeatable": true, + "required": false, + "takes_value": true, + "value_names": [ + "ASSET_ID" + ] + }, + { + "help": "List the assets carrying one flag instead of only counting them", + "id": "filter", + "long": "filter", + "positional": false, + "possible_values": [ + { + "help": "A keeper", + "name": "pick" + }, + { + "help": "Not yet culled either way — the never-flagged default", + "name": "neutral" + }, + { + "help": "Marked for rejection; the set `--sweep` moves to trash", + "name": "reject" + } + ], + "repeatable": false, + "required": false, + "takes_value": true, + "value_names": [ + "FLAG" + ] + }, + { + "help": "Move every rejected asset to trash. The only destructive step, and soft per retention — swept assets stay restorable until the window elapses", + "id": "sweep", + "long": "sweep", + "positional": false, + "repeatable": false, + "required": false, + "takes_value": false + }, + { + "default_values": [ + "30" + ], + "help": "Retention window, in days, the sweep's soft delete stamps", + "id": "retain_days", + "long": "retain-days", + "positional": false, + "repeatable": false, + "required": false, + "takes_value": true, + "value_names": [ + "DAYS" + ] + } + ], + "name": "cull" + }, + { + "about": "Run the offline end-to-end data-plane showcase (real cryptography, no network)", + "args": [ + { + "help": "Working directory for the demo libraries (a temp dir is used if omitted)", + "id": "workdir", + "long": "workdir", + "positional": false, + "repeatable": false, + "required": false, + "takes_value": true, + "value_names": [ + "PATH" + ] + }, + { + "help": "A real image/file to import (a small synthetic file is used if omitted)", + "id": "image", + "long": "image", + "positional": false, + "repeatable": false, + "required": false, + "takes_value": true, + "value_names": [ + "PATH" + ] + } + ], + "name": "demo" + }, + { + "about": "Import files into a local Capsule library", + "args": [ + { + "help": "Source file or directory to import. Repeatable: a split Takeout export extracted into several folders is imported by naming every part in one run, so a media file and a sidecar that landed in different parts are still paired", + "id": "paths", + "positional": true, + "repeatable": true, + "required": true, + "takes_value": true, + "value_names": [ + "PATH" + ] + }, + { + "help": "Read the source as an export from this service instead of as a plain directory tree, so its out-of-band metadata (capture time, GPS, captions, favorites, album membership) is folded into the imported assets", + "id": "provider", + "long": "provider", + "positional": false, + "possible_values": [ + { + "help": "A Google Photos export produced by Google Takeout, extracted to disk", + "name": "takeout" + } + ], + "repeatable": false, + "required": false, + "takes_value": true, + "value_names": [ + "PROVIDER" + ] + }, + { + "help": "Path to the Capsule library", + "id": "library", + "long": "library", + "positional": false, + "repeatable": false, + "required": true, + "takes_value": true, + "value_names": [ + "PATH" + ] + }, + { + "help": "Move files instead of copying them", + "id": "move", + "long": "move", + "positional": false, + "repeatable": false, + "required": false, + "takes_value": false + }, + { + "help": "Re-import files even if they already exist (duplicate override)", + "id": "force", + "long": "force", + "positional": false, + "repeatable": false, + "required": false, + "takes_value": false + }, + { + "help": "Read the library passphrase from stdin instead of prompting, so imports work in scripts and CI where there is no terminal", + "id": "passphrase_stdin", + "long": "passphrase-stdin", + "positional": false, + "repeatable": false, + "required": false, + "takes_value": false + }, + { + "help": "Push the library to the server after importing — sugar for a `capsule push` run over the same library. The import itself stays offline", + "id": "push", + "long": "push", + "positional": false, + "repeatable": false, + "required": false, + "takes_value": false + }, + { + "help": "Stage the follow-on push (`--push`) in tier order, gating the preview and original tiers on the connection class", + "id": "staged", + "long": "staged", + "positional": false, + "repeatable": false, + "required": false, + "takes_value": false + } + ], + "name": "import" + }, + { + "about": "Manage the local library", + "name": "library", + "subcommands": [ + { + "about": "Show library information", + "args": [ + { + "help": "Path to the library", + "id": "path", + "positional": true, + "repeatable": false, + "required": true, + "takes_value": true, + "value_names": [ + "PATH" + ] + } + ], + "name": "info" + }, + { + "about": "Create a new Capsule library", + "args": [ + { + "help": "Directory for the new library", + "id": "path", + "positional": true, + "repeatable": false, + "required": true, + "takes_value": true, + "value_names": [ + "PATH" + ] + }, + { + "default_values": [ + "My Library" + ], + "help": "Human-readable library name", + "id": "name", + "long": "name", + "positional": false, + "repeatable": false, + "required": false, + "takes_value": true, + "value_names": [ + "NAME" + ] + } + ], + "name": "init" + }, + { + "about": "Rebuild the SQLite index from sidecar files", + "args": [ + { + "help": "Path to the library", + "id": "path", + "positional": true, + "repeatable": false, + "required": true, + "takes_value": true, + "value_names": [ + "PATH" + ] + } + ], + "name": "rebuild" + } + ] + }, + { + "about": "List the assets the sync feed has delivered", + "args": [ + { + "help": "Include assets the server has tombstoned (deleted) as well as live ones", + "id": "include_deleted", + "long": "include-deleted", + "positional": false, + "repeatable": false, + "required": false, + "takes_value": false + } + ], + "name": "list" + }, + { + "about": "Match metadata for current file", + "args": [ + { + "help": "Path to the file to match metadata for", + "id": "path", + "positional": true, + "repeatable": false, + "required": true, + "takes_value": true, + "value_names": [ + "PATH" + ] + } + ], + "name": "match" + }, + { + "about": "Upload a local Capsule library to the server", + "args": [ + { + "help": "Path to the Capsule library to push", + "id": "library", + "long": "library", + "positional": false, + "repeatable": false, + "required": true, + "takes_value": true, + "value_names": [ + "PATH" + ] + }, + { + "help": "Read the library passphrase from stdin instead of prompting, so pushes work in scripts and CI where there is no terminal", + "id": "passphrase_stdin", + "long": "passphrase-stdin", + "positional": false, + "repeatable": false, + "required": false, + "takes_value": false + }, + { + "help": "Report what would be uploaded without opening a single upload session", + "id": "dry_run", + "long": "dry-run", + "positional": false, + "repeatable": false, + "required": false, + "takes_value": false + }, + { + "help": "Re-drive every blob regardless of what the server already holds", + "id": "force", + "long": "force", + "positional": false, + "repeatable": false, + "required": false, + "takes_value": false + }, + { + "help": "Open the tier sessions in ladder order (index → preview → original), gating the above-index tiers on the connection class, instead of opening all eagerly", + "id": "staged", + "long": "staged", + "positional": false, + "repeatable": false, + "required": false, + "takes_value": false + } + ], + "name": "push" + }, + { + "about": "Reset all local CLI data", + "args": [ + { + "help": "Reset configuration", + "id": "config", + "long": "config", + "positional": false, + "repeatable": false, + "required": false, + "takes_value": false + }, + { + "help": "Reset data directory", + "id": "data", + "long": "data", + "positional": false, + "repeatable": false, + "required": false, + "takes_value": false + }, + { + "help": "Reset cache directory", + "id": "cache", + "long": "cache", + "positional": false, + "repeatable": false, + "required": false, + "takes_value": false + }, + { + "help": "Reset all data", + "id": "all", + "long": "all", + "positional": false, + "repeatable": false, + "required": false, + "takes_value": false + } + ], + "name": "reset" + }, + { + "about": "Show current status", + "name": "status" + }, + { + "about": "Sync local and remote data", + "args": [ + { + "help": "Discard the saved cursor and re-drain the feed from the start. The per-album anti-rewind floor still applies, so this cannot resurrect stale entries", + "id": "force", + "long": "force", + "positional": false, + "repeatable": false, + "required": false, + "takes_value": false + }, + { + "help": "Perform a dry run without making changes", + "id": "dry_run", + "long": "dry-run", + "positional": false, + "repeatable": false, + "required": false, + "takes_value": false + } + ], + "name": "sync" + } + ] +} diff --git a/capsule-cli/src/bin/gen_cli_surface.rs b/capsule-cli/src/bin/gen_cli_surface.rs new file mode 100644 index 00000000..6551eb4b --- /dev/null +++ b/capsule-cli/src/bin/gen_cli_surface.rs @@ -0,0 +1,82 @@ +//! Deterministic dump of the `capsule` command tree (slice `S-Z8`). +//! +//! Serializes [`capsule_cli::cli::command_tree`] to `capsule-cli/cli-surface.json` and, with +//! `--check`, fails when the committed copy is stale. It is the CLI's half of the rule that +//! artifacts cross the toolchain boundary rather than toolchains (`design/developer-docs.md`): +//! the documentation build installs bun and nothing else, so it cannot ask cargo what +//! `capsule --help` says. It reads this file instead, and this binary is what makes a drifted +//! file fail CI. +//! +//! Deliberately the same shape as `capsule-server/src/bin/gen_openapi.rs` — `[FILE]` default, +//! `--check`, byte comparison, trailing newline — because two description artifacts that are +//! refreshed and gated differently are two things to remember instead of one. +//! +//! It needs no database, no library, no key material, no network: `command_tree()` walks a +//! `clap::Command` built from compile-time attributes. That is what lets `--check` run in the +//! Rust check gate beside `openapi-check-kynos`. +//! +//! ## Its output is English on purpose +//! +//! `xtask i18n-guard` scans `capsule-cli/src/**` because the `capsule` binary renders prose to +//! a user in their own language. This binary does not: it runs from `mise run cli-surface` and +//! from CI, is never installed, and its reader is a developer looking at a task's output. +//! `xtask::i18n_guard::NEVER_SCANNED` carves `capsule-cli/src/bin/` out of that root for +//! exactly that reason, so the lines below are plain English and stay that way — routing a +//! build tool's status line through `locales/` would put a string no user can reach into every +//! translation catalog. +//! +//! Usage: +//! - `gen_cli_surface [FILE]` writes the document (default `capsule-cli/cli-surface.json`). +//! - `gen_cli_surface --check [FILE]` fails if the committed document is stale, writing nothing. + +use std::path::PathBuf; + +use clap::Parser; +use color_eyre::eyre::{Context, Result, bail}; + +#[derive(Parser)] +#[command(author, version, about, long_about = None)] +struct Cli { + /// Output path for the command-tree document (relative to the repo root). + #[arg(value_name = "FILE", default_value = "capsule-cli/cli-surface.json")] + output: PathBuf, + + /// Verify the committed document is up to date instead of writing it (CI drift gate). + #[arg(long)] + check: bool, +} + +fn main() -> Result<()> { + color_eyre::install()?; + let cli = Cli::parse(); + + // Pretty-printed with a trailing newline: the artifact is reviewed as a diff, so a + // one-line blob would hide exactly the change a reviewer is there to see. + let mut json = serde_json::to_string_pretty(&capsule_cli::cli::command_tree()) + .wrap_err("serializing the command tree to JSON")?; + json.push('\n'); + + if cli.check { + let committed = std::fs::read_to_string(&cli.output).wrap_err_with(|| { + format!("cannot read committed document at {}", cli.output.display()) + })?; + if committed != json { + bail!( + "command tree at {} is out of sync with the `capsule` argument surface; run \ + `mise run cli-surface` and commit the result", + cli.output.display() + ); + } + println!("command tree is up to date: {}", cli.output.display()); + } else { + if let Some(parent) = cli.output.parent() { + std::fs::create_dir_all(parent) + .wrap_err_with(|| format!("creating {}", parent.display()))?; + } + std::fs::write(&cli.output, &json) + .wrap_err_with(|| format!("writing {}", cli.output.display()))?; + println!("Wrote {}", cli.output.display()); + } + + Ok(()) +} diff --git a/capsule-cli/src/cli/mod.rs b/capsule-cli/src/cli/mod.rs index e3cc16ba..0b5b7612 100644 --- a/capsule-cli/src/cli/mod.rs +++ b/capsule-cli/src/cli/mod.rs @@ -1,15 +1,519 @@ +//! The `capsule` argument surface, and the machine-readable description of it that the +//! documentation site is generated from. +//! +//! [`Cli`] stays `pub(crate)`: the parsed command is dispatch state, not API. What crosses +//! the crate boundary is [`command_tree`], the description artifact behind +//! `/reference/cli/` (slice `S-Z8`). + pub(crate) mod commands; -use clap::Parser; +use clap::{Arg, ArgAction, Command, CommandFactory, Parser}; pub(crate) use commands::*; +use serde_json::{Map, Value}; + +/// Every field name the command-tree document uses, named once. +/// +/// The document is hand-built rather than derived from a struct, because its shape is a +/// projection of `clap`'s builder API and no Rust type in this crate has that shape. Naming +/// the keys here is what keeps that from meaning "spelled ad hoc": this block is the +/// vocabulary `capsule-docs/scripts/gen-reference.mjs` reads on the other side, and a typo +/// in a key is a field the generator silently never finds. +mod field { + pub(super) const SCHEMA: &str = "schema"; + pub(super) const NAME: &str = "name"; + pub(super) const ABOUT: &str = "about"; + pub(super) const LONG_ABOUT: &str = "long_about"; + pub(super) const ARGS: &str = "args"; + pub(super) const SUBCOMMANDS: &str = "subcommands"; + pub(super) const ID: &str = "id"; + pub(super) const POSITIONAL: &str = "positional"; + pub(super) const REQUIRED: &str = "required"; + pub(super) const TAKES_VALUE: &str = "takes_value"; + pub(super) const REPEATABLE: &str = "repeatable"; + pub(super) const LONG: &str = "long"; + pub(super) const SHORT: &str = "short"; + pub(super) const VALUE_NAMES: &str = "value_names"; + pub(super) const POSSIBLE_VALUES: &str = "possible_values"; + pub(super) const DEFAULT_VALUES: &str = "default_values"; + pub(super) const HELP: &str = "help"; + pub(super) const LONG_HELP: &str = "long_help"; +} + +/// Schema version of the emitted command-tree document. +/// +/// Bumped only when a consumer must change to keep reading it — adding an optional field is +/// not a bump, renaming or removing one is. `capsule-docs/scripts/gen-reference.mjs` refuses +/// a version it was not written against rather than rendering a half-understood document. +const COMMAND_TREE_SCHEMA: u32 = 1; #[derive(Parser, Debug)] #[command(name = "capsule")] #[command(about = "A command line interface for Capsule - the photo management platform")] #[command( - long_about = "Capsule CLI provides tools for managing your photos and albums:\n• Authentication management\n• Sync local and remote data\n• Check status and list files\n• Manage albums and collections" + // Markdown list markers, not `•`: this text is the root `long_about` in the committed + // command tree, and the reference page renders it as prose. Bullet characters soft-wrap + // into one run-on paragraph there, while `-` renders as the list it already is. A + // terminal shows `-` as a list too, so `capsule --help` loses nothing. + long_about = "Capsule CLI provides tools for managing your photos and albums:\n\n- Authentication management\n- Sync local and remote data\n- Check status and list files\n- Manage albums and collections" )] pub(crate) struct Cli { #[command(subcommand)] pub command: Commands, } + +/// The `capsule` command tree as JSON — the committed description artifact +/// `capsule-cli/cli-surface.json`, emitted by the `gen_cli_surface` binary and rendered +/// into `/reference/cli/` by the documentation build (slice `S-Z8`). +/// +/// # This output is deterministic, and independent of the process locale +/// +/// Both properties are load-bearing, because the artifact is committed and drift-gated: +/// `mise run cli-surface-check` fails on any byte difference, so a value that varies by +/// machine, environment, or run would fail CI on a tree nobody changed. +/// +/// - **Deterministic.** Subcommands are sorted by name rather than emitted in +/// declaration order, so reordering an enum variant cannot churn the artifact. +/// Arguments keep declaration order, which for a `clap` derive is field order, which +/// for a positional *is* its position — sorting them would destroy that meaning. +/// Object keys come out sorted because `serde_json::Map` is a `BTreeMap` here +/// (`preserve_order` is off). Nothing is read from the clock, the filesystem, or the +/// environment. +/// - **Locale-independent.** Every string below comes from a `clap` attribute or a doc +/// comment — compile-time English `&'static str` — and `StyledStr`'s `Display` is +/// documented as colour-unaware, so no ANSI escape can leak in from a terminal that +/// supports colour. The process locale is never consulted: this function does **not** +/// call [`crate::i18n::cli_bundle`], which negotiates `LC_ALL`/`LC_MESSAGES`/`LANG`. +/// +/// **If help text is ever localized, it must be resolved here through +/// `Bundle::for_locale("en")`, never through `cli_bundle()`.** The artifact describes one +/// surface in one language; a bundle negotiated from the environment would make +/// `cli-surface-check` pass or fail according to the developer's `LANG`, and the drift +/// gate would stop meaning anything. Localizing the *rendered* help a user sees is a +/// separate concern from describing the surface. +/// +/// The tree describes the surface this crate *declares*. `clap`'s generated `--help` (and +/// `--version`, were one configured) is deliberately absent: [`Command::build`] is not +/// called, so no synthesized argument is described, and the reference page does not repeat +/// `--help` under all sixteen commands. Hidden commands and arguments are skipped for the +/// same reason they are hidden. +#[must_use] +pub fn command_tree() -> Value { + let mut root = describe_command(&Cli::command()); + root.insert( + field::SCHEMA.to_owned(), + Value::from(u64::from(COMMAND_TREE_SCHEMA)), + ); + Value::Object(root) +} + +/// Describe one command and, recursively, its subcommands. +/// +/// Recursion is bounded by the derive: the tree is a finite `enum` nesting, so there is no +/// cycle to guard against and no depth limit to pick. +fn describe_command(command: &Command) -> Map { + let mut out = Map::new(); + out.insert(field::NAME.to_owned(), Value::from(command.get_name())); + + if let Some(about) = command.get_about() { + out.insert(field::ABOUT.to_owned(), Value::from(about.to_string())); + } + // Emitted only when it says something `about` does not, so the artifact does not carry + // the same paragraph twice for every command whose doc comment is one line long. + if let Some(long_about) = command.get_long_about() { + let long_about = long_about.to_string(); + if Some(long_about.as_str()) != command.get_about().map(ToString::to_string).as_deref() { + out.insert(field::LONG_ABOUT.to_owned(), Value::from(long_about)); + } + } + + let args: Vec = command + .get_arguments() + .filter(|arg| !arg.is_hide_set()) + .map(|arg| Value::Object(describe_arg(arg))) + .collect(); + if !args.is_empty() { + out.insert(field::ARGS.to_owned(), Value::from(args)); + } + + let mut subcommands: Vec<&Command> = command + .get_subcommands() + .filter(|subcommand| !subcommand.is_hide_set()) + .collect(); + subcommands.sort_by(|a, b| a.get_name().cmp(b.get_name())); + if !subcommands.is_empty() { + let described: Vec = subcommands + .into_iter() + .map(|subcommand| Value::Object(describe_command(subcommand))) + .collect(); + out.insert(field::SUBCOMMANDS.to_owned(), Value::from(described)); + } + + out +} + +/// Describe one argument: how it is spelled, whether it takes a value, and what the help +/// says about it. +fn describe_arg(arg: &Arg) -> Map { + let mut out = Map::new(); + out.insert(field::ID.to_owned(), Value::from(arg.get_id().as_str())); + out.insert( + field::POSITIONAL.to_owned(), + Value::from(arg.is_positional()), + ); + out.insert( + field::REQUIRED.to_owned(), + Value::from(arg.is_required_set()), + ); + out.insert(field::TAKES_VALUE.to_owned(), Value::from(takes_value(arg))); + out.insert( + field::REPEATABLE.to_owned(), + Value::from(is_repeatable(arg)), + ); + + if let Some(long) = arg.get_long() { + out.insert(field::LONG.to_owned(), Value::from(long)); + } + if let Some(short) = arg.get_short() { + out.insert(field::SHORT.to_owned(), Value::from(short.to_string())); + } + // Both of these are asked only of a value-taking argument, because the derive answers + // them for a flag too and both answers are internal detail rather than surface. A + // `bool` field gets a `value_name` synthesized from its identifier (`PASSWORD_STDIN`) + // that no user may type, and `get_possible_values` reports the `true`/`false` its bool + // parser accepts — rendering either would document `--password-stdin ` + // taking `true` or `false`, which is not the flag `capsule --help` offers. + if takes_value(arg) { + if let Some(names) = arg.get_value_names() { + let names: Vec = names + .iter() + .map(|name| Value::from(name.to_string())) + .collect(); + out.insert(field::VALUE_NAMES.to_owned(), Value::from(names)); + } + + let possible: Vec = arg + .get_possible_values() + .iter() + .filter(|value| !value.is_hide_set()) + .map(|value| { + let mut entry = Map::new(); + entry.insert(field::NAME.to_owned(), Value::from(value.get_name())); + if let Some(help) = value.get_help() { + entry.insert(field::HELP.to_owned(), Value::from(help.to_string())); + } + Value::Object(entry) + }) + .collect(); + if !possible.is_empty() { + out.insert(field::POSSIBLE_VALUES.to_owned(), Value::from(possible)); + } + } + + // `OsStr` here is `clap`'s, which derefs to the standard one. Every default in this + // surface is ASCII, so the lossy conversion is exact; a non-UTF-8 default would be + // unrepresentable in JSON either way. + let defaults: Vec = arg + .get_default_values() + .iter() + .map(|value| Value::from(value.to_string_lossy().into_owned())) + .collect(); + if !defaults.is_empty() { + out.insert(field::DEFAULT_VALUES.to_owned(), Value::from(defaults)); + } + + if let Some(help) = arg.get_help() { + out.insert(field::HELP.to_owned(), Value::from(help.to_string())); + } + if let Some(long_help) = arg.get_long_help() { + let long_help = long_help.to_string(); + if Some(long_help.as_str()) != arg.get_help().map(ToString::to_string).as_deref() { + out.insert(field::LONG_HELP.to_owned(), Value::from(long_help)); + } + } + + out +} + +/// Whether the argument consumes a value (`--library `) rather than being a flag +/// (`--force`). +/// +/// Read from the action rather than from `num_args`, which the derive leaves unset unless +/// a `#[arg(num_args = …)]` says otherwise. `ArgAction` is `#[non_exhaustive]`, so a new +/// value-taking action would arrive here as `false` — visible as a missing placeholder on +/// the reference page, never as a wrong one. +fn takes_value(arg: &Arg) -> bool { + matches!(arg.get_action(), ArgAction::Set | ArgAction::Append) +} + +/// Whether the argument may be given more than once (`--pick --pick `). +/// +/// `Count` is unreachable on today's surface — no argument in this CLI is a `-vvv`-style +/// counter — and is matched anyway because it is the other action that means "give this +/// again", and omitting it would make the first counting flag document itself as single-use. +fn is_repeatable(arg: &Arg) -> bool { + matches!(arg.get_action(), ArgAction::Append | ArgAction::Count) + || arg + .get_num_args() + .is_some_and(|range| range.max_values() > 1) +} + +#[cfg(test)] +mod tests { + use super::*; + + fn subcommand<'a>(parent: &'a Value, name: &str) -> &'a Value { + parent + .get("subcommands") + .and_then(Value::as_array) + .expect("the command has subcommands") + .iter() + .find(|entry| entry.get("name").and_then(Value::as_str) == Some(name)) + .unwrap_or_else(|| panic!("subcommand `{name}` is described")) + } + + fn arg<'a>(command: &'a Value, id: &str) -> &'a Value { + command + .get("args") + .and_then(Value::as_array) + .expect("the command has arguments") + .iter() + .find(|entry| entry.get("id").and_then(Value::as_str) == Some(id)) + .unwrap_or_else(|| panic!("argument `{id}` is described")) + } + + /// The property the committed artifact and its `--check` gate rest on. A tree that + /// varied between calls would fail `cli-surface-check` on a tree nobody changed. + #[test] + fn the_tree_is_byte_identical_across_calls() { + let first = serde_json::to_string_pretty(&command_tree()).expect("the tree serializes"); + let second = serde_json::to_string_pretty(&command_tree()).expect("the tree serializes"); + assert_eq!(first, second); + } + + #[test] + fn the_tree_carries_its_schema_version() { + assert_eq!( + command_tree().get("schema").and_then(Value::as_u64), + Some(u64::from(COMMAND_TREE_SCHEMA)) + ); + } + + /// No terminal escape may reach a committed file: it would make the artifact depend on + /// whether the emitting shell claimed colour support. + #[test] + fn the_tree_carries_no_terminal_escapes() { + let json = serde_json::to_string(&command_tree()).expect("the tree serializes"); + assert!( + !json.contains('\u{1b}'), + "an ANSI escape reached the description artifact" + ); + } + + #[test] + fn subcommands_are_sorted_by_name() { + let tree = command_tree(); + let names: Vec<&str> = tree + .get("subcommands") + .and_then(Value::as_array) + .expect("the root has subcommands") + .iter() + .filter_map(|entry| entry.get("name").and_then(Value::as_str)) + .collect(); + let mut sorted = names.clone(); + sorted.sort_unstable(); + assert_eq!(names, sorted); + assert!(names.contains(&"auth")); + assert!(names.contains(&"import")); + } + + /// Arguments keep declaration order, which for a positional is its position. Sorting + /// them would silently reorder `capsule library init --name`. + #[test] + fn arguments_keep_declaration_order_so_positionals_stay_in_position() { + let tree = command_tree(); + let init = subcommand(subcommand(&tree, "library"), "init"); + let ids: Vec<&str> = init + .get("args") + .and_then(Value::as_array) + .expect("`library init` has arguments") + .iter() + .filter_map(|entry| entry.get("id").and_then(Value::as_str)) + .collect(); + assert_eq!(ids, vec!["path", "name"]); + assert_eq!( + arg(init, "path").get("positional"), + Some(&Value::from(true)) + ); + assert_eq!( + arg(init, "name").get("positional"), + Some(&Value::from(false)) + ); + assert_eq!( + arg(init, "name").get("default_values"), + Some(&Value::from(vec![Value::from("My Library")])) + ); + } + + #[test] + fn a_flag_is_distinguished_from_a_value_taking_option() { + let tree = command_tree(); + let import = subcommand(&tree, "import"); + + let library = arg(import, "library"); + assert_eq!(library.get("takes_value"), Some(&Value::from(true))); + assert_eq!(library.get("long"), Some(&Value::from("library"))); + assert_eq!( + library.get("value_names"), + Some(&Value::from(vec![Value::from("PATH")])) + ); + + let force = arg(import, "force"); + assert_eq!(force.get("takes_value"), Some(&Value::from(false))); + assert_eq!(force.get("repeatable"), Some(&Value::from(false))); + // The derive answers `value_name` and `possible_values` for a flag as well, with + // its own internals: `FORCE` as a placeholder nobody types, and the `true`/`false` + // its bool parser accepts. Describing either would invent a surface. + assert!(force.get("value_names").is_none()); + assert!(force.get("possible_values").is_none()); + } + + #[test] + fn a_repeatable_argument_is_marked_repeatable() { + let tree = command_tree(); + let paths = arg(subcommand(&tree, "import"), "paths"); + assert_eq!(paths.get("repeatable"), Some(&Value::from(true))); + assert_eq!(paths.get("required"), Some(&Value::from(true))); + + let pick = arg(subcommand(&tree, "cull"), "pick"); + assert_eq!(pick.get("repeatable"), Some(&Value::from(true))); + } + + #[test] + fn an_enumerated_value_carries_its_variants() { + let tree = command_tree(); + let provider = arg(subcommand(&tree, "import"), "provider"); + let names: Vec<&str> = provider + .get("possible_values") + .and_then(Value::as_array) + .expect("`--provider` enumerates its values") + .iter() + .filter_map(|entry| entry.get("name").and_then(Value::as_str)) + .collect(); + assert_eq!(names, vec!["takeout"]); + + let filter = arg(subcommand(&tree, "cull"), "filter"); + let flags: Vec<&str> = filter + .get("possible_values") + .and_then(Value::as_array) + .expect("`--filter` enumerates its values") + .iter() + .filter_map(|entry| entry.get("name").and_then(Value::as_str)) + .collect(); + assert_eq!(flags, vec!["pick", "neutral", "reject"]); + } + + /// The property the drift gate rests on that no other test reaches: the artifact is + /// byte-compared, so if any string in it were negotiated from the environment, + /// `cli-surface-check` would pass or fail according to the developer's `LANG`. + /// + /// `LC_ALL` is what `crate::i18n::cli_bundle` reads first, and `tr-TR` is the locale + /// that breaks case-folding implementations, so between them they exercise both the + /// negotiation path and the classic locale-sensitivity trap. `nextest` runs each test in + /// its own process, which is what makes mutating the environment here safe. + #[test] + fn the_tree_is_identical_under_two_different_locales() { + // Captured and put back below. `nextest` gives each test its own process, so this + // cannot reach another test — but a leaked `LC_ALL=tr_TR` would still be visible to + // anything else in *this* process, and a test that changes global state and does not + // change it back is one a reader has to prove harmless every time they see it. + let saved: Vec<(&str, Option)> = ["LC_ALL", "LANG"] + .iter() + .map(|name| (*name, std::env::var(name).ok())) + .collect(); + + let render = |locale: &str| { + // SAFETY: single-threaded test body in a process nextest gives this test alone. + unsafe { + std::env::set_var("LC_ALL", locale); + std::env::set_var("LANG", locale); + } + serde_json::to_string_pretty(&command_tree()).expect("the tree serializes") + }; + let english = render("en_US.UTF-8"); + let turkish = render("tr_TR.UTF-8"); + let japanese = render("ja_JP.UTF-8"); + + for (name, value) in saved { + // SAFETY: as above. + unsafe { + match value { + Some(value) => std::env::set_var(name, value), + None => std::env::remove_var(name), + } + } + } + + assert_eq!(english, turkish); + assert_eq!(english, japanese); + // Guards against the whole comparison passing because every render was empty. + assert!(english.contains("\"name\": \"capsule\"")); + } + + /// Both branches of the `long_about`/`long_help` dedup: a distinct long form is carried, + /// an identical one is dropped rather than stored twice. + #[test] + fn a_long_form_is_carried_only_when_it_differs_from_the_short_one() { + let distinct = Command::new("x") + .about("Short.") + .long_about("Short.\n\nAnd more.") + .arg( + clap::Arg::new("a") + .long("a") + .help("Short help.") + .long_help("Short help.\n\nAnd more."), + ); + let described = describe_command(&distinct); + assert_eq!( + described.get(field::LONG_ABOUT).and_then(Value::as_str), + Some("Short.\n\nAnd more.") + ); + let arg_entry = &described + .get(field::ARGS) + .and_then(Value::as_array) + .expect("the command has arguments")[0]; + assert_eq!( + arg_entry.get(field::LONG_HELP).and_then(Value::as_str), + Some("Short help.\n\nAnd more.") + ); + + let same = Command::new("x").about("Short.").long_about("Short.").arg( + clap::Arg::new("a") + .long("a") + .help("Short help.") + .long_help("Short help."), + ); + let described = describe_command(&same); + assert!(described.get(field::LONG_ABOUT).is_none()); + assert!( + described + .get(field::ARGS) + .and_then(Value::as_array) + .expect("the command has arguments")[0] + .get(field::LONG_HELP) + .is_none() + ); + } + + /// `Command::build` is deliberately not called, so the artifact describes only what + /// this crate declares — `--help` is not repeated under every command. + #[test] + fn the_tree_omits_claps_synthesized_help_argument() { + let tree = command_tree(); + assert!(subcommand(&tree, "status").get("args").is_none()); + assert!( + !serde_json::to_string(&tree) + .expect("the tree serializes") + .contains("\"id\":\"help\"") + ); + } +} diff --git a/capsule-docs/.gitignore b/capsule-docs/.gitignore index 6240da8b..dfc3d2ef 100644 --- a/capsule-docs/.gitignore +++ b/capsule-docs/.gitignore @@ -1,5 +1,13 @@ # build output dist/ +# generated reference pages (capsule-docs/scripts/gen-reference.mjs) +# +# Rule 2 of design/developer-docs.md: reference pages are generated, never written. A +# committed copy is a second source of truth that can disagree with the artifact it came +# from, and the whole point is that it cannot. The hand-written overview for each surface +# is its sibling file (reference/cli.md), not a file inside these directories. +src/content/docs/reference/cli/ +src/content/docs/reference/api/ # generated types .astro/ diff --git a/capsule-docs/astro.config.mjs b/capsule-docs/astro.config.mjs index 5188a74f..53922189 100644 --- a/capsule-docs/astro.config.mjs +++ b/capsule-docs/astro.config.mjs @@ -3,6 +3,7 @@ import tailwindcss from '@tailwindcss/vite'; // @ts-check import { defineConfig } from 'astro/config'; import starlightLinksValidator from 'starlight-links-validator'; +import { referenceSidebar } from './scripts/reference-groups.mjs'; import rehypeNoTranslate from './src/lib/rehype-notranslate.mjs'; // https://astro.build/config @@ -149,8 +150,13 @@ export default defineConfig({ { // Hand-curated, not autogenerated: generated reference pages must not // determine navigation order. See design/developer-docs.md. + // + // Curated in `scripts/reference-groups.mjs` rather than inline, because + // the generator reads the same list to decide which pages exist. One + // edit moves a page and its navigation entry together, and a group + // with navigation but no page is not expressible. label: 'Reference', - items: [{ slug: 'reference' }], + items: referenceSidebar(), }, ], customCss: ['./src/styles/global.css'], diff --git a/capsule-docs/package.json b/capsule-docs/package.json index 1e21abc3..03aa9971 100644 --- a/capsule-docs/package.json +++ b/capsule-docs/package.json @@ -4,12 +4,12 @@ "version": "0.1.0", "license": "AGPL-3.0-only", "scripts": { - "dev": "astro dev", - "start": "astro dev", - "build": "astro build", - "preview": "wrangler pages dev ./dist", + "dev": "bun scripts/gen-reference.mjs && astro dev", + "start": "bun scripts/gen-reference.mjs && astro dev", + "build": "bun scripts/gen-reference.mjs && astro build", + "preview": "bun run build && wrangler pages dev ./dist", "astro": "astro", - "deploy": "wrangler pages deploy ./dist", + "deploy": "bun run build && wrangler pages deploy ./dist", "test": "vitest run" }, "dependencies": { diff --git a/capsule-docs/scripts/docs-truth.mjs b/capsule-docs/scripts/docs-truth.mjs index 7e15a592..5f23aeea 100644 --- a/capsule-docs/scripts/docs-truth.mjs +++ b/capsule-docs/scripts/docs-truth.mjs @@ -24,6 +24,11 @@ * toolchain: that rule governs regenerating an artifact, and these checks * cross-reference committed text against committed text. * + * That last claim is what excludes the generated `/reference/` pages, which + * `scripts/lib/walk.mjs` prunes by path: they are gitignored build output, so + * they exist on a machine that has built the site and on no CI runner, and a + * check that read them would answer differently in the two places. + * * Usage: `bun capsule-docs/scripts/docs-truth.mjs` from the repository root, * or `mise run check-docs-truth`. */ diff --git a/capsule-docs/scripts/gen-reference.mjs b/capsule-docs/scripts/gen-reference.mjs new file mode 100644 index 00000000..2bddd21b --- /dev/null +++ b/capsule-docs/scripts/gen-reference.mjs @@ -0,0 +1,1190 @@ +#!/usr/bin/env node + +/** + * gen-reference — emit `/reference/` from the committed description artifacts. + * + * `design/developer-docs.md` fixes the boundary at *artifacts, not toolchains*: the CI + * `docs` job installs bun and nothing else, so this script may never shell out to cargo. It + * reads committed JSON and writes Markdown, which is the whole of its contract. + * + * The pages it writes are ordinary content-collection entries under + * `src/content/docs/reference/`, which is what buys the rest for free — Pagefind indexes + * them, `starlight-links-validator` checks their links, the `PageTitle` override renders + * their status badge, and the notranslate rehype pass marks up their technical terms. A + * mounted OpenAPI application would have had none of that. + * + * They are also **gitignored**, and that is the point of rule 2: a generated page is never + * edited, so committing one only creates a copy that can disagree with its source. Fix the + * clap `about` or the schema description and regenerate. + * + * Six failure modes are deliberately fatal rather than degraded, because + * `developer-docs.md` calls a stale reference page worse than a missing one — a missing page + * is obvious and a wrong one is believed: + * + * 1. an artifact is absent or unparseable — exit naming the path; + * 2. an artifact's `schema` is one this script was not written against — exit rather than + * render a half-understood document; + * 3. an operation matches no group in `reference-groups.mjs` — exit naming it, so a new + * endpoint family cannot publish unlisted; + * 4. a request or response carrier offers **more than one media type** — this renderer + * shows one body per carrier, so a second would be dropped silently and the page would + * claim an endpoint accepts only JSON when it also accepts CBOR; + * 5. a schema composes with **`oneOf`, `allOf`, or `anyOf`** — this renderer flattens a + * schema to a property table, which cannot express a union or an intersection, and + * would render one as an empty or a half-true model; + * 6. artifact prose carries a **setext heading** — see [`demoteHeadings`]. + * + * The last three are unreachable on the committed document today. They are fatal rather + * than deferred because each is a case where the renderer's *silent* answer is a confident + * lie, and a build failure naming the operation is how the first one to appear gets + * handled instead of shipped. + * + * Usage: `bun capsule-docs/scripts/gen-reference.mjs` from anywhere; `package.json` runs it + * before `astro dev` and `astro build`. + */ + +import { mkdirSync, readFileSync, rmSync, writeFileSync } from 'node:fs'; +import { dirname, join, resolve } from 'node:path'; +import { fileURLToPath } from 'node:url'; +import { API_GROUPS, CLI_PAGES, groupForPath } from './reference-groups.mjs'; + +/** Repo-relative path of the committed command-tree artifact. */ +export const CLI_SURFACE = 'capsule-cli/cli-surface.json'; + +/** Command-tree schema version this script understands. */ +const CLI_SCHEMA = 1; + +/** Repo-relative path of the committed Kynos OpenAPI document. */ +export const OPENAPI_DOCUMENT = 'capsule-server/openapi.json'; + +/** + * OpenAPI major.minor this generator renders. + * + * Pinned rather than accepted loosely because `AGENTS.md` requires the served document to be + * 3.2 and forbids emitting a 3.1 or 3.0 one: a document that arrived at 3.1 would mean the + * emitter regressed, and rendering it anyway would publish the regression as documentation. + */ +const OPENAPI_VERSION = '3.2'; + +/** Repo-relative directory the generated CLI pages are written to. */ +const CLI_OUT = 'capsule-docs/src/content/docs/reference/cli'; + +/** Repo-relative directory the generated REST pages are written to. */ +const API_OUT = 'capsule-docs/src/content/docs/reference/api'; + +/** HTTP methods a path item may carry. Anything else in a path item is not an operation. */ +const METHODS = [ + 'get', + 'put', + 'post', + 'delete', + 'options', + 'head', + 'patch', + 'trace', +]; + +/** + * How deep a `$ref` chain is followed before deeper models are named but not expanded. + * + * Set above the committed document's needs, not at them. The deepest chain any group + * reaches is 3 — `SyncPageResponse` → `SyncEntry` → `SyncBlobRef` → `WireBlobRole`, on + * `/reference/api/sync/` — so at the previous value of 2 that enum was named on the page + * and defined nowhere. Four documents the complete closure of every group today with a + * level of headroom; the measured cost of the whole closure over the bounded walk is one + * additional schema across all eleven pages. + * + * A model reachable only deeper than this is still *named* on the page, as bare code rather + * than a link, so the bound degrades to "less detail" and never to a link that goes nowhere. + * That is why exceeding it is not fatal, unlike the cases in the module header. + */ +const MAX_SCHEMA_DEPTH = 4; + +/** The banner every generated page carries, as an HTML comment and as prose. */ +const GENERATED_BY = 'capsule-docs/scripts/gen-reference.mjs'; + +/** A line that opens a fenced block: ``` or ~~~ with any info string. */ +const FENCE_OPEN = /^\s*(`{3,}|~{3,})/; + +/** A line that *closes* one: the same run with nothing after it but whitespace. */ +const FENCE_CLOSE = /^\s*(`{3,}|~{3,})\s*$/; + +/** + * A setext underline: a run of `=` or `-` alone on its line. + * + * **One character is enough.** CommonMark's `setext heading underline` is `= +` or `- +`, + * not two or more, so `Title` over a lone `-` is an h2 exactly as `Title` over `---` is. + * Requiring two let a single-character underline through the check that exists to catch it, + * which shipped an undemoted heading into a page body — the defect, arriving through the + * guard against the defect. + * + * Up to three leading spaces, and trailing whitespace, both per the spec. + */ +const SETEXT_UNDERLINE = /^\s{0,3}(={1,}|-{1,})\s*$/; + +/** + * Shift every ATX heading in `markdown` down by `offset` levels, clamped at h6. + * + * Description prose in both artifacts is written for its own context and carries its own + * headings — `openapi.json` has operation descriptions opening at `#`. Interpolated + * unchanged, such a heading becomes a second h1 on a page whose h1 is the Starlight title, + * which breaks the document outline and the on-page table of contents. + * + * Fenced blocks are skipped: a `#` on the first column of a shell example is a comment, not + * a heading, and demoting it would corrupt the example. + * + * **ATX only, and a setext heading is fatal.** Rewriting a line based on the line below it + * is a different and more fragile transformation than prefixing hashes — a `---` under a + * paragraph is a thematic break, and over one it is frontmatter — so this function does not + * attempt it. Silently leaving one alone is worse than not supporting it: an `=====` + * underline in an operation description would put an undemoted h1 in the page body, which + * is the exact defect demotion exists to prevent, and it would publish looking fine. + * Neither committed artifact uses one today (verified across all 971 descriptions in the + * OpenAPI document), so the first one to appear stops the build and gets ATX in its doc + * comment. + * + * @param {string} markdown Prose that may contain headings. + * @param {number} offset Levels to add. + * @returns {string} The prose with its headings demoted. + */ +export function demoteHeadings(markdown, offset) { + let fence = null; + /** @type {string[]} */ + const lines = []; + /** Indices of lines that sat inside a fenced block, which is code, not prose. */ + const fenced = new Set(); + markdown.split('\n').forEach((line) => { + const fenceMatch = FENCE_OPEN.exec(line); + if (fence === null) { + if (fenceMatch) { + fence = { + char: fenceMatch[1][0], + length: fenceMatch[1].length, + }; + lines.push(line); + fenced.add(lines.length - 1); + return; + } + } else { + // A *closing* fence carries no info string. Without that anchor a + // ```` ```js ```` line nested inside a ```` ```sh ```` example closes the + // block early, which both demotes the `#` comments inside the example and + // leaves every real heading after it untouched. + const closer = FENCE_CLOSE.exec(line); + if ( + closer && + closer[1][0] === fence.char && + closer[1].length >= fence.length + ) { + fence = null; + } + // Inside a fence, and pushed with a marker the setext scan below reads as + // "not prose": an `=====` in a code example is code. + lines.push(line); + fenced.add(lines.length - 1); + return; + } + const heading = /^(#{1,6})(\s)/.exec(line); + if (!heading) { + lines.push(line); + return; + } + const level = Math.min(6, heading[1].length + offset); + lines.push('#'.repeat(level) + line.slice(heading[1].length)); + }); + + // Checked after the pass so the fence state above decides what is prose. A setext + // underline is a run of `=` or `-` alone on a line, directly under a non-blank one that + // is not itself a heading, a list item, or a table row. + for (let i = 1; i < lines.length; i += 1) { + if (fenced.has(i) || fenced.has(i - 1)) continue; + if (!SETEXT_UNDERLINE.test(lines[i])) continue; + const above = lines[i - 1]; + if (above.trim() === '') continue; + if (/^\s*(#{1,6}\s|[-*+>|]|\d+[.)]\s)/.test(above)) continue; + throw new Error( + `artifact prose carries a setext heading ("${above.trim()}" underlined with ` + + `"${lines[i].trim()}"). This generator demotes ATX headings only, and an ` + + 'undemoted heading in a page body is the defect demotion exists to prevent. ' + + 'Rewrite it as an ATX heading (`## Title`) in the doc comment it comes from.', + ); + } + + return lines.join('\n'); +} + +/** Where the site's content lives, for turning a repo path into a route. */ +const SITE_CONTENT = 'capsule-docs/src/content/docs'; + +/** + * Rewrite links inside artifact prose so they mean the same thing on the site. + * + * The prose in both artifacts is written in its own context — a Rust doc comment or a `clap` + * annotation — and is republished here in another. Two link forms travel badly, and both are + * live in the committed OpenAPI document: + * + * 1. **A repo-relative path to a design document.** `[chunk contract](../../../capsule-docs/…/upload-protocol.md)` + * resolves from the crate source and from nowhere on the site. It has an exact + * equivalent — the Starlight route the same file serves — so it is rewritten, not + * dropped: the reader keeps the reference. + * 2. **A rustdoc intra-doc link.** ``[`revoke_all_signing_bytes`](capsule_core::crypto::revoke::revoke_all_signing_bytes)`` + * is a path rustdoc resolves and no web server does. There is no equivalent, so the + * link is dropped and its text kept. + * + * Absolute URLs, site routes, and anchors are left alone. + * + * The alternative was to fix the annotations in `capsule-server`, which rule 2 would normally + * demand. It is the wrong fix here: those links are correct for rustdoc, which is also a + * published surface, and "correct in the crate, wrong on the site" is a property of + * republishing rather than an error in the source. + * + * @param {string} markdown Prose from an artifact. + * @returns {string} The prose with its links made meaningful on the site. + */ +function rewriteLinks(markdown) { + return markdown.replace( + /(!?\[)([^\]]*)(\]\(\s*)([^)\s]+)(\s*\))/g, + (whole, open, text, mid, target, close) => { + if (/^(?:[a-z][a-z0-9+.-]*:|\/|#)/i.test(target)) return whole; + const normalized = target.replace(/^(?:\.\.\/)+/, ''); + if ( + normalized.startsWith(`${SITE_CONTENT}/`) && + normalized.endsWith('.md') + ) { + const route = normalized + .slice(`${SITE_CONTENT}/`.length) + .replace(/(?:\/index)?\.md$/, ''); + return `${open}${text}${mid}/${route}/${close}`; + } + // No equivalent: keep the words, drop the link. + return text; + }, + ); +} + +/** + * Prepare artifact prose for a one-line context: rewrite its links, flatten it to a single + * line. Deliberately does **not** escape — see [`escapeCell`]. + * + * @param {string} text + * @returns {string} + */ +function prose(text) { + return rewriteLinks(text) + .replace(/\s*\n\s*/g, ' ') + .trim(); +} + +/** + * Escape one **finished** table cell — after every fragment that composes it has been + * assembled, never fragment by fragment. + * + * That ordering is the whole point. A cell is built from several sources — the help text, + * then `Values: …`, `Default: …`, `Example: …` appended after it — and escaping only the + * first leaves the others raw. A default of `a|b` then opens a column of its own and shifts + * every cell to its right, which is exactly the defect this function exists to prevent and + * exactly the one a per-fragment escape reintroduces. + * + * Two characters are escaped, and only these two: + * + * - `|`, which opens a column. GFM requires the escape inside a code span too, and renders + * it as a bare pipe, so `` `a\|b` `` shows the pipe the artifact meant. + * - `<`, because Markdown passes raw HTML through. `` — the most likely idiom in help + * text for a command line — parses as a tag and vanishes, taking everything up to the + * next `>` with it if it never closes. + * + * **Artifact prose is trusted Markdown.** It comes from Rust doc comments and `clap` + * annotations in this repository, reviewed like any other source, so emphasis, links, and + * inline code in it are the author's intent and are passed through rather than sanitized. + * These two escapes are not a security boundary; they repair characters whose meaning + * *changes* when prose written for a doc comment is republished inside a table. An unclosed + * `<` outside a code span in a doc comment is the author's bug, and it is fixed in the doc + * comment. + * + * Never apply this to generator-authored markup: the `
` in a response body cell is + * markup this file wrote and means to keep. + * + * @param {string} text A fully assembled cell. + * @returns {string} + */ +function escapeCell(text) { + return text.replace(/\|/g, '\\|').replace(/} + */ +export function readCliSurface(root) { + const surface = readArtifact(root, CLI_SURFACE); + if (surface?.schema !== CLI_SCHEMA) { + throw new Error( + `${CLI_SURFACE} declares schema ${surface?.schema}, and this generator was ` + + `written against schema ${CLI_SCHEMA}. Update ${GENERATED_BY} rather than ` + + 'rendering a document it does not understand.', + ); + } + if (typeof surface.name !== 'string' || surface.name === '') { + throw new Error( + `${CLI_SURFACE} carries no command name. An emitter regression producing a ` + + 'well-formed but empty document would otherwise publish a page with an empty ' + + 'heading and a stable badge — the confidently-wrong page this generator ' + + 'exists to make impossible.', + ); + } + if ( + !Array.isArray(surface.subcommands) || + surface.subcommands.length === 0 + ) { + throw new Error( + `${CLI_SURFACE} describes no subcommands. \`capsule\` has several; a document ` + + 'saying otherwise is a regression in the emitter, not a CLI that shrank.', + ); + } + return surface; +} + +/** + * How an argument is spelled on the command line. + * + * A positional is written as its placeholder, an option by its flags. Only a value-taking + * option gets a placeholder — the description artifact suppresses clap's synthesized one + * for flags, and inventing `--force ` here would document a surface that rejects it. + * + * @param {Record} arg + * @returns {string} + */ +function spell(arg) { + const placeholder = `<${(arg.value_names ?? [arg.id.toUpperCase()]).join('> <')}>`; + if (arg.positional) { + return `${placeholder}${arg.repeatable ? '...' : ''}`; + } + const flags = []; + if (arg.short) flags.push(`-${arg.short}`); + if (arg.long) flags.push(`--${arg.long}`); + const spelled = flags.length > 0 ? flags.join(', ') : arg.id; + return arg.takes_value ? `${spelled} ${placeholder}` : spelled; +} + +/** + * Terminate a sentence that does not terminate itself. + * + * `clap` strips the full stop off a doc comment, so help text arrives unpunctuated and the + * facts appended after it ("Repeatable.", "Values: …") would run straight on from the last + * word of the description. + * + * @param {string} text + * @returns {string} + */ +function sentence(text) { + return /[.!?:]$/.test(text) ? text : `${text}.`; +} + +/** + * One argument's description cell: whether it is required and repeatable, what it does, + * what it accepts, and what it defaults to. + * + * @param {Record} arg + * @returns {string} + */ +function describeArg(arg) { + const parts = []; + if (arg.required) parts.push('**Required.**'); + const help = arg.long_help ?? arg.help; + if (help) parts.push(sentence(prose(help))); + // Not appended when the help already says it, or `capsule cull --pick` reads + // "Flag an asset as a keeper (repeatable). Repeatable." + if (arg.repeatable && !/repeatable/i.test(help ?? '')) { + parts.push('Repeatable.'); + } + if (arg.possible_values?.length) { + parts.push( + `Values: ${arg.possible_values.map((value) => `\`${value.name}\``).join(', ')}.`, + ); + } + if (arg.default_values?.length) { + parts.push( + `Default: ${arg.default_values.map((value) => `\`${value}\``).join(', ')}.`, + ); + } + // One pass, over the assembled cell: a `|` in a default or an enumerated value is as + // capable of opening a column as one in the help text. + return escapeCell(parts.join(' ')) || '—'; +} + +/** + * Collapse runs of blank lines outside fenced blocks, and trim the trailing one. + * + * Section assembly leaves a blank run wherever a table ended one block and a heading opened + * the next. Markdown does not care; a reader diffing two generations does, and so does + * markdownlint on any machine that has built the site. Applied outside fences only, because + * a blank run *inside* an example is part of the example — a whole-document regex would have + * the generator quietly editing the code it is quoting. + * + * @param {string} markdown + * @returns {string} + */ +function tidyBlankLines(markdown) { + let fence = null; + const out = []; + for (const line of markdown.split('\n')) { + if (fence === null) { + const open = FENCE_OPEN.exec(line); + if (open) { + fence = { char: open[1][0], length: open[1].length }; + } else if ( + line.trim() === '' && + out.length > 0 && + out[out.length - 1].trim() === '' + ) { + continue; + } + } else { + const closer = FENCE_CLOSE.exec(line); + if ( + closer && + closer[1][0] === fence.char && + closer[1].length >= fence.length + ) { + fence = null; + } + } + out.push(line); + } + return out.join('\n').trimEnd(); +} + +/** + * A Markdown table, or the empty string when there are no rows — an empty table renders as + * a stray header and says nothing. + * + * @param {string[]} headers + * @param {string[][]} rows + * @returns {string} + */ +function table(headers, rows) { + if (rows.length === 0) return ''; + const lines = [ + `| ${headers.join(' | ')} |`, + `| ${headers.map(() => '---').join(' | ')} |`, + ...rows.map((row) => `| ${row.join(' | ')} |`), + ]; + return `${lines.join('\n')}\n`; +} + +/** + * Render one command and, recursively, its subcommands. + * + * @param {Record} command + * @param {string[]} path Command words leading here, including this command's own name. + * @param {number} level Heading level for this command (2 under the Starlight title). + * @returns {string} + */ +function renderCommand(command, path, level) { + const invocation = path.join(' '); + const args = command.args ?? []; + const positionals = args.filter((arg) => arg.positional); + const options = args.filter((arg) => !arg.positional); + const subcommands = command.subcommands ?? []; + + const usage = [invocation]; + for (const arg of positionals) { + const spelled = spell(arg); + usage.push(arg.required ? spelled : `[${spelled}]`); + } + // Required options are spelled out rather than folded into `[OPTIONS]`. `capsule import` + // requires `--library `, and a usage line that hides it hands the reader a command + // that fails to parse — a reference page that is wrong, which is the one thing this + // pipeline exists to prevent. + for (const option of options.filter((candidate) => candidate.required)) { + usage.push(spell(option)); + } + if (options.some((option) => !option.required)) usage.push('[OPTIONS]'); + if (subcommands.length > 0) usage.push(''); + + const sections = [ + `${'#'.repeat(level)} ${invocation}`, + '', + `\`\`\`text\n${usage.join(' ')}\n\`\`\``, + '', + ]; + + // Demoted relative to this command's own heading, so a doc comment that opens at `#` + // nests under the command it describes instead of outranking it. + const about = command.long_about ?? command.about; + if (about) { + sections.push(demoteHeadings(rewriteLinks(about), level), ''); + } + + const positionalTable = table( + ['Argument', 'Description'], + positionals.map((arg) => [`\`${spell(arg)}\``, describeArg(arg)]), + ); + if (positionalTable) sections.push(positionalTable); + + const optionTable = table( + ['Option', 'Description'], + options.map((arg) => [`\`${spell(arg)}\``, describeArg(arg)]), + ); + if (optionTable) sections.push(optionTable); + + if (subcommands.length > 0) { + sections.push( + table( + ['Command', 'Description'], + subcommands.map((subcommand) => [ + `[\`${[...path, subcommand.name].join(' ')}\`](#${[...path, subcommand.name].join('-')})`, + subcommand.about ? cell(subcommand.about) : '—', + ]), + ), + ); + } + + return [ + sections.filter((section) => section !== '').join('\n\n'), + ...subcommands.map((subcommand) => + renderCommand( + subcommand, + [...path, subcommand.name], + Math.min(6, level + 1), + ), + ), + ].join('\n\n'); +} + +/** + * The generated `/reference/cli/commands/` page. + * + * @param {Record} surface The parsed command tree. + * @returns {string} Markdown, frontmatter included. + */ +export function renderCliPage(surface) { + const page = CLI_PAGES[0]; + return `${[ + '---', + `title: ${yamlString(page.label)}`, + `description: ${yamlString(page.description)}`, + 'status: stable', + '---', + '', + ``, + '', + `Generated from \`${CLI_SURFACE}\`, the committed command tree \`capsule-cli\` emits and`, + '`mise run cli-surface-check` keeps current. To change a description on this page, change', + 'the `clap` annotation it comes from and regenerate — this file is build output.', + '', + tidyBlankLines(renderCommand(surface, [surface.name], 2)), + ].join('\n')}\n`; +} + +/** + * The committed Kynos OpenAPI document. + * + * @param {string} root Repository root. + * @returns {Record} + */ +export function readOpenApiDocument(root) { + const document = readArtifact(root, OPENAPI_DOCUMENT); + const version = String(document?.openapi ?? ''); + if (!version.startsWith(`${OPENAPI_VERSION}.`)) { + throw new Error( + `${OPENAPI_DOCUMENT} declares OpenAPI ${version || '(nothing)'}, and this ` + + `generator renders ${OPENAPI_VERSION}. The served document is pinned to ` + + `${OPENAPI_VERSION} with \`openapi_as(SpecVersion::V3_2)\`; a lower version ` + + 'means the emitter regressed, and rendering it would publish the regression.', + ); + } + if (!document.paths || typeof document.paths !== 'object') { + throw new Error(`${OPENAPI_DOCUMENT} declares no paths.`); + } + assertRenderable(document); + return document; +} + +/** Schema keywords this renderer cannot express. */ +const COMPOSITION_KEYWORDS = ['oneOf', 'allOf', 'anyOf']; + +/** + * Fail on anything in the document this renderer would answer wrongly rather than not at + * all. See the fatal list in the module header for why each is fatal. + * + * @param {Record} document + * @throws {Error} naming the operation or the schema. + */ +function assertRenderable(document) { + for (const [path, item] of Object.entries(document.paths)) { + for (const method of METHODS) { + const operation = item?.[method]; + if (!operation) continue; + const at = `${method.toUpperCase()} ${path}`; + + const carriers = [ + ['request body', operation.requestBody], + ...Object.entries(operation.responses ?? {}).map( + ([status, response]) => [`response ${status}`, response], + ), + ]; + for (const [which, carrier] of carriers) { + const media = Object.keys(carrier?.content ?? {}); + if (media.length > 1) { + throw new Error( + `${at}: its ${which} offers ${media.length} media types ` + + `(${media.sort().join(', ')}), and this generator renders one body ` + + 'per carrier. Rendering it would document the endpoint as ' + + 'accepting only the first. Teach ' + + `${GENERATED_BY} to render every media type before the server ` + + 'starts offering a choice.', + ); + } + } + } + } + + for (const [name, schema] of Object.entries( + document.components?.schemas ?? {}, + )) { + assertNoComposition(name, name, schema); + } +} + +/** + * Refuse `oneOf`/`allOf`/`anyOf` anywhere in a named schema, not only at its root. + * + * Checking the root alone is the shape of the bug it was meant to prevent: a composition one + * level down, in `properties.` or in `items`, falls through `typeOf` to the string + * `object` and renders as a row claiming the field is a plain object. That is a confident + * lie about a union, which is exactly what this check exists to stop, and it passed. + * + * Only *inline* subschemas are walked. A `$ref` is not followed, because its target is a + * named schema that this scan reaches on its own pass — following it would report the same + * composition once per reference, under whichever name happened to be scanned first. + * + * @param {string} name The named schema this subtree belongs to. + * @param {string} at A readable path to the subschema, for the error. + * @param {unknown} schema The subschema. + * @throws {Error} naming the schema and the path within it. + */ +function assertNoComposition(name, at, schema) { + if (!schema || typeof schema !== 'object' || Array.isArray(schema)) return; + if (schema.$ref) return; + + const composed = COMPOSITION_KEYWORDS.filter((word) => schema[word]); + if (composed.length > 0) { + throw new Error( + `schema ${name} composes with ${composed.join(' and ')} at ${at}, which this ` + + 'generator cannot express: it flattens a schema to a property table, ' + + 'and a union or an intersection is not a property table. It would ' + + `render as an empty or a half-true model. Teach ${GENERATED_BY} to ` + + 'render composition before the server starts emitting it.', + ); + } + + for (const [property, subschema] of Object.entries( + schema.properties ?? {}, + )) { + assertNoComposition(name, `${at}.${property}`, subschema); + } + assertNoComposition(name, `${at}[]`, schema.items); + // `additionalProperties` is a schema when it is an object, and `true`/`false` otherwise. + if (typeof schema.additionalProperties === 'object') { + assertNoComposition( + name, + `${at}.additionalProperties`, + schema.additionalProperties, + ); + } +} + +/** + * Bucket every operation in the document into its group, in a stable order. + * + * **Fails on an operation no group claims.** That is the whole value of a hand-curated + * table: a new endpoint family cannot publish under a heading nobody chose, and — the case + * that actually bites — cannot silently fail to publish at all while the build stays green. + * + * @param {Record} document The parsed OpenAPI document. + * @returns {Map, operationId?: string }>>} + * Keyed by group slug, in `API_GROUPS` order, with every group present. + */ +export function bucketOperations(document) { + /** @type {Map} */ + const buckets = new Map(API_GROUPS.map((group) => [group.slug, []])); + + // Sorted rather than taken in document order: JSON object order is an emitter detail, + // and a page whose sections reshuffle when the server's route registration is reordered + // produces a diff nobody can read. + for (const path of Object.keys(document.paths).sort()) { + const group = groupForPath(path); + const item = document.paths[path] ?? {}; + const methods = METHODS.filter((method) => item[method]); + if (!group) { + const named = methods + .map((method) => `${method.toUpperCase()} ${path}`) + .join(', '); + throw new Error( + `no reference group claims ${named || path}. Add its prefix to a group in ` + + 'capsule-docs/scripts/reference-groups.mjs — an endpoint family must not ' + + 'publish under a heading nobody chose, and must not silently fail to publish.', + ); + } + for (const method of methods) { + buckets.get(group.slug).push({ + path, + method: method.toUpperCase(), + operation: item[method], + operationId: item[method].operationId, + }); + } + } + return buckets; +} + +/** + * Render a schema's type as a short string: `string`, `string | null`, `integer[]`, or the + * name of a referenced schema. + * + * @param {Record} schema + * @returns {string} + */ +function typeOf(schema) { + if (!schema || typeof schema !== 'object') return 'unknown'; + if (schema.$ref) return refName(schema.$ref); + if (schema.type === 'array') { + return `${typeOf(schema.items ?? {})}[]`; + } + const type = schema.type; + // OpenAPI 3.1 and later spell nullability as a type union rather than as `nullable`, so + // `type` is an array here and interpolating it directly yields `string,null`. + if (Array.isArray(type)) return type.join(' | '); + if (typeof type === 'string') return type; + if (schema.enum) return 'string'; + return 'object'; +} + +/** + * The schema name a local `$ref` points at. + * + * @param {string} ref + * @returns {string} + */ +function refName(ref) { + return ref.split('/').pop(); +} + +/** + * The set of schema names a page must document, walked from its operations to + * `MAX_SCHEMA_DEPTH`. + * + * Depth-bounded rather than exhaustive so a self-referential or mutually-referential schema + * cannot spin: the current document has no cycle, but a renderer that would hang on one is + * a renderer that fails the day someone adds a tree. + * + * The bound and the cycle guard interact, which is the subtle part — see the comment on + * `seen` below. The rule the page states, and the one implemented here, is: a schema is + * documented when some path reaches it within the bound, whichever path the walk takes + * first. + * + * @param {Record} document + * @param {Array<{ operation: Record }>} operations + * @returns {string[]} Schema names, sorted. + */ +function schemasUsedBy(document, operations) { + /** @type {Map} Schema name -> shallowest depth it was reached at. */ + const seen = new Map(); + + const visit = (schema, depth) => { + if (!schema || typeof schema !== 'object' || depth > MAX_SCHEMA_DEPTH) + return; + if (Array.isArray(schema)) { + for (const entry of schema) visit(entry, depth); + return; + } + if (schema.$ref) { + const name = refName(schema.$ref); + // Keyed on the *shallowest* depth this name has been reached at, not on having + // been seen at all. A plain visited set makes the answer depend on traversal + // order: reach `SyncEntry` at depth 2 first and its children are cut by the + // bound, and the later path that reaches it at depth 1 — where its children are + // in range — is then refused as already-seen. `WireBlobRole` on + // `/reference/api/sync/` was documented or not according to which operation the + // walk happened to read first. Re-expanding on a shallower arrival still + // terminates: a name can only improve `MAX_SCHEMA_DEPTH + 1` times, and a cycle + // never arrives shallower twice. + if (seen.has(name) && seen.get(name) <= depth) return; + seen.set(name, depth); + visit(document.components?.schemas?.[name], depth + 1); + return; + } + for (const value of Object.values(schema)) visit(value, depth); + }; + + for (const { operation } of operations) { + visit(operation.requestBody ?? {}, 0); + visit(operation.responses ?? {}, 0); + visit(operation.parameters ?? [], 0); + } + return [...seen.keys()].sort(); +} + +/** + * The media type and schema of a request or response body, or null when it carries none. + * + * @param {Record | undefined} carrier + * @returns {{ mediaType: string, schema: Record } | null} + */ +function bodyOf(carrier) { + const content = carrier?.content; + if (!content) return null; + // Exactly one, or none: `assertRenderable` has already refused a carrier offering a + // choice, so `sort()[0]` is the only entry rather than an arbitrary pick. + const mediaType = Object.keys(content).sort()[0]; + if (!mediaType) return null; + return { mediaType, schema: content[mediaType]?.schema ?? {} }; +} + +/** + * A schema reference rendered as a link into this page's appendix, when the appendix + * documents it, and as bare code when it does not. + * + * @param {string[]} documented Schema names the page's appendix carries. + * @param {Record} schema + * @returns {string} + */ +function schemaLink(documented, schema) { + const rendered = typeOf(schema); + const bare = rendered.replace(/\[\]$/, ''); + // A nullable type renders as `string | null`, and every use of this is a table cell, so + // the separator has to be escaped or it opens a fourth column and shifts the row. + // GFM requires the escape inside a code span too, and renders it as a bare pipe. + const shown = rendered.replace(/\|/g, '\\|'); + return documented.includes(bare) + ? `[\`${shown}\`](#${bare.toLowerCase()})` + : `\`${shown}\``; +} + +/** + * Render one operation. + * + * @param {{ path: string, method: string, operation: Record }} entry + * @param {string[]} documented Schema names the page's appendix carries. + * @returns {string} + */ +function renderOperation({ path, method, operation }, documented) { + const sections = [`## ${method} ${path}`]; + + if (operation.summary) { + sections.push(demoteHeadings(rewriteLinks(operation.summary), 2)); + } + if (operation.description) { + // Demoted by 3: an operation is an h2, so a description opening at `#` becomes an + // h4 under it rather than a second page title. + sections.push(demoteHeadings(rewriteLinks(operation.description), 3)); + } + + const security = operation.security; + if (Array.isArray(security) && security.length > 0) { + const schemes = security + .flatMap((requirement) => Object.keys(requirement)) + .sort(); + sections.push( + `**Authentication:** required — ${schemes.map((scheme) => `\`${scheme}\``).join(', ')}.`, + ); + } else { + sections.push('**Authentication:** none.'); + } + + const parameters = operation.parameters ?? []; + if (parameters.length > 0) { + sections.push( + table( + ['Parameter', 'In', 'Type', 'Description'], + [...parameters] + .sort( + (a, b) => + a.in.localeCompare(b.in) || + a.name.localeCompare(b.name), + ) + .map((parameter) => [ + `\`${parameter.name}\``, + `\`${parameter.in}\``, + schemaLink(documented, parameter.schema ?? {}), + escapeCell( + [ + parameter.required ? '**Required.**' : '', + parameter.description + ? sentence(prose(parameter.description)) + : '', + parameter.example === undefined + ? '' + : `Example: \`${parameter.example}\`.`, + ] + .filter(Boolean) + .join(' '), + ) || '—', + ]), + ), + ); + } + + const requestBody = bodyOf(operation.requestBody); + if (requestBody) { + sections.push( + `**Request body** (${operation.requestBody.required ? 'required' : 'optional'}, ` + + `\`${requestBody.mediaType}\`): ${schemaLink(documented, requestBody.schema)}`, + ); + } + + const responses = Object.entries(operation.responses ?? {}).sort( + ([a], [b]) => Number(a) - Number(b) || a.localeCompare(b), + ); + if (responses.length > 0) { + sections.push( + table( + ['Status', 'Body', 'Description'], + responses.map(([status, response]) => { + const body = bodyOf(response); + const headers = Object.keys(response.headers ?? {}).sort(); + return [ + `\`${status}\``, + body + ? `${schemaLink(documented, body.schema)}
\`${body.mediaType}\`` + : '—', + escapeCell( + [ + response.description + ? sentence(prose(response.description)) + : '', + headers.length > 0 + ? `Headers: ${headers.map((header) => `\`${header}\``).join(', ')}.` + : '', + ] + .filter(Boolean) + .join(' '), + ) || '—', + ]; + }), + ), + ); + } + + return sections.filter(Boolean).join('\n\n'); +} + +/** + * Render one schema as an appendix entry. + * + * @param {string} name + * @param {Record} document The document, for the schema's own definition. + * @param {string[]} documented Every schema name this page's appendix carries. + * @returns {string} + */ +function renderSchema(name, document, documented) { + const schema = document.components?.schemas?.[name] ?? {}; + const sections = [`### ${name}`]; + + // Through `rewriteLinks` like every other prose site: a model's own doc comment is as + // free to cite a design document by repo path, or an item by its rustdoc path, as a + // handler's is. `TokenResponse` is one of several that do. + if (schema.description) { + sections.push(demoteHeadings(rewriteLinks(schema.description), 3)); + } + + if (schema.enum) { + sections.push( + `One of: ${schema.enum.map((value) => `\`${value}\``).join(', ')}.`, + ); + return sections.join('\n\n'); + } + + const required = new Set(schema.required ?? []); + const properties = Object.entries(schema.properties ?? {}); + if (properties.length === 0) { + sections.push(`Type: \`${typeOf(schema)}\`.`); + return sections.join('\n\n'); + } + + sections.push( + table( + ['Field', 'Type', 'Description'], + properties.map(([field, property]) => [ + `\`${field}\``, + schemaLink(documented, property), + escapeCell( + [ + required.has(field) ? '**Required.**' : '', + property.description + ? sentence(prose(property.description)) + : '', + ] + .filter(Boolean) + .join(' '), + ) || '—', + ]), + ), + ); + return sections.join('\n\n'); +} + +/** + * The generated `/reference/api//` page. + * + * @param {import('./reference-groups.mjs').ApiGroup} group + * @param {Array<{ path: string, method: string, operation: Record }>} operations + * @param {Record} document + * @returns {string} Markdown, frontmatter included. + */ +export function renderApiPage(group, operations, document) { + const documented = schemasUsedBy(document, operations); + + const head = [ + '---', + `title: ${yamlString(group.label)}`, + `description: ${yamlString(group.description)}`, + 'status: stable', + '---', + '', + ``, + '', + `Generated from \`${OPENAPI_DOCUMENT}\`, the OpenAPI ${OPENAPI_VERSION} document`, + '`capsule-server` emits and `mise run openapi-check-kynos` keeps current. To change a', + 'description on this page, change the annotation on the handler or model it comes from', + 'and regenerate — this file is build output. The auth model, error contract, and', + 'conventions common to every endpoint are on the [REST API overview](/reference/api/).', + ].join('\n'); + + const body = operations + .map((entry) => renderOperation(entry, documented)) + .join('\n\n'); + + const appendix = + documented.length === 0 + ? '' + : [ + '## Schemas', + '', + 'The models these endpoints carry. A field whose type names another model links', + 'to it when this page documents that model, which it does when some path from', + `an operation reaches it within ${MAX_SCHEMA_DEPTH} references. A model only`, + 'ever reached deeper than that is named without being expanded.', + '', + documented + .map((name) => renderSchema(name, document, documented)) + .join('\n\n'), + ].join('\n'); + + return `${tidyBlankLines([head, body, appendix].filter(Boolean).join('\n\n'))}\n`; +} + +/** + * Emit every generated reference page under `root`. + * + * Artifacts are read and validated first, before anything is written, so a missing or + * unparseable one leaves no half-written section behind. + * + * @param {string} root Repository root. + * @returns {string[]} Repo-relative paths written, in a stable order. + */ +export function generate(root) { + const surface = readCliSurface(root); + const document = readOpenApiDocument(root); + const buckets = bucketOperations(document); + + /** @type {Array<{ path: string, body: string }>} */ + const pages = [ + { + path: `${CLI_OUT}/${CLI_PAGES[0].slug}.md`, + body: renderCliPage(surface), + }, + ...API_GROUPS.map((group) => ({ + path: `${API_OUT}/${group.slug}.md`, + body: renderApiPage(group, buckets.get(group.slug) ?? [], document), + })), + ]; + + // Cleared, not merged into. A page this run no longer emits — a group renamed, a surface + // dropped — would otherwise stay on disk, and because the directory is gitignored + // `git status` never shows it: Astro would keep routing, indexing, and link-validating a + // page no artifact describes, on this machine and on no CI runner. Only the generated + // directories, so the hand-written overviews beside them survive. + for (const directory of [CLI_OUT, API_OUT]) { + rmSync(join(root, directory), { recursive: true, force: true }); + } + + for (const { path, body } of pages) { + mkdirSync(dirname(join(root, path)), { recursive: true }); + writeFileSync(join(root, path), body); + } + return pages.map(({ path }) => path); +} + +function main() { + // scripts/ -> capsule-docs/ -> repo root + const root = resolve(dirname(fileURLToPath(import.meta.url)), '..', '..'); + const written = generate(root); + process.stdout.write( + `gen-reference: wrote ${written.length} page(s)\n${written.map((path) => ` ${path}`).join('\n')}\n`, + ); +} + +// Run only when invoked as a script, so the test file can import the renderers. +if ( + process.argv[1] && + resolve(process.argv[1]) === fileURLToPath(import.meta.url) +) { + main(); +} diff --git a/capsule-docs/scripts/gen-reference.test.mjs b/capsule-docs/scripts/gen-reference.test.mjs new file mode 100644 index 00000000..7628c65d --- /dev/null +++ b/capsule-docs/scripts/gen-reference.test.mjs @@ -0,0 +1,1064 @@ +import { + copyFileSync, + existsSync, + mkdirSync, + mkdtempSync, + readFileSync, + rmSync, + writeFileSync, +} from 'node:fs'; +import { tmpdir } from 'node:os'; +import { dirname, join, resolve } from 'node:path'; +import { fileURLToPath } from 'node:url'; +import { afterEach, beforeEach, describe, expect, it } from 'vitest'; +import { + bucketOperations, + CLI_SURFACE, + demoteHeadings, + generate, + OPENAPI_DOCUMENT, + readCliSurface, + readOpenApiDocument, + renderApiPage, + renderCliPage, +} from './gen-reference.mjs'; +import { headingAnchors } from './lib/markdown.mjs'; +import { API_GROUPS, groupForPath } from './reference-groups.mjs'; + +/** + * A repo-shaped temporary root: the two description artifacts at the paths the generator + * reads, and the content directory it writes into. + * + * Fixtures rather than the committed artifacts wherever the assertion is about the + * *generator*. The committed ones are used only where the assertion is about this + * repository — that every operation it declares reaches a page. + */ +function fixtureRoot() { + const root = mkdtempSync(join(tmpdir(), 'gen-reference-')); + mkdirSync(join(root, 'capsule-cli'), { recursive: true }); + mkdirSync(join(root, 'capsule-server'), { recursive: true }); + mkdirSync(join(root, 'capsule-docs/src/content/docs/reference'), { + recursive: true, + }); + return root; +} + +const MINIMAL_CLI = { + schema: 1, + name: 'capsule', + about: 'A command line interface for Capsule', + subcommands: [ + { + name: 'import', + about: 'Import files into a local Capsule library', + args: [ + { + id: 'paths', + positional: true, + required: true, + repeatable: true, + takes_value: true, + value_names: ['PATH'], + help: 'Source file or directory to import', + }, + { + id: 'provider', + long: 'provider', + positional: false, + required: false, + repeatable: false, + takes_value: true, + value_names: ['PROVIDER'], + possible_values: [ + { name: 'takeout', help: 'A Takeout export' }, + ], + help: 'Read the source as an export from this service', + }, + { + id: 'force', + long: 'force', + short: 'f', + positional: false, + required: false, + repeatable: false, + takes_value: false, + help: 'Re-import files even if they already exist', + }, + ], + }, + ], +}; + +/** HTTP methods a path item may carry, per OpenAPI. */ +const METHODS = new Set([ + 'get', + 'put', + 'post', + 'delete', + 'options', + 'head', + 'patch', + 'trace', +]); + +let root; + +beforeEach(() => { + root = fixtureRoot(); +}); + +afterEach(() => { + rmSync(root, { recursive: true, force: true }); +}); + +function writeCli(surface) { + writeFileSync( + join(root, CLI_SURFACE), + `${JSON.stringify(surface, null, 2)}\n`, + ); +} + +function writeOpenApi(document) { + mkdirSync(join(root, 'capsule-server'), { recursive: true }); + writeFileSync( + join(root, OPENAPI_DOCUMENT), + `${JSON.stringify(document, null, 2)}\n`, + ); +} + +const MINIMAL_OPENAPI = { + openapi: '3.2.0', + info: { title: 'API', version: '0.0.0' }, + paths: { + '/v1/version': { + get: { + summary: 'The protocol range this server speaks.', + operationId: 'version', + responses: { + 200: { + description: 'The range.', + content: { + 'application/json': { + schema: { + $ref: '#/components/schemas/VersionResponse', + }, + }, + }, + }, + }, + }, + }, + '/v1/auth/login': { + post: { + summary: 'Exchange an email and password for a session.', + description: + 'Leading prose.\n\n# Two statuses, because there are two outcomes\n\nMore prose.', + operationId: 'login_user', + security: [{ bearer: [] }], + requestBody: { + required: true, + content: { + 'application/json': { + schema: { + $ref: '#/components/schemas/LoginRequest', + }, + }, + }, + }, + responses: { + 401: { description: 'Invalid credentials' }, + 200: { + description: 'A session was opened.', + headers: { + 'WWW-Authenticate': { schema: { type: 'string' } }, + }, + content: { + 'application/json': { + schema: { + $ref: '#/components/schemas/TokenResponse', + }, + }, + }, + }, + }, + }, + }, + '/v1/albums/{album_id}/ops': { + post: { + summary: 'Apply a lifecycle write.', + operationId: 'album_ops', + parameters: [ + { + name: 'album_id', + in: 'path', + description: "The album's identifier.", + required: true, + schema: { type: 'string' }, + }, + ], + responses: { 204: { description: 'Applied.' } }, + }, + }, + }, + components: { + securitySchemes: { + bearer: { type: 'http', scheme: 'bearer', bearerFormat: 'JWT' }, + }, + schemas: { + VersionResponse: { + type: 'object', + title: 'VersionResponse', + required: ['min'], + properties: { + min: { type: 'integer', description: 'Lowest supported.' }, + }, + }, + LoginRequest: { + type: 'object', + title: 'LoginRequest', + required: ['email'], + properties: { + email: { + type: 'string', + description: 'The account email.', + }, + device_id: { + type: ['string', 'null'], + description: 'Advisory. Gates nothing | really.', + }, + }, + }, + TokenResponse: { + type: 'object', + title: 'TokenResponse', + properties: { + access: { type: 'string' }, + kind: { $ref: '#/components/schemas/TokenKind' }, + }, + }, + TokenKind: { + type: 'string', + enum: ['bearer'], + description: 'What the token is.', + }, + }, + }, +}; + +describe('demoteHeadings', () => { + // `openapi.json` really does carry `# Two statuses, because there are two outcomes` + // inside an operation description. Rendered as-is it injects a second H1 into a page + // whose H1 is the Starlight title, and breaks the document outline for a screen reader. + it('demotes an ATX heading by the offset', () => { + expect(demoteHeadings('# Why it signs you in\n', 3)).toBe( + '#### Why it signs you in\n', + ); + expect(demoteHeadings('## Second\n', 3)).toBe('##### Second\n'); + }); + + it('clamps at h6 rather than emitting a run of seven hashes', () => { + expect(demoteHeadings('##### Deep\n', 3)).toBe('###### Deep\n'); + }); + + it('leaves prose and a hash that is not a heading alone', () => { + expect(demoteHeadings('a #tag and #hash\n', 2)).toBe( + 'a #tag and #hash\n', + ); + expect(demoteHeadings('body text\n', 2)).toBe('body text\n'); + }); + + // A closing fence carries no info string. Treating any ``` run as a closer ends the + // block at the nested opener, which both demotes the example's comments and leaves + // every real heading after it untouched — two failures from one input. + it('does not let a nested fence with an info string close the block', () => { + const source = ['```sh', '# a', '```js', '# b', '```', '# c'].join( + '\n', + ); + expect(demoteHeadings(source, 2)).toBe( + ['```sh', '# a', '```js', '# b', '```', '### c'].join('\n'), + ); + }); + + // Leaving one alone silently is worse than not supporting it: an undemoted h1 in the + // page body is the exact defect demotion exists to prevent, and it would publish + // looking fine. + it('refuses a setext heading rather than silently leaving it undemoted', () => { + expect(() => demoteHeadings('Title\n=====\n', 2)).toThrow(/setext/i); + expect(() => demoteHeadings('Title\n-----\n', 2)).toThrow(/Title/); + }); + + // CommonMark's underline is `= +` / `- +`, not two or more. Requiring two let the + // single-character form through the check that exists to catch it — the defect arriving + // through the guard against the defect. + it('refuses a single-character setext underline', () => { + expect(() => demoteHeadings('Title\n-\n', 2)).toThrow(/setext/i); + expect(() => demoteHeadings('Title\n=\n', 2)).toThrow(/setext/i); + }); + + it('refuses an indented or trailing-spaced single-character underline', () => { + expect(() => demoteHeadings('Title\n -\n', 2)).toThrow(/setext/i); + expect(() => demoteHeadings('Title\n= \n', 2)).toThrow(/setext/i); + }); + + it('does not mistake a thematic break or a table for a setext underline', () => { + expect(() => demoteHeadings('para\n\n---\n', 2)).not.toThrow(); + expect(() => demoteHeadings('| a |\n| --- |\n', 2)).not.toThrow(); + expect(() => demoteHeadings('- item\n---\n', 2)).not.toThrow(); + }); + + it('does not mistake an underline inside a fenced example for one', () => { + expect(() => + demoteHeadings('```text\nTitle\n=====\n```\n', 2), + ).not.toThrow(); + }); + + it('does not demote a hash inside a fenced block', () => { + const source = ['```sh', '# not a heading', '```', '# heading'].join( + '\n', + ); + expect(demoteHeadings(source, 2)).toBe( + ['```sh', '# not a heading', '```', '### heading'].join('\n'), + ); + }); +}); + +describe('readCliSurface', () => { + it('names the missing artifact rather than emitting a stub page', () => { + expect(() => readCliSurface(root)).toThrow( + /capsule-cli\/cli-surface\.json/, + ); + }); + + // A stub would be the "confidently wrong" page `developer-docs.md` exists to prevent: + // it publishes, it looks like reference, and it documents nothing. + it('refuses a schema version it was not written against', () => { + writeCli({ ...MINIMAL_CLI, schema: 2 }); + expect(() => readCliSurface(root)).toThrow(/schema/i); + }); + + // A well-formed but empty document used to render an empty h2, an empty usage fence, and + // a `status: stable` badge — a page that builds green and documents nothing. + it('refuses a document that parses but describes nothing', () => { + writeCli({ schema: 1 }); + expect(() => readCliSurface(root)).toThrow(/no command name/); + writeCli({ schema: 1, name: 'capsule', subcommands: [] }); + expect(() => readCliSurface(root)).toThrow(/no subcommands/); + }); + + it('refuses an unparseable artifact', () => { + writeFileSync(join(root, CLI_SURFACE), '{ not json'); + expect(() => readCliSurface(root)).toThrow(/cli-surface\.json/); + }); +}); + +describe('renderCliPage', () => { + it('renders every command in the tree', () => { + const page = renderCliPage(MINIMAL_CLI); + expect(page).toContain('## capsule'); + expect(page).toContain('### capsule import'); + expect(page).toContain('Import files into a local Capsule library'); + }); + + it('opens with frontmatter carrying a status the schema accepts', () => { + const page = renderCliPage(MINIMAL_CLI); + expect(page.startsWith('---\n')).toBe(true); + expect(page).toMatch(/^status: stable$/m); + expect(page).toMatch(/^title: "Commands"$/m); + }); + + // A generated page is linted like any other on a machine that has built the site, and + // a run of blank lines is the shape section assembly leaves behind. + it('ends with exactly one newline and no run of blank lines', () => { + const page = renderCliPage(MINIMAL_CLI); + expect(page.endsWith('\n')).toBe(true); + expect(page.endsWith('\n\n')).toBe(false); + expect(page).not.toMatch(/\n{3,}/); + }); + + it('says it is generated, and by what', () => { + expect(renderCliPage(MINIMAL_CLI)).toContain('gen-reference.mjs'); + }); + + // The page body must not contain an h1: Starlight renders the frontmatter title as the + // page's only h1, and a second one breaks the outline. + it('emits no h1 in the body', () => { + const body = renderCliPage(MINIMAL_CLI) + .split('\n---\n') + .slice(1) + .join('\n---\n'); + expect(body.split('\n').filter((l) => /^# /.test(l))).toEqual([]); + }); + + it('spells a positional, a value-taking option, and a flag differently', () => { + const page = renderCliPage(MINIMAL_CLI); + expect(page).toContain('`...`'); + expect(page).toContain('`--provider `'); + expect(page).toContain('`-f, --force`'); + }); + + // A usage line that folds a required option into `[OPTIONS]` hands the reader a command + // that fails to parse. `capsule import` really does require `--library `. + it('spells required options in the usage line rather than hiding them', () => { + const surface = structuredClone(MINIMAL_CLI); + surface.subcommands[0].args.push({ + id: 'library', + long: 'library', + positional: false, + required: true, + repeatable: false, + takes_value: true, + value_names: ['PATH'], + help: 'Path to the library', + }); + expect(renderCliPage(surface)).toContain( + 'capsule import ... --library [OPTIONS]', + ); + }); + + it('renders the usage line from the argument surface', () => { + expect(renderCliPage(MINIMAL_CLI)).toContain( + 'capsule import ... [OPTIONS]', + ); + }); + + // Markdown passes raw HTML through, so an unescaped placeholder disappears from the + // rendered page — and an unclosed one takes the rest of the cell with it. + it('escapes an angle bracket in help text', () => { + const surface = structuredClone(MINIMAL_CLI); + surface.subcommands[0].args[2].help = 'pass a here'; + const page = renderCliPage(surface); + expect(page).toContain('pass a <token> here'); + expect(page).not.toContain('pass a here'); + }); + + // The defect a per-fragment escape reintroduces: the help text is escaped, the fragments + // appended after it are not, and a `|` in a default opens a column of its own. + it('escapes a pipe in a default value and in an enumerated value', () => { + const surface = structuredClone(MINIMAL_CLI); + surface.subcommands[0].args[1].possible_values = [{ name: 'x|y' }]; + surface.subcommands[0].args[1].default_values = ['a|b']; + const page = renderCliPage(surface); + expect(page).toContain('Values: `x\\|y`.'); + expect(page).toContain('Default: `a\\|b`.'); + expect(page).not.toContain('`x|y`'); + expect(page).not.toContain('`a|b`'); + }); + + it('keeps every table row at the width of its header', () => { + const surface = structuredClone(MINIMAL_CLI); + surface.subcommands[0].args[1].default_values = ['a|b']; + surface.subcommands[0].args[2].help = 'takes a | and a '; + for (const line of renderCliPage(surface).split('\n')) { + if (!line.startsWith('|')) continue; + const columns = line.replace(/\\\|/g, '').split('|').length; + expect(columns).toBe(4); + } + }); + + it('does not append "Repeatable." when the help already says it', () => { + const surface = structuredClone(MINIMAL_CLI); + surface.subcommands[0].args[0].help = + 'Flag an asset as a keeper (repeatable)'; + const page = renderCliPage(surface); + expect(page).toContain('(repeatable).'); + expect(page).not.toContain('(repeatable). Repeatable.'); + }); + + // The anchors are hand-built by joining command words, and the headings are slugged by + // Starlight. Pinning them to one slugger here catches a divergence in a unit test rather + // than in a link-validator failure at build time. + it('emits only anchors its own headings answer', () => { + const page = renderCliPage(MINIMAL_CLI); + const anchors = headingAnchors(page); + const linked = [...page.matchAll(/\]\(#([^)]+)\)/g)].map( + (match) => match[1], + ); + expect(linked.length).toBeGreaterThan(0); + for (const anchor of linked) { + expect(anchors.has(anchor)).toBe(true); + } + }); + + it('lists an enumerated option value', () => { + expect(renderCliPage(MINIMAL_CLI)).toContain('`takeout`'); + }); + + // `clap` strips the full stop off a doc comment, so without this the appended facts run + // straight on: "…folded into the imported assets Values: `takeout`." + it('terminates help text before appending the facts after it', () => { + const page = renderCliPage(MINIMAL_CLI); + expect(page).toContain( + 'Read the source as an export from this service. Values: `takeout`.', + ); + expect(page).toContain( + '**Required.** Source file or directory to import. Repeatable.', + ); + }); + + it('escapes a pipe so it cannot break out of a table cell', () => { + const page = renderCliPage({ + schema: 1, + name: 'capsule', + subcommands: [ + { + name: 'x', + args: [ + { + id: 'p', + long: 'p', + positional: false, + required: false, + repeatable: false, + takes_value: false, + help: 'reads a | b', + }, + ], + }, + ], + }); + expect(page).toContain('reads a \\| b'); + }); +}); + +describe('generate', () => { + it('is deterministic: two runs produce byte-identical pages', () => { + writeCli(MINIMAL_CLI); + writeOpenApi(MINIMAL_OPENAPI); + const first = generate(root).map((path) => + readFileSync(join(root, path), 'utf8'), + ); + const second = generate(root).map((path) => + readFileSync(join(root, path), 'utf8'), + ); + expect(second).toEqual(first); + expect(first.length).toBeGreaterThan(0); + }); + + it('fails, writing nothing, when an artifact is missing', () => { + expect(() => generate(root)).toThrow(/cli-surface\.json/); + expect( + existsSync( + join( + root, + 'capsule-docs/src/content/docs/reference/cli/commands.md', + ), + ), + ).toBe(false); + }); + + // A page a later run no longer emits stays on disk otherwise, and the directory is + // gitignored, so nothing ever shows it: Astro keeps routing and indexing a page no + // artifact describes, on this machine and on no CI runner. + it('clears output it no longer emits', () => { + writeCli(MINIMAL_CLI); + writeOpenApi(MINIMAL_OPENAPI); + generate(root); + const orphan = join( + root, + 'capsule-docs/src/content/docs/reference/api/retired-group.md', + ); + writeFileSync(orphan, '---\ntitle: Gone\nstatus: stable\n---\n'); + generate(root); + expect(existsSync(orphan)).toBe(false); + }); + + // The overviews are siblings of the generated directories precisely so that clearing + // one cannot take a hand-written page with it. + it('does not clear the hand-written overview beside the generated directory', () => { + writeCli(MINIMAL_CLI); + writeOpenApi(MINIMAL_OPENAPI); + const overview = join( + root, + 'capsule-docs/src/content/docs/reference/cli.md', + ); + writeFileSync(overview, '---\ntitle: CLI\nstatus: draft\n---\n'); + generate(root); + expect(existsSync(overview)).toBe(true); + }); +}); + +describe('link rewriting in artifact prose', () => { + // Both forms are live in the committed OpenAPI document. Rendered unchanged they are + // three broken links that fail `starlight-links-validator` and the docs build with it. + it('rewrites a repo-relative design-doc path to its site route', () => { + const document = structuredClone(MINIMAL_OPENAPI); + document.paths['/v1/auth/login'].post.description = + 'Every rule the [chunk contract](../../../capsule-docs/src/content/docs/design/import/upload-protocol.md) fixes.'; + const rendered = renderApiPage( + API_GROUPS.find((entry) => entry.slug === 'auth'), + bucketOperations(document).get('auth'), + document, + ); + expect(rendered).toContain( + '[chunk contract](/design/import/upload-protocol/)', + ); + }); + + it('drops a rustdoc intra-doc link and keeps its text', () => { + const document = structuredClone(MINIMAL_OPENAPI); + document.paths['/v1/auth/login'].post.description = + 'A signature over [`revoke_all_signing_bytes`](capsule_core::crypto::revoke::revoke_all_signing_bytes), in CBOR.'; + const rendered = renderApiPage( + API_GROUPS.find((entry) => entry.slug === 'auth'), + bucketOperations(document).get('auth'), + document, + ); + expect(rendered).toContain( + 'A signature over `revoke_all_signing_bytes`, in CBOR.', + ); + expect(rendered).not.toContain('capsule_core::crypto::revoke'); + }); + + // A model's own doc comment is as free to cite a design document by repo path, or an + // item by its rustdoc path, as a handler's is — and the schema appendix is a separate + // render path that had to be wired up for it. + it('rewrites links in a schema-level description too', () => { + const document = structuredClone(MINIMAL_OPENAPI); + document.components.schemas.LoginRequest.description = + 'Shaped by the [chunk contract](../../../capsule-docs/src/content/docs/design/import/upload-protocol.md) and signed with [`revoke_all_signing_bytes`](capsule_core::crypto::revoke::revoke_all_signing_bytes).'; + const rendered = renderApiPage( + API_GROUPS.find((entry) => entry.slug === 'auth'), + bucketOperations(document).get('auth'), + document, + ); + expect(rendered).toContain( + '[chunk contract](/design/import/upload-protocol/)', + ); + expect(rendered).toContain('signed with `revoke_all_signing_bytes`.'); + expect(rendered).not.toContain('capsule_core::crypto::revoke'); + }); + + it('leaves an absolute URL, a site route, and an anchor alone', () => { + const document = structuredClone(MINIMAL_OPENAPI); + document.paths['/v1/auth/login'].post.description = + 'See [a](https://example.invalid/x), [b](/design/i18n/), and [c](#later).'; + const rendered = renderApiPage( + API_GROUPS.find((entry) => entry.slug === 'auth'), + bucketOperations(document).get('auth'), + document, + ); + expect(rendered).toContain('[a](https://example.invalid/x)'); + expect(rendered).toContain('[b](/design/i18n/)'); + expect(rendered).toContain('[c](#later)'); + }); +}); + +describe('groupForPath', () => { + it('matches the longest prefix, not the first declared', () => { + expect(groupForPath('/v1/auth/login')?.slug).toBe('auth'); + expect(groupForPath('/v1/albums/{album_id}/ops')?.slug).toBe('albums'); + expect(groupForPath('/s/{opaque_id}/blob/{hash}')?.slug).toBe('shares'); + expect(groupForPath('/d/{opaque_id}')?.slug).toBe('drops'); + }); + + it('returns null for a path no group claims', () => { + expect(groupForPath('/v1/search')).toBe(null); + }); +}); + +describe('bucketOperations', () => { + // The gate that keeps the hand-curated navigation honest: a new endpoint family cannot + // publish unlisted, and cannot silently not publish at all. + it('fails, naming the operation, when an endpoint matches no group', () => { + const document = structuredClone(MINIMAL_OPENAPI); + document.paths['/v1/search'] = { + get: { summary: 'Search', operationId: 'search', responses: {} }, + }; + expect(() => bucketOperations(document)).toThrow(/GET \/v1\/search/); + expect(() => bucketOperations(document)).toThrow( + /reference-groups\.mjs/, + ); + }); + + it('buckets every operation exactly once', () => { + const buckets = bucketOperations(MINIMAL_OPENAPI); + const total = [...buckets.values()].reduce( + (sum, operations) => sum + operations.length, + 0, + ); + expect(total).toBe(3); + expect(buckets.get('auth')?.[0].operationId).toBe('login_user'); + }); + + it('orders operations within a group by path, then by method', () => { + const document = structuredClone(MINIMAL_OPENAPI); + document.paths['/v1/auth/aaa'] = { + post: { summary: 'a', operationId: 'a', responses: {} }, + get: { summary: 'b', operationId: 'b', responses: {} }, + }; + const auth = bucketOperations(document).get('auth'); + expect( + auth.map((operation) => `${operation.method} ${operation.path}`), + ).toEqual([ + 'GET /v1/auth/aaa', + 'POST /v1/auth/aaa', + 'POST /v1/auth/login', + ]); + }); +}); + +describe('readOpenApiDocument', () => { + it('names the missing artifact rather than emitting a stub page', () => { + expect(() => readOpenApiDocument(root)).toThrow( + /capsule-server\/openapi\.json/, + ); + }); + + it('refuses a document that is not OpenAPI 3.2', () => { + writeOpenApi({ ...MINIMAL_OPENAPI, openapi: '3.1.0' }); + expect(() => readOpenApiDocument(root)).toThrow(/3\.2/); + }); + + // This renderer shows one body per carrier. Picking the first of several silently would + // document an endpoint that also accepts CBOR as accepting only JSON. + it('refuses a carrier offering more than one media type, naming the operation', () => { + const document = structuredClone(MINIMAL_OPENAPI); + document.paths['/v1/auth/login'].post.requestBody.content[ + 'application/cbor' + ] = { schema: { $ref: '#/components/schemas/LoginRequest' } }; + writeOpenApi(document); + expect(() => readOpenApiDocument(root)).toThrow( + /POST \/v1\/auth\/login/, + ); + expect(() => readOpenApiDocument(root)).toThrow(/media types/); + }); + + it('refuses a multi-media-type response too', () => { + const document = structuredClone(MINIMAL_OPENAPI); + document.paths['/v1/version'].get.responses[200].content[ + 'application/cbor' + ] = { schema: { $ref: '#/components/schemas/VersionResponse' } }; + writeOpenApi(document); + expect(() => readOpenApiDocument(root)).toThrow(/GET \/v1\/version/); + expect(() => readOpenApiDocument(root)).toThrow(/response 200/); + }); + + // A property table cannot express a union or an intersection, so a composed schema + // would render as an empty or a half-true model. + it.each([ + 'oneOf', + 'allOf', + 'anyOf', + ])('refuses a schema composed with %s, naming the schema', (keyword) => { + const document = structuredClone(MINIMAL_OPENAPI); + document.components.schemas.TokenResponse = { + title: 'TokenResponse', + [keyword]: [ + { $ref: '#/components/schemas/VersionResponse' }, + { type: 'object' }, + ], + }; + writeOpenApi(document); + expect(() => readOpenApiDocument(root)).toThrow(/TokenResponse/); + expect(() => readOpenApiDocument(root)).toThrow(new RegExp(keyword)); + }); + + // Checking only the root is the shape of the bug it prevents: a composition one level + // down falls through `typeOf` to `object` and renders as a row claiming the field is a + // plain object — a confident lie about a union. + it.each([ + 'properties.choice', + 'items', + 'additionalProperties', + ])('refuses a composition nested at %s, naming the schema and the path', (where) => { + const document = structuredClone(MINIMAL_OPENAPI); + const composed = { + oneOf: [{ type: 'string' }, { type: 'integer' }], + }; + const target = { type: 'object', title: 'TokenResponse' }; + if (where === 'properties.choice') { + target.properties = { choice: composed }; + } else if (where === 'items') { + target.items = composed; + } else { + target.additionalProperties = composed; + } + document.components.schemas.TokenResponse = target; + writeOpenApi(document); + expect(() => readOpenApiDocument(root)).toThrow(/TokenResponse/); + expect(() => readOpenApiDocument(root)).toThrow(/oneOf/); + }); + + // A `$ref` is scanned on the target's own pass. Following it would report the same + // composition once per reference, under whichever name was scanned first. + it('reports a composed schema once, under its own name', () => { + const document = structuredClone(MINIMAL_OPENAPI); + document.components.schemas.TokenKind = { + title: 'TokenKind', + oneOf: [{ type: 'string' }, { type: 'integer' }], + }; + writeOpenApi(document); + expect(() => readOpenApiDocument(root)).toThrow(/schema TokenKind/); + }); + + it('accepts the committed document', () => { + expect(() => + readOpenApiDocument( + resolve(dirname(fileURLToPath(import.meta.url)), '..', '..'), + ), + ).not.toThrow(); + }); +}); + +describe('renderApiPage', () => { + const group = API_GROUPS.find((entry) => entry.slug === 'auth'); + + function page() { + return renderApiPage( + group, + bucketOperations(MINIMAL_OPENAPI).get('auth'), + MINIMAL_OPENAPI, + ); + } + + it('renders the method and path as the operation heading', () => { + expect(page()).toContain('## POST /v1/auth/login'); + }); + + // `openapi.json` really does carry `# Two statuses, because there are two outcomes`. + // Interpolated unchanged it is a second h1 on the page. + it('demotes a heading inside an operation description', () => { + const rendered = page(); + expect(rendered).toContain( + '#### Two statuses, because there are two outcomes', + ); + const body = rendered.split('\n---\n').slice(1).join('\n---\n'); + expect(body.split('\n').filter((line) => /^# /.test(line))).toEqual([]); + }); + + it('says which operations require authentication', () => { + expect(page()).toMatch(/[Bb]earer/); + }); + + it('renders the request body schema and its fields', () => { + const rendered = page(); + expect(rendered).toContain('LoginRequest'); + expect(rendered).toContain('`email`'); + expect(rendered).toContain('The account email.'); + }); + + it('escapes a pipe inside a schema description', () => { + expect(page()).toContain('Gates nothing \\| really.'); + }); + + // Every use of a type is a table cell, so an unescaped separator opens a fourth column + // and shifts the row — `device_id` in the committed document is exactly this shape. + it('renders a nullable union as an escaped type, not as [object Object]', () => { + const rendered = page(); + expect(rendered).not.toContain('[object Object]'); + expect(rendered).toContain('`string \\| null`'); + expect(rendered).not.toContain('`string | null`'); + }); + + it('keeps every table row at the width of its header', () => { + for (const line of page().split('\n')) { + if (!line.startsWith('|')) continue; + // A cell may legitimately contain an escaped pipe; an unescaped one is a column. + const columns = line.replace(/\\\|/g, '').split('|').length; + expect(columns).toBeLessThanOrEqual(6); + } + }); + + it('resolves a $ref one level and links deeper refs to their anchor', () => { + const rendered = renderApiPage( + API_GROUPS.find((entry) => entry.slug === 'version'), + bucketOperations(MINIMAL_OPENAPI).get('version'), + MINIMAL_OPENAPI, + ); + expect(rendered).toContain('VersionResponse'); + expect(rendered).toContain('Lowest supported.'); + }); + + it('lists responses in ascending status order', () => { + const rendered = page(); + expect(rendered.indexOf('| `200`')).toBeLessThan( + rendered.indexOf('| `401`'), + ); + }); + + it('renders a path parameter', () => { + const rendered = renderApiPage( + API_GROUPS.find((entry) => entry.slug === 'albums'), + bucketOperations(MINIMAL_OPENAPI).get('albums'), + MINIMAL_OPENAPI, + ); + expect(rendered).toContain('`album_id`'); + expect(rendered).toContain("The album's identifier."); + }); + + it('opens with frontmatter the content schema accepts', () => { + const rendered = page(); + expect(rendered.startsWith('---\n')).toBe(true); + expect(rendered).toMatch(/^status: stable$/m); + expect(rendered).toMatch(/^title: "[^"]+"$/m); + }); + + it('emits only anchors its own headings answer', () => { + const rendered = page(); + const anchors = headingAnchors(rendered); + const linked = [...rendered.matchAll(/\]\(#([^)]+)\)/g)].map( + (match) => match[1], + ); + expect(linked.length).toBeGreaterThan(0); + for (const anchor of linked) { + expect(anchors.has(anchor)).toBe(true); + } + }); + + it('ends with exactly one newline and no run of blank lines', () => { + const rendered = page(); + expect(rendered.endsWith('\n')).toBe(true); + expect(rendered.endsWith('\n\n')).toBe(false); + expect(rendered).not.toMatch(/\n{3,}/); + }); +}); + +describe('the schema appendix', () => { + // The depth bound and the cycle guard interact. Keyed on "seen at all", the answer + // depends on traversal order: reach a model at depth 2 first and its children are cut + // by the bound, and the later path that reaches it at depth 1 is then refused as + // already-seen. Keyed on the shallowest depth, both paths get their chance. + it('expands a model when a shallower path reaches it after a deeper one', () => { + const document = structuredClone(MINIMAL_OPENAPI); + const schemas = document.components.schemas; + schemas.Leaf = { + type: 'object', + title: 'Leaf', + properties: { + mark: { type: 'string', description: 'The leaf mark.' }, + }, + }; + // A chain long enough that the deep path runs past MAX_SCHEMA_DEPTH exactly at + // `Tail`, so `Leaf` is out of reach along it. + schemas.Tail = { + type: 'object', + title: 'Tail', + properties: { leaf: { $ref: '#/components/schemas/Leaf' } }, + }; + for (const [name, next] of [ + ['Link3', 'Tail'], + ['Link2', 'Link3'], + ['Link1', 'Link2'], + ]) { + schemas[name] = { + type: 'object', + title: name, + properties: { next: { $ref: `#/components/schemas/${next}` } }, + }; + } + // Walked first (`requestBody` before `responses`): reaches `Tail` too deep to + // expand it. The response then reaches the same name at depth 0. + schemas.LoginRequest.properties.deep = { + $ref: '#/components/schemas/Link1', + }; + document.paths['/v1/auth/login'].post.responses[200].content[ + 'application/json' + ].schema = { $ref: '#/components/schemas/Tail' }; + + const rendered = renderApiPage( + API_GROUPS.find((entry) => entry.slug === 'auth'), + bucketOperations(document).get('auth'), + document, + ); + expect(rendered).toContain('### Tail'); + expect(rendered).toContain('### Leaf'); + expect(rendered).toContain('The leaf mark.'); + }); + + // The concrete instance of that bug in the committed document: `WireBlobRole` is + // reachable from `/reference/api/sync/` and was named on the page while defined nowhere. + it('documents every model the committed sync page links to', () => { + const repoRoot = resolve( + dirname(fileURLToPath(import.meta.url)), + '..', + '..', + ); + const document = readOpenApiDocument(repoRoot); + const group = API_GROUPS.find((entry) => entry.slug === 'sync'); + const rendered = renderApiPage( + group, + bucketOperations(document).get('sync'), + document, + ); + expect(rendered).toContain('### WireBlobRole'); + const headings = new Set( + [...rendered.matchAll(/^#{2,6} (.+)$/gm)].map((match) => + match[1] + .toLowerCase() + .replace(/[^a-z0-9 -]/g, '') + .replace(/ /g, '-'), + ), + ); + for (const [, anchor] of rendered.matchAll(/\]\(#([^)]+)\)/g)) { + expect(headings.has(anchor)).toBe(true); + } + }); +}); + +describe('the committed artifacts', () => { + // The assertion about *this repository* rather than about the generator: every + // operation the server declares reaches a page. A group table that quietly stopped + // covering a family would fail here as well as in `bucketOperations`. + const repoRoot = resolve( + dirname(fileURLToPath(import.meta.url)), + '..', + '..', + ); + + it('bucket every declared operation into a group', () => { + const document = readOpenApiDocument(repoRoot); + const declared = Object.entries(document.paths).flatMap(([, item]) => + Object.keys(item).filter((key) => METHODS.has(key)), + ); + const buckets = bucketOperations(document); + const bucketed = [...buckets.values()].reduce( + (sum, operations) => sum + operations.length, + 0, + ); + expect(bucketed).toBe(declared.length); + expect(bucketed).toBeGreaterThan(50); + }); + + // Renders into a temp root seeded from the two committed artifacts, never into the + // working tree. `check-docs` runs `test-docs` and `build-docs` in parallel, and + // `generate` clears its output directories before writing: generating into the real + // tree races the Astro build reading it, which fails intermittently and only under the + // gate. The artifacts are the committed ones, so the assertion is still about this + // repository. + it('generate one page per group plus the CLI page', () => { + copyFileSync(join(repoRoot, CLI_SURFACE), join(root, CLI_SURFACE)); + copyFileSync( + join(repoRoot, OPENAPI_DOCUMENT), + join(root, OPENAPI_DOCUMENT), + ); + + const written = generate(root); + + expect(written).toContain( + 'capsule-docs/src/content/docs/reference/cli/commands.md', + ); + for (const group of API_GROUPS) { + expect(written).toContain( + `capsule-docs/src/content/docs/reference/api/${group.slug}.md`, + ); + } + // Every page the run reported is a page it actually wrote, under the temp root. + for (const path of written) { + expect(existsSync(join(root, path))).toBe(true); + } + expect(written).toHaveLength(API_GROUPS.length + 1); + }); +}); + +describe('tidyBlankLines, through the pages that use it', () => { + // A blank run inside a fenced example is part of the example. A whole-document collapse + // has the generator quietly editing the code it is quoting. + it('keeps a blank run inside a fenced example', () => { + const surface = structuredClone(MINIMAL_CLI); + surface.subcommands[0].long_about = + 'Example:\n\n```sh\ncapsule import a\n\n\ncapsule import b\n```'; + expect(renderCliPage(surface)).toContain( + 'capsule import a\n\n\ncapsule import b', + ); + }); +}); diff --git a/capsule-docs/scripts/lib/walk.mjs b/capsule-docs/scripts/lib/walk.mjs index be17464a..5846d516 100644 --- a/capsule-docs/scripts/lib/walk.mjs +++ b/capsule-docs/scripts/lib/walk.mjs @@ -12,6 +12,16 @@ * not check it out. A walk that descends into it passes on the runner and * fails on any machine that has run `git submodule update` — the same trap * `.markdownlint-cli2.jsonc` documents for its own ignore list. + * + * 3. The generated `/reference/` pages are the same trap in the other + * direction: gitignored build output that exists on any machine that has + * run `mise run build-docs` and on no CI runner, sitting *inside* the + * content tree every check scopes itself to. Walking them makes a check's + * verdict depend on whether the site happens to be built, which is exactly + * the property `docs-truth.mjs` claims not to have when it states its scope + * as committed text against committed text. They are pruned by path rather + * than by basename because `cli/` and `api/` are ordinary directory names + * that other trees are entitled to use. */ import { readdirSync } from 'node:fs'; @@ -30,6 +40,16 @@ const SKIP_DIRS = new Set([ 'target', ]); +/** + * Directory subtrees never descended into, matched on the repo-relative path. + * + * Build output that lands inside a scanned tree, so a basename rule cannot express it. + */ +const SKIP_PREFIXES = [ + 'capsule-docs/src/content/docs/reference/cli/', + 'capsule-docs/src/content/docs/reference/api/', +]; + /** * Yield repo-relative paths of every file under `root` whose name matches * `predicate`, depth-first, with `SKIP_DIRS` pruned and symlinks skipped. @@ -47,7 +67,13 @@ export function walkFiles(root, predicate) { // `isDirectory()`/`isFile()` are false for a symlink, which is how // the `docs` symlink is dropped without a special case for it. if (entry.isDirectory()) { - if (!SKIP_DIRS.has(entry.name)) visit(abs); + const relDir = `${relative(root, abs).split(sep).join('/')}/`; + if ( + !SKIP_DIRS.has(entry.name) && + !SKIP_PREFIXES.some((prefix) => relDir.startsWith(prefix)) + ) { + visit(abs); + } continue; } if (!entry.isFile()) continue; diff --git a/capsule-docs/scripts/reference-groups.mjs b/capsule-docs/scripts/reference-groups.mjs new file mode 100644 index 00000000..3d7194f2 --- /dev/null +++ b/capsule-docs/scripts/reference-groups.mjs @@ -0,0 +1,204 @@ +/** + * The `/reference/` page table — the one place the reference section's shape is decided. + * + * `design/developer-docs.md` requires the `Reference` sidebar to be hand-curated, "in the + * same style as `Design` and for the same reason: generated pages must not be allowed to + * determine navigation order". Autogenerating it from the emitted directory would order + * pages by filename, which is a fact about slugs rather than a decision about reading + * order. + * + * Hand-curated does not have to mean written twice. Both `gen-reference.mjs` and + * `astro.config.mjs` import this file: the generator buckets the description artifacts into + * these pages, the config builds the sidebar from the same list. Editing the order here + * moves the page and its navigation entry together, and a group that has navigation but no + * page — or a page nothing links to — is not expressible. + * + * This module is deliberately data plus two pure functions, with no `node:` imports, so the + * Astro config can import it in the browser-facing build without dragging filesystem code + * along. + */ + +/** + * A generated page. + * + * @typedef {object} ReferencePage + * @property {string} slug Last path segment of the route, and the emitted file's basename. + * @property {string} label Sidebar label and page title. + * @property {string} description Frontmatter description, shown in search results. + */ + +/** + * The CLI pages, in reading order. + * + * One page rather than one per command: `capsule` has 16 commands whose help is a sentence + * each, and sixteen pages of one paragraph would put the whole surface behind sixteen + * clicks. The command tree is small enough to read end to end. + * + * @type {ReferencePage[]} + */ +export const CLI_PAGES = [ + { + slug: 'commands', + label: 'Commands', + description: + 'Every capsule command, argument, and option, generated from the committed command tree.', + }, +]; + +/** + * A generated REST page: one endpoint family, with the path prefixes that belong to it. + * + * @typedef {ReferencePage & { pathPrefixes: string[] }} ApiGroup + */ + +/** + * The REST pages, in reading order — a client's order, not the document's: what version you + * are talking to, what the server advertises, how to authenticate, then the data plane, then + * the account surfaces. + * + * Grouped by hand because there is nothing to group by automatically. The Kynos document + * carries no `tags` on any of its 59 operations, so an autogenerated section would be 59 + * pages ordered by filename — which `design/developer-docs.md` forbids in as many words, + * and which would scatter the four `/v1/upload` operations of one protocol across the + * alphabet. + * + * The names track the surface map in `design/api-surfaces.md`, so a reader who arrives from + * a design document finds the page named after the surface they were reading about. + * + * @type {ApiGroup[]} + */ +export const API_GROUPS = [ + { + slug: 'version', + label: 'Version', + description: + 'The protocol handshake every client performs before it does anything else.', + pathPrefixes: ['/v1/version'], + }, + { + slug: 'well-known', + label: 'Server discovery', + description: + 'What a server publishes about itself: attestation keys, capabilities, deprecations, and revoked tokens.', + pathPrefixes: ['/.well-known/capsule/'], + }, + { + slug: 'auth', + label: 'Authentication and devices', + description: + 'Registration, sign-in, the second factor, session and device management, and key escrow.', + pathPrefixes: ['/v1/auth/'], + }, + { + slug: 'upload', + label: 'Upload', + description: + 'The resumable, encrypted upload protocol: open a session, drive it, and read its receipt.', + pathPrefixes: ['/v1/upload'], + }, + { + slug: 'albums', + label: 'Albums and lifecycle writes', + description: + 'Album creation, the lifecycle write surface, and the membership upgrade handshake.', + pathPrefixes: ['/v1/albums'], + }, + { + slug: 'sync', + label: 'Sync and blob fetch', + description: + 'The change feed a client drains after a cursor, and the range-capable blob endpoint it fetches from.', + pathPrefixes: ['/v1/sync', '/v1/blob/'], + }, + { + slug: 'storage', + label: 'Storage verification', + description: + 'Proving the server still holds what it said it held, and the receipts that attest to it.', + pathPrefixes: ['/v1/storage/', '/v1/assets/'], + }, + { + slug: 'shares', + label: 'Share links', + description: + 'Issuing and revoking a share link, and the unauthenticated surface that serves one.', + pathPrefixes: ['/v1/shares', '/s/'], + }, + { + slug: 'drops', + label: 'Guest drops', + description: + 'Guest upload links, the unauthenticated drop surface, and the owner-side inbox and adoption.', + pathPrefixes: ['/v1/drops', '/d/'], + }, + { + slug: 'quota', + label: 'Quota', + description: 'What the account has used, and what it is allowed.', + pathPrefixes: ['/v1/quota'], + }, + { + slug: 'moderation', + label: 'Moderation', + description: + "The account's own moderation record — every action taken against it, in order.", + pathPrefixes: ['/v1/moderation/'], + }, +]; + +/** + * The group a path belongs to, by longest matching prefix, or `null` when none matches. + * + * Longest-prefix rather than first-match so the table's *reading* order and its *matching* + * order are independent. First-match would make adding a narrower group — `/v1/auth/totp/` + * under `/v1/auth/` — silently depend on placing it above the broader one, which is a trap + * a reader reordering the table for navigation reasons would spring without noticing. + * + * @param {string} path An OpenAPI path template, e.g. `/v1/albums/{album_id}/ops`. + * @returns {ApiGroup | null} + */ +export function groupForPath(path) { + let best = null; + let bestLength = -1; + for (const group of API_GROUPS) { + for (const prefix of group.pathPrefixes) { + if (path.startsWith(prefix) && prefix.length > bestLength) { + best = group; + bestLength = prefix.length; + } + } + } + return best; +} + +/** + * Sidebar items for the `Reference` group, in the order they are read. + * + * Returns Starlight sidebar entries: the section overview first, then one nested group per + * surface whose own overview leads its generated pages. + * + * @returns {Array<{ slug: string } | { label: string, items: Array<{ slug: string }> }>} + */ +export function referenceSidebar() { + return [ + { slug: 'reference' }, + { + label: 'CLI', + items: [ + { slug: 'reference/cli' }, + ...CLI_PAGES.map((page) => ({ + slug: `reference/cli/${page.slug}`, + })), + ], + }, + { + label: 'REST API', + items: [ + { slug: 'reference/api' }, + ...API_GROUPS.map((group) => ({ + slug: `reference/api/${group.slug}`, + })), + ], + }, + ]; +} diff --git a/capsule-docs/src/content/docs/design/developer-docs.md b/capsule-docs/src/content/docs/design/developer-docs.md index 1d7763d0..cad635c8 100644 --- a/capsule-docs/src/content/docs/design/developer-docs.md +++ b/capsule-docs/src/content/docs/design/developer-docs.md @@ -11,9 +11,10 @@ resulting page lands. It does not decide which surfaces exist: that is the obeys are [API Practices](/development/api-practices/). Implemented in `capsule-docs/` (Astro + Starlight) plus one description emitter per surface, living -in the crate that owns the surface and driven by its `mise` task. Every surface named below is -**Planned** or **Blocked** — `capsule-docs/src/content/docs/reference/` holds no generated content -today. +in the crate that owns the surface and driven by its `mise` task. Two surfaces have landed: the REST +contract at `/reference/api/` and the command line at `/reference/cli/`, each generated from a +committed description artifact by `capsule-docs/scripts/gen-reference.mjs` and each drift-gated in +the Rust gate. The rest are **Planned**. ## The Problem @@ -83,8 +84,8 @@ in the docs gate, which cannot run it. | Surface | Description artifact | Emitted by | Drift gate | Page | Status | | --- | --- | --- | --- | --- | --- | -| REST | Kynos OpenAPI 3.2 document | `capsule-server::openapi()` via an emitter binary | `openapi-check-kynos` | `/reference/api/` | **Emitter and gate landed; rendering blocked** | -| CLI | command-tree JSON, man pages, shell completions | `capsule-cli` (clap) | new `--check` on the dump | `/reference/cli/` | Planned | +| REST | Kynos OpenAPI 3.2 document | `capsule-server::openapi()` via `gen_openapi` | `openapi-check-kynos` | `/reference/api/` | **Landed** | +| CLI | command-tree JSON | `capsule_cli::cli::command_tree()` via `gen_cli_surface` | `cli-surface-check` | `/reference/cli/` | **Landed** | | Rust SDK | rustdoc HTML (uncommitted) | `cargo doc -p capsule-sdk` | broken intra-doc links denied | `/reference/sdk/rust/` → `/reference/crates/` | Planned | | Swift bindings | uniffi surface JSON, dumped from the compiled cdylib | a dump step on `mise-tasks/gen-bindings` | new `--check` on the dump | `/reference/sdk/swift/` | Planned | | Kotlin bindings | as above | as above | as above | `/reference/sdk/kotlin/` | Planned | @@ -96,10 +97,12 @@ Each therefore needs a small committed dump alongside its existing generation st symbol-presence assertions already in `mise-tasks/gen-bindings` are the seed of that dump — they already enumerate the verbs each binding must export — but they assert, they do not yet emit. -**Why REST is blocked.** The committed `capsule-sdk/openapi.json` is emitted from the retired Salvo -server. Its Kynos replacement exposes `openapi() -> Document` but has a single route ported and no -emitter binary, so no Kynos document exists yet. The REST reference is generated from the Kynos -document when there is one; publishing the Salvo-derived file would document a server nothing runs. +**How REST got unblocked.** It was blocked on there being no Kynos document to publish: the +committed contract was emitted from the retired Salvo server, and publishing that would have +documented a server nothing runs. `capsule-server/openapi.json` is now the Kynos document — emitted +by `gen_openapi` from the route types, gated by `openapi-check-kynos` — and `/reference/api/` +renders it. The CLI followed the same shape rather than a second mechanism: a `command_tree()` dump, +a `gen_cli_surface` emitter, and `cli-surface-check` in the same gate. **What is deliberately not a reference surface:** diff --git a/capsule-docs/src/content/docs/reference/api.md b/capsule-docs/src/content/docs/reference/api.md new file mode 100644 index 00000000..3025628d --- /dev/null +++ b/capsule-docs/src/content/docs/reference/api.md @@ -0,0 +1,96 @@ +--- +title: REST API +description: The auth model, negotiation headers, and error contract common to every Capsule endpoint +status: draft +--- + +Capsule's server surface is REST over HTTP, described by a single OpenAPI 3.2 document that +`capsule-server` emits from its own route types. This page is the hand-written half of that +reference: the things true of every endpoint, which a per-endpoint page would otherwise repeat +fifty-nine times. The endpoints themselves are the generated pages listed in the sidebar. + +Read [API Surfaces](/design/api-surfaces/#surface--transport-map) first if you want the map from +a surface to the module that owns it and the design document that explains it. This reference +says what the wire looks like; that one says why. + +## The server holds no keys + +The most important thing to know before reading any endpoint: **every write is sealed on the +client before it is sent, and the sync feed returns opaque envelopes.** The server stores, +addresses, and serves ciphertext, and authorizes who may do so. It cannot read an asset, and no +endpoint accepts a plaintext one. + +That is also why this reference has no try-it panel. A playground could exercise the version +handshake, the auth flows, and blob fetch; everything else would either return ciphertext or +reject an unsealed body, and a reader who succeeded at it would leave believing the API accepts +plaintext. The honest equivalent is the [command line](/reference/cli/), which performs the +sealing. + +## Authentication + +Credentials ride the standard `Authorization: Bearer` header. An access token is short-lived and +issued by `POST /v1/auth/login`; `POST /v1/auth/refresh` rotates the pair. Each generated page +marks every operation as requiring authentication or not, read from the document's own security +requirements rather than from prose here. + +An account with a confirmed second factor does not get a session from `login` — it gets a +challenge, and the sign-in finishes at `POST /v1/auth/login/verify-totp`. A client that treats +`202` as a failure will appear to work until the first user enables TOTP. + +A `401` carries a `WWW-Authenticate` challenge, per RFC 9110. + +## Negotiation + +The protocol-negotiation header contract — what a client sends, what a server answers with — +is [API Surfaces](/design/api-surfaces/#negotiation-across-transports). It is a design +commitment, and it is not yet implemented on every route; what the generated pages show, +under each operation's parameters and response headers, is what the wire carries at this +commit. Read those for the truth about an endpoint today, and the design document for where +it is going. + +`GET /v1/version` is the unauthenticated reachability probe a client performs before the +handshake. It has no failure variant by construction. What a server publishes about itself — +attestation keys, capabilities, announced deprecations, revoked token identifiers — is under +[Server discovery](/reference/api/well-known/). + +## Errors + +Failures are `application/problem+json` (RFC 9457) bodies. Beyond the standard members, every +problem this server renders carries a **`code`**: a stable identifier from the `error.*` catalog +namespace described in [Internationalization](/design/i18n/). Clients localize the code; the +`detail` message stays English. + +**Switch on the code, never on the status alone.** HTTP status is deliberately coarse: + +| Rejection class | HTTP status | +| --- | --- | +| Structural (bad envelope, unknown enum, sizes) | `400` | +| Unauthenticated or expired token | `401` | +| Unauthorized (capability, quota-hard, suspension) | `403` | +| Not found, including indistinguishable-404 surfaces | `404` | +| Stale state (chain, cursor, directory regression) | `409` | +| Payload too large | `413` | +| Unsupported chunk media type | `415` | +| Protocol outside `[Min, Max]` | `426` | +| Rate limited | `429` | + +A `404` is sometimes a deliberate indistinguishability, not an absence: a surface that must not +reveal whether a resource exists to an unauthorized caller answers the same way in both cases. + +## Clients + +Do not hand-write a client. `capsule-sdk` is generated from this same document, and the +generated request and response types are the only ones guaranteed to track it. The layering +rules server code follows are [API Practices](/development/api-practices/). + +## How these pages stay true + +`capsule-server` emits `capsule-server/openapi.json` from its route types — no database, no key +material, no network, because the router is built purely to describe it — and +`mise run openapi-check-kynos` fails the Rust gate if the committed document disagrees with the +server. The documentation build reads that file and nothing else. + +A generated page is never edited. If something on it is wrong, the annotation it came from is +wrong: fix the handler or model documentation, run `mise run openapi-kynos`, and commit the +document. The pipeline and the reasoning behind it are +[Developer Documentation](/design/developer-docs/). diff --git a/capsule-docs/src/content/docs/reference/cli.md b/capsule-docs/src/content/docs/reference/cli.md new file mode 100644 index 00000000..6e92621e --- /dev/null +++ b/capsule-docs/src/content/docs/reference/cli.md @@ -0,0 +1,65 @@ +--- +title: CLI +description: What the capsule command line is for, how to install it, and where its contract lives +status: draft +--- + +`capsule` is the command line for a Capsule library. It is the only client that performs the +whole data plane end to end — it scans and imports files, seals them locally, opens upload +sessions, drains the sync feed, and rebuilds an index from what is on disk — which is why the +examples in this documentation are commands rather than HTTP requests. The server never sees a +key, so a request transcript would show ciphertext going in and ciphertext coming out; a +transcript of `capsule` shows the operation. + +This page is the hand-written half of the CLI reference. [Commands](/reference/cli/commands/) is +the generated half: every command, argument, and option, emitted from the same `clap` definitions +the binary parses with. + +## Install + +Release builds are published as an archive per target on the +[releases page](https://github.com/justin13888/Capsule/releases), each carrying a single +`capsule` executable. From a checkout, `cargo run -p capsule-cli --` runs the same binary +against the working tree. + +## The two things it holds + +A `capsule` invocation reads at most two pieces of durable state, and it helps to know which: + +- **A library** — a directory named by `--library`, holding the encrypted assets, their sidecar + metadata, and a SQLite index that can be rebuilt from the sidecars alone + (`capsule library rebuild`). Every offline command operates on one. +- **A session** — the token pair `capsule auth login` persists, owner-readable, under the + user's configuration directory. Every networked command reads it, and `capsule reset` removes + it. + +Three commands unseal a library, and all three take its passphrase: `capsule import`, +`capsule push`, and `capsule cull`. Each accepts `--passphrase-stdin` as well as prompting, +so each runs unattended. The `capsule library` subcommands are not among them — `info` and +`rebuild` read the version file, the sidecars, and the index, none of which is sealed, so +they need no passphrase and offer no flag for one. + +## Where the contract lives + +- The behaviour of the import pipeline is [Import Pipeline](/design/import/pipeline/); what + `capsule push` speaks is the [Upload Protocol](/design/import/upload-protocol/), and what + `capsule sync` drains is [Download & Sync](/design/import/download-sync/). +- The server endpoints behind the networked commands are the + [REST API](/reference/api/) reference. +- Terminal output is localized through the catalogs described in + [Internationalization](/design/i18n/). Help text is not yet: the command tree this reference + is generated from is English, deliberately and by pinning, so the artifact cannot vary with + the machine that emits it. + +## How the generated page stays true + +`capsule-cli` emits `capsule-cli/cli-surface.json` — a description of the command tree, read +straight from the `clap` definitions — and `mise run cli-surface-check` fails the Rust gate if +the committed copy disagrees with the code. The documentation build reads that file and +nothing else; it never runs cargo. So a new option cannot reach users without either appearing +on the generated page or failing CI. + +A generated page is never edited. If something on it is wrong, the annotation it came from is +wrong: fix the `clap` `about` or doc comment, run `mise run cli-surface`, and commit the +artifact. The pipeline and the reasoning behind it are [Developer +Documentation](/design/developer-docs/). diff --git a/capsule-docs/src/content/docs/reference/index.md b/capsule-docs/src/content/docs/reference/index.md index cff217da..dd94b024 100644 --- a/capsule-docs/src/content/docs/reference/index.md +++ b/capsule-docs/src/content/docs/reference/index.md @@ -4,17 +4,40 @@ description: Generated reference for Capsule's developer surfaces status: draft --- -This section will hold generated reference for every Capsule developer surface — the REST contract, -the command line, the Rust SDK, the Swift and Kotlin bindings, and the browser surface. +Reference for every Capsule developer surface. Each section is generated from a **description +artifact**: a small, committed, machine-readable file that the surface's own toolchain emits and +its own gate keeps current. The documentation build reads those files and never invokes cargo, +uniffi, or wasm-bindgen — which is what keeps a reference page from disagreeing with the code it +describes. [Developer Documentation](/design/developer-docs/) is the contract; this page is the +index to it. -**Nothing is published here yet.** The pipeline that emits these pages is specified in -[Developer Documentation](/design/developer-docs/), which names each surface, the artifact it is -generated from, and the gate that proves that artifact current. Until a surface's emitter and drift -gate exist, its page is deliberately absent rather than hand-written and stale. +Reference pages are generated, never written. Every section's hand-written prose is confined to +its overview page — what the surface is for, how to reach it, where its contract lives. If a +generated page is wrong, the annotation in the source is wrong. -In the meantime: +## Published -- The REST surface-to-transport map is [API Surfaces](/design/api-surfaces/), and the contract rules - server code follows are [API Practices](/development/api-practices/). +| Surface | Overview | Generated from | Kept current by | +| --- | --- | --- | --- | +| REST | [REST API](/reference/api/) | `capsule-server/openapi.json` | `mise run openapi-check-kynos` | +| Command line | [CLI](/reference/cli/) | `capsule-cli/cli-surface.json` | `mise run cli-surface-check` | + +## Not published yet + +These surfaces are named here rather than given an empty route, because a dead link is worse +than an honest absence. + +- **Rust SDK** and **workspace rustdoc.** The workspace is `publish = false`, so docs.rs will + never build it; rustdoc is built by the Rust gate and deployed beside this site rather than + committed. Planned as `/reference/sdk/rust/` and `/reference/crates/`. +- **Swift and Kotlin bindings.** The generated bindings are gitignored build output, so the + bun-only documentation build cannot read them; each needs a committed surface dump alongside + its existing generation step first. +- **Browser surface.** The same problem for `capsule_wasm.d.ts`, with the additional constraint + that its drift gate cannot run where the other Rust gates do. + +Until then: + +- The REST surface-to-transport map is [API Surfaces](/design/api-surfaces/), and the contract + rules server code follows are [API Practices](/development/api-practices/). - Code module to owning design doc is the [Module Map](/design/module-map/). -- `capsule --help` is the current source of truth for the command line. diff --git a/mise.toml b/mise.toml index 6260dcc6..122f1fe0 100644 --- a/mise.toml +++ b/mise.toml @@ -90,6 +90,7 @@ run = [ "mise run i18n-check", "mise run i18n-guard", "mise run openapi-check-kynos", + "mise run cli-surface-check", "mise run architecture-check", "mise run license-check", "mise run translate-readme-check", @@ -239,6 +240,20 @@ run = "cargo run -q -p capsule-server --bin gen_openapi" description = "Verify the committed Kynos OpenAPI 3.2 document matches the server" run = "cargo run -q -p capsule-server --bin gen_openapi -- --check" +# Dump the `capsule` command tree to the committed capsule-cli/cli-surface.json that the +# documentation build renders into `/reference/cli/` (slice `S-Z8`). The same shape as the +# OpenAPI pair above, and for the same reason: the docs job installs bun and nothing else, so +# it reads a committed artifact rather than asking cargo what the CLI looks like. State-free +# and deterministic — `command_tree()` walks compile-time clap attributes. Re-run and commit +# after any change to the argument surface. +[tasks.cli-surface] +description = "Dump the capsule command tree to capsule-cli/cli-surface.json" +run = "cargo run -q -p capsule-cli --bin gen_cli_surface" + +[tasks.cli-surface-check] +description = "Verify the committed capsule command tree matches the CLI" +run = "cargo run -q -p capsule-cli --bin gen_cli_surface -- --check" + # Regenerate the translated README..md files from README.md and the committed # per-locale translation data (xtask/translations/readme/). See xtask/src/translate_readme.rs. [tasks.translate-readme] diff --git a/xtask/src/i18n_guard.rs b/xtask/src/i18n_guard.rs index 404795d3..8727de72 100644 --- a/xtask/src/i18n_guard.rs +++ b/xtask/src/i18n_guard.rs @@ -32,6 +32,8 @@ //! argument list of a terminal-output or error macro. See the rule below — the CLI is //! the one Rust surface that renders prose to a human, and it had never been scanned, //! which is how the entire `capsule import` arm printed hardcoded English for months. +//! [`NEVER_SCANNED`] carves out `capsule-cli/src/bin/`, which is build tooling rather +//! than that surface. //! //! ## What counts as user-facing in a Rust binary //! @@ -86,6 +88,20 @@ use eyre::{Context, ContextCompat, Result, bail}; use regex::Regex; use serde_json::Value; +/// Subtrees inside a scanned root that are not the user-facing surface the root stands for. +/// +/// `capsule-cli/src/bin/` holds description-artifact emitters — `gen_cli_surface`, and +/// whatever joins it — that run from `mise` tasks and CI and are never installed. Their +/// audience is a developer reading a task's output, not a user of `capsule`, so the rule +/// this module enforces ("every string a user reads is a catalog key") does not apply to +/// them: routing a build tool's status line through `locales/` would put a string no user +/// can reach into every translation catalog. +/// +/// This is a carve-out for an *audience*, which is the distinction the module doc is built +/// on, not a narrowing of the rule for the surface itself. `capsule-cli/src/**` outside +/// this prefix is scanned exactly as before. +const NEVER_SCANNED: &[&str] = &["capsule-cli/src/bin/"]; + /// Repo-relative path of the documented allowlist (one `path\tstring` per line; /// `#` comments and blank lines ignored). Entries suppress a single known, /// intentionally-untranslated finding at that file for that exact captured string. @@ -208,6 +224,9 @@ fn scan_surface( .unwrap_or(&path) .to_string_lossy() .replace('\\', "/"); + if NEVER_SCANNED.iter().any(|prefix| file.starts_with(prefix)) { + continue; + } for f in detect(&content) { if is_key(&f.text) { continue;