Skip to content

Commit 1313fc2

Browse files
feat(state-dir): .codemap/ directory consolidation + self-healing files (impl of PR #53 plan) (#54)
* feat(state-dir): resolveStateDir + DB at <state-dir>/index.db (Tracer 1) - application/state-dir.ts: resolveStateDir({root, cliFlag, env}) per plan §D7. Constants STATE_DIR_DEFAULT='.codemap', STATE_DB_NAME='index.db'. 12 unit tests cover precedence + relative/absolute paths. - config.ts: ResolvedCodemapConfig gains stateDir; resolveCodemapConfig 3rd arg opts.stateDir; databasePath defaults to <stateDir>/index.db. User-supplied databasePath wins (escape hatch). - config.ts: loadUserConfig reads <state-dir>/config.{ts,js,json} (D8); legacy <root>/codemap.config.* dropped (pre-v1). - runtime.ts: getStateDir() getter. - bootstrap.ts: --state-dir + CODEMAP_STATE_DIR; precedence per D7. - bootstrap-codemap.ts (new): single helper extracts the loadUserConfig+resolveCodemapConfig+initCodemap+configureResolver dance from 9 cmd-* files. Tracer 4's ensureStateDir attaches here. - All 9 cmd-* files refactored; stateDir threaded through interfaces + main.ts dispatch + ServerOpts (mcp/serve). - audit-worktree.ts: cached entries at <sha>/.codemap/index.db (recursive layout — each cached worktree is its own self-contained codemap project). - audit-engine.ts: makeWorktreeReindex stops hard-coding db path; openCodemapDatabase() reads from initialised runtime. - sqlite-db.ts: openCodemapDatabase mkdirs the parent (state-dir may not exist on fresh project). Dogfood: - .codemap/.gitignore (self-managed, blacklist) — codemap repo + fixtures/minimal/ - root .gitignore: dropped .codemap.* and .codemap/audit-cache/ (nested .gitignore handles them) - fixtures/minimal/.codemap.db* removed (stale legacy) 703 tests pass. * feat(state-dir): ensureStateGitignore reconciler — self-healing (Tracer 2) - STATE_GITIGNORE_BODY constant — single source of truth for the canonical blacklist. - ensureStateGitignore(stateDir) — pure-shape return ({before, after, written}); idempotent (no write on steady state); auto-mkdir; user-edits rewritten back per D11 (file is codemap-managed; header line declares it). - 5 tests cover: fresh write, idempotent, user-modified, older-version self-heal, returned shape matches disk. Bumping STATE_GITIGNORE_BODY in a future PR is the entire migration — every consumer's project repairs itself on next codemap run. * feat(state-dir): ensureStateConfig reconciler (Tracer 3) * feat(state-dir): bootstrap orchestrator + agents-init delegation (Tracer 4) * docs(state-dir): sync README + architecture + glossary + agent rule/skill + changeset (Tracer 5) - README.md: --state-dir flag, config-file location, self-healing callout - docs/architecture.md: state-dir resolver in src/config.ts intro; bulk .codemap.db → .codemap/index.db; User config section rewritten with self-healing details (ensureStateGitignore + ensureStateConfig from src/application/, single attachment point in cli/bootstrap-codemap.ts) - docs/glossary.md: new entries for '.codemap/' / <state-dir> / CODEMAP_STATE_DIR; '.codemap/index.db'; '.codemap/.gitignore' / self-healing files - docs/roadmap.md: drop the consolidated-dir backlog item (it shipped) - docs/research/fallow.md: Adjacent-shipped bullet referencing PR #53 (plan) + #54 (impl) - .agents/rules/codemap.md + templates/agents/rules/codemap.md: bulk .codemap.db → .codemap/index.db; one-paragraph addition explaining state-dir + self-healing .gitignore (Rule 10 lockstep) - .agents/skills/codemap/SKILL.md + templates/agents/skills/codemap/SKILL.md: bulk path update (Rule 10 lockstep) - .changeset/codemap-dir-consolidation.md: minor — full design rationale + cleanup steps - docs/plans/codemap-dir-consolidation.md: deleted per docs-governance Rule 3 * docs(state-dir): refresh stale path refs across docs/ + slim self-authored comments Doc staleness sweep (after Tracer 5): - docs/glossary.md, docs/agents.md, docs/benchmark.md, docs/why-codemap.md, docs/research/competitive-scan-2026-04.md, docs/research/fallow.md (B.6 row): bulk `.codemap.db` → `.codemap/index.db` everywhere except the intentional 'old → new' migration callouts. - docs/architecture.md, docs/research/fallow.md, docs/packaging.md: `codemap.config.{ts,json}` → `<state-dir>/config.{ts,js,json}`. - docs/agents.md § Git: rewritten to describe the self-managed <state-dir>/.gitignore reconciler instead of root-.gitignore patching. - docs/benchmark.md: 'where the DB lives' updated; manual .gitignore note dropped (reconciler handles it). Concise-comments sweep on this turn's authored comments: - src/application/state-config.ts: 2 inline comments slimmed (TS/JS-validation-only and passthrough-rationale). * fix(state-dir): address CodeRabbit findings (1 inline + 4 outside-diff + 2 nitpicks) Real bug: - main.ts: runListBaselinesCmd was being called without stateDir — `codemap query --baselines` would fall back to the default DB instead of the caller-selected one. Fixed. Stale doc refs: - audit-worktree.ts: 3 JSDoc strings still said `.codemap.db` after Tracer 1's CACHE_ENTRY_DB_REL move; bumped to `.codemap/index.db`. - bootstrap.ts: printCliUsage() had two `.codemap.db` refs + missing --state-dir/CODEMAP_STATE_DIR docs in Environment+Options. Fixed. - config.ts: Zod databasePath.describe() said default was `<root>/.codemap.db`; corrected to `<state-dir>/index.db`. - .agents/skills + templates skills: 2 hard-coded `.codemap/` refs reworded to `<state-dir>/` with `(default .codemap/)` callout (state-dir is configurable). Nitpicks applied: - state-dir.test.ts: dropped redundant `require('node:fs')` for mkdirSync (already imported). - bootstrap-codemap.ts: consolidated two single-import lines from state-dir into one statement. Nitpicks declined: - changeset code-fence missing 'text' lang — purely cosmetic. - cmd-index.ts JSDoc on runIndexCmd — 'all public APIs need JSDoc' is a fabricated rule (sibling cmds inconsistent; same hallucination rejected on PR #50).
1 parent b62f308 commit 1313fc2

50 files changed

Lines changed: 816 additions & 388 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.agents/rules/codemap.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ alwaysApply: true
66

77
> **STOP.** Before you call Grep, Glob, SemanticSearch, or Read to answer a **structural** question about this repository — query the Codemap SQLite index first. This is not optional when the question matches a trigger pattern below.
88
9-
A local database (default **`.codemap.db`**) indexes structure: symbols, imports, exports, components, dependencies, markers, CSS variables, CSS classes, CSS keyframes.
9+
A local database (default **`.codemap/index.db`**) indexes structure: symbols, imports, exports, components, dependencies, markers, CSS variables, CSS classes, CSS keyframes. The `.codemap/` directory holds every codemap-managed file (`index.db` + WAL/SHM, `audit-cache/`, project `recipes/`, `config.{ts,js,json}`, self-managed `.gitignore`); override the dir with `--state-dir <path>` or `CODEMAP_STATE_DIR`. The `.codemap/.gitignore` is **codemap-managed and reconciled on every boot** (`ensureStateGitignore`) — bumping its canonical body in a PR auto-applies on every consumer's next run.
1010

1111
**This file** is for **developing Codemap** in this clone. **End users** of the published package get the agent rule from **`templates/agents/`** (via **`codemap agents init`**). **Generic defaults:** SQL and triggers stay project-agnostic — **edit** this rule for repo-specific paths and queries.
1212

@@ -51,7 +51,7 @@ actions:
5151

5252
Validation: SQL is rejected at load time if it starts with DML/DDL (DELETE/DROP/UPDATE/etc.); the runtime `PRAGMA query_only=1` is the parser-proof backstop.
5353

54-
**Baselines** (`query_baselines` table inside `.codemap.db`, no parallel JSON files): `--save-baseline[=<name>]` snapshots a result set; `--baseline[=<name>]` diffs the current result against it (added / removed rows; identity = `JSON.stringify(row)`). Name defaults to the `--recipe` id; ad-hoc SQL needs an explicit `=<name>`. Survives `--full` and SCHEMA bumps.
54+
**Baselines** (`query_baselines` table inside `.codemap/index.db`, no parallel JSON files): `--save-baseline[=<name>]` snapshots a result set; `--baseline[=<name>]` diffs the current result against it (added / removed rows; identity = `JSON.stringify(row)`). Name defaults to the `--recipe` id; ad-hoc SQL needs an explicit `=<name>`. Survives `--full` and SCHEMA bumps.
5555

5656
**Audit (`bun src/index.ts audit`)**: structural-drift command; emits `{head, deltas: {files, dependencies, deprecated}}` (each delta carries its own `base` metadata). Three mutually-exclusive snapshot sources: `--base <ref>` materialises a git committish via `git worktree add` to a sha-keyed cache under `.codemap/audit-cache/`, reindexes a temp DB, then diffs (sub-100ms second run; requires git; `base.source: "ref"`); `--baseline <prefix>` auto-resolves `<prefix>-files` / `<prefix>-dependencies` / `<prefix>-deprecated` from saved `query_baselines` entries (`base.source: "baseline"`); `--<delta>-baseline <name>` is the explicit per-delta override (composes with both). v1 ships no `verdict` / threshold config — consumers compose `--json` + `jq` for CI exit codes. Auto-runs an incremental index before the diff (use `--no-index` to skip for frozen-DB CI).
5757

‎.agents/skills/codemap/SKILL.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -41,11 +41,11 @@ Replace placeholders (`'...'`) with your module path, file glob, or symbol name.
4141
- **`--summary`** — counts only. With **`--json`**: **`{"count": N}`**. With **`--group-by`**: **`{"group_by": "<mode>", "groups": [{key, count}]}`**.
4242
- **`--changed-since <ref>`** — post-filter rows by **`path`** / **`file_path`** / **`from_path`** / **`to_path`** / **`resolved_path`** against **`git diff --name-only <ref>...HEAD ∪ git status --porcelain`**. Rows with no recognised path column pass through.
4343
- **`--group-by owner|directory|package`** — partition into buckets and emit **`{"group_by", "groups": [{key, count, rows}]}`**. **`owner`** reads CODEOWNERS (last matching rule wins); **`directory`** is the first path segment; **`package`** uses **`package.json`** **`workspaces`** or **`pnpm-workspace.yaml`**. **Mutually exclusive with `--save-baseline` / `--baseline`.**
44-
- **`--save-baseline[=<name>]`** — snapshot the result rows to the **`query_baselines`** table inside `.codemap.db` (no parallel JSON files; survives `--full` and SCHEMA bumps). Name defaults to the `--recipe` id; ad-hoc SQL needs an explicit `=<name>`. Re-saving with the same name overwrites in place.
44+
- **`--save-baseline[=<name>]`** — snapshot the result rows to the **`query_baselines`** table inside `<state-dir>/index.db` (default `.codemap/index.db`; no parallel JSON files; survives `--full` and SCHEMA bumps). Name defaults to the `--recipe` id; ad-hoc SQL needs an explicit `=<name>`. Re-saving with the same name overwrites in place.
4545
- **`--baseline[=<name>]`** — diff the current result against the saved baseline. Output `{baseline:{...}, current_row_count, added: [...], removed: [...]}` (with `--json`) or a two-section terminal dump. Identity = per-row multiset equality (canonical `JSON.stringify` keyed frequency map; duplicates preserved). Pair with `--summary` for `{baseline:{...}, current_row_count, added: N, removed: N}`. **Mutually exclusive with `--group-by`.**
4646
- **`--baselines`** lists saved baselines (no `rows_json` payload); **`--drop-baseline <name>`** deletes one. Both reject every other flag — they're list-only / drop-only operations.
4747
- **Per-row recipe `actions`** — recipes that define an **`actions: [{type, auto_fixable?, description?}]`** template append it to every row in **`--json`** output (recipe-only; ad-hoc SQL never carries actions). Under `--baseline`, actions attach to the **`added`** rows only (the rows the agent should act on). Inspect via **`--recipes-json`**.
48-
- **Project-local recipes** — drop **`<id>.sql`** (and optional **`<id>.md`** for description body + actions) into **`<projectRoot>/.codemap/recipes/`** to make team-internal SQL a first-class CLI verb. `--recipes-json` and the `codemap://recipes` MCP resource list project recipes alongside bundled ones with **`source: "bundled" | "project"`** discriminating them. Project recipes win on id collision; entries that override a bundled id carry **`shadows: true`** so agents reading the catalog at session start know when a recipe behaves differently from the documented bundled version. `<id>.md` supports YAML frontmatter for the per-row action template — **block-list shape only** (loader's hand-rolled parser; no inline-flow `[{...}]`): `---\nactions:\n - type: my-verb\n auto_fixable: false\n description: "..."\n---`. Validation: SQL is rejected at load time if it starts with DML/DDL (DELETE/DROP/UPDATE/etc.); the runtime `PRAGMA query_only=1` is the parser-proof backstop. `.codemap.db` is gitignored; **`.codemap/recipes/` is NOT** — recipes are git-tracked source code authored for human review.
48+
- **Project-local recipes** — drop **`<id>.sql`** (and optional **`<id>.md`** for description body + actions) into **`<state-dir>/recipes/`** (default `<projectRoot>/.codemap/recipes/`) to make team-internal SQL a first-class CLI verb. `--recipes-json` and the `codemap://recipes` MCP resource list project recipes alongside bundled ones with **`source: "bundled" | "project"`** discriminating them. Project recipes win on id collision; entries that override a bundled id carry **`shadows: true`** so agents reading the catalog at session start know when a recipe behaves differently from the documented bundled version. `<id>.md` supports YAML frontmatter for the per-row action template — **block-list shape only** (loader's hand-rolled parser; no inline-flow `[{...}]`): `---\nactions:\n - type: my-verb\n auto_fixable: false\n description: "..."\n---`. Validation: SQL is rejected at load time if it starts with DML/DDL (DELETE/DROP/UPDATE/etc.); the runtime `PRAGMA query_only=1` is the parser-proof backstop. `.codemap/index.db` is gitignored; **`.codemap/recipes/` is NOT** — recipes are git-tracked source code authored for human review.
4949

5050
**Audit (`bun src/index.ts audit`)** — separate top-level command for structural-drift verdicts. Composes B.6 baselines into a per-delta `{head, deltas}` envelope; v1 ships `files` / `dependencies` / `deprecated`. Two snapshot-source shapes:
5151

@@ -79,7 +79,7 @@ Each emitted delta carries its own `base` metadata so mixed-baseline audits are
7979

8080
- **`codemap://recipes`** — full catalog JSON (same as `--recipes-json`). Each entry carries `source: "bundled" | "project"` and `shadows: true` on project entries that override a bundled recipe id. Read this at session start so you know when a `--recipe foo` call will run a project override instead of the documented bundled version.
8181
- **`codemap://recipes/{id}`** — single recipe `{id, description, body?, sql, actions?, source, shadows?}`. Replaces `--print-sql <id>`.
82-
- **`codemap://schema`** — DDL of every table in `.codemap.db` (queried live from `sqlite_schema`).
82+
- **`codemap://schema`** — DDL of every table in `.codemap/index.db` (queried live from `sqlite_schema`).
8383
- **`codemap://skill`** — full text of bundled `templates/agents/skills/codemap/SKILL.md`. Agents that don't preload the skill at session start can fetch it here.
8484

8585
**Implementation:** `src/cli/cmd-mcp.ts` (CLI shell — argv + lifecycle) + `src/application/mcp-server.ts` (transport — SDK glue). Tool bodies live in `src/application/tool-handlers.ts` (pure transport-agnostic — same handlers `codemap serve` dispatches over HTTP); resource fetchers in `src/application/resource-handlers.ts`. Mirrors the `cmd-audit.ts ↔ audit-engine.ts` seam. `--changed-since` git lookups are memoised per `(root, ref)` pair across batch items so a `query_batch` of N items sharing the same ref does one git invocation, not N.
Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
---
2+
"@stainless-code/codemap": minor
3+
---
4+
5+
`.codemap/` directory consolidation + self-healing files. Every codemap-managed path lives under a single configurable state directory (default `.codemap/`, override via `--state-dir <path>` or `CODEMAP_STATE_DIR`). Cleans up the dual-pattern surface (`<root>/.codemap.db` + `<root>/.codemap/<thing>/`) that's been growing with every cache PR; collapses the user `.gitignore` patching surface to zero.
6+
7+
**New layout:**
8+
9+
```
10+
<root>/
11+
└── .codemap/ ← override via --state-dir / CODEMAP_STATE_DIR
12+
├── .gitignore ← codemap-managed (self-healing); tracked
13+
├── config.{ts,js,json} ← was <root>/codemap.config.*; tracked
14+
├── recipes/ ← user-authored SQL; tracked (existing)
15+
├── index.db ← was .codemap.db
16+
├── index.db-shm ← was .codemap.db-shm
17+
├── index.db-wal ← was .codemap.db-wal
18+
└── audit-cache/ ← was .codemap/audit-cache/ (existing)
19+
```
20+
21+
**Self-healing files (D11):** `<state-dir>/.gitignore` and `<state-dir>/config.json` are owned by idempotent `ensure*` reconcilers (`src/application/state-dir.ts`, `src/application/state-config.ts`) that run on every codemap boot — read → validate → reconcile → write only on drift. **The setup logic IS the migration**: future codemap versions add new generated artifacts to `STATE_GITIGNORE_BODY` (or extend the Zod schema), and every consumer's project repairs itself on the next `codemap` invocation. No more per-feature `.gitignore` patching in `agents-init.ts`.
22+
23+
**Pre-v1 — no migration shim:**
24+
25+
- `<root>/.codemap.db` → `<state-dir>/index.db` (rename basename)
26+
- `<root>/codemap.config.{ts,json}` → `<state-dir>/config.{ts,js,json}` (move file)
27+
- Existing dev clones: `rm .codemap.db .codemap.db-shm .codemap.db-wal` once and re-index; move `codemap.config.*` into `.codemap/` (or set `--config <old-path>` to keep using the legacy location explicitly).
28+
29+
**New flags + env:**
30+
31+
- `--state-dir <path>` — override the state directory (resolves relative to project root).
32+
- `CODEMAP_STATE_DIR` — same, env-var form.
33+
34+
**Internal refactor:** new `src/cli/bootstrap-codemap.ts` extracts the `loadUserConfig + resolveCodemapConfig + initCodemap + configureResolver` dance from 9 cmd-\* files into one helper that also runs the self-healing reconcilers. Adding a new self-healing file is now a one-line addition there.
35+
36+
Inspired by flowbite-react's `.flowbite-react/.gitignore` + `setup-*` pattern; expressed in codemap's own conventions (`ensure*` reconcilers, Zod schema as `z.infer` source of truth, pure `{before, after, written}` return shapes for testability).
37+
38+
Plan: PR #53 (merged). Implementation: PR #54.

‎.codemap/.gitignore‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
# codemap-managed — edits will be overwritten by `ensureStateGitignore`.
2+
# Blacklist of generated artifacts; tracked sources (recipes/, config.*)
3+
# default to tracked. Bump alongside any new cache (Rule 9 analogue).
4+
index.db
5+
index.db-shm
6+
index.db-wal
7+
audit-cache/

‎.gitignore‎

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,4 @@
11
node_modules/
2-
.codemap.*
3-
.codemap/audit-cache/
42
.DS_Store
53
dist/
64
*.tgz

0 commit comments

Comments
 (0)