Skip to content

Commit d604a9e

Browse files
SutuSebastianSutuSebastian
committed
feat: initial Codemap — SQLite structural index, CLI, and API
- Indexer with oxc/Lightning CSS; Node (better-sqlite3) and Bun (bun:sqlite) - CLI (codemap) and programmatic createCodemap / runCodemapIndex - CI (format, lint, test, typecheck, build, Node smoke), Changesets release workflow - Docs, agent rules/skills, .github contributing and PR template
1 parent 759869d commit d604a9e

77 files changed

Lines changed: 6204 additions & 110 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.agents/rules/agents-first-convention.mdc‎

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
---
2+
description: When creating or moving rules/skills, always store the source file in .agents/ and symlink from .cursor/
3+
alwaysApply: true
4+
---
5+
6+
# Agents-First File Convention
7+
8+
When creating **any** new rule or skill, follow this convention:
9+
10+
## Rules (`.mdc` files)
11+
12+
1. Create the file in `.agents/rules/<name>.mdc`
13+
2. Create a symlink in `.cursor/rules/`:
14+
15+
```bash
16+
ln -s ../../.agents/rules/<name>.mdc .cursor/rules/<name>.mdc
17+
```
18+
19+
## Skills (`SKILL.md` files)
20+
21+
1. Create the directory and file in `.agents/skills/<name>/SKILL.md`
22+
2. Create a symlink in `.cursor/skills/`:
23+
24+
```bash
25+
ln -s ../../.agents/skills/<name> .cursor/skills/<name>
26+
```
27+
28+
## Why
29+
30+
- `.agents/` is the **source of truth** — it is IDE-agnostic and works across different AI coding tools.
31+
- `.cursor/` only contains **symlinks** pointing back to `.agents/`.
32+
- This keeps configuration portable and avoids duplication.
33+
34+
## Never
35+
36+
- Never place original rule/skill content directly in `.cursor/rules/` or `.cursor/skills/`.
37+
- Never create a rule or skill without both the `.agents/` file and the `.cursor/` symlink.

‎.agents/rules/codemap.mdc‎

Lines changed: 110 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,110 @@
1+
---
2+
alwaysApply: true
3+
---
4+
5+
# Codemap (structural codebase index)
6+
7+
> **STOP.** Before you call Grep, Glob, SemanticSearch, or Read to answer a **structural** question about this repository — query the Codemap SQLite index first. This is not optional when the question matches a trigger pattern below.
8+
9+
A local database (default **`.codemap.db`**) indexes structure: symbols, imports, exports, components, dependencies, markers, CSS variables, CSS classes, CSS keyframes.
10+
11+
**Generic defaults:** This rule is **project-agnostic**. After you vendor or symlink it (or use a future `codemap` CLI that writes agent files), **edit your copy** to add app-specific triggers and SQL — upstream text is only a baseline.
12+
13+
## CLI (this repo vs installed package)
14+
15+
| Context | Incremental index | Query |
16+
| ------- | ----------------- | ----- |
17+
| Developing in this repo | `bun src/index.ts` | `bun src/index.ts query "<SQL>"` |
18+
| After `bun link` / global `codemap` | `codemap` | `codemap query "<SQL>"` |
19+
| Published package (when available) | `bunx @stainless-code/codemap` | `bunx @stainless-code/codemap query "<SQL>"` |
20+
21+
Index another project: **`--root /path/to/repo`**, or set **`CODEMAP_ROOT`** or **`CODEMAP_TEST_BENCH`** (e.g. in **`.env`** in this repo — see [docs/bench-repo.md](../../docs/bench-repo.md)). Full rebuild: **`--full`**. Targeted re-index: **`--files path/to/a.ts path/to/b.tsx`**.
22+
23+
## Session start (do this ONCE per conversation)
24+
25+
Run incremental indexing to catch changes made outside this session:
26+
27+
```bash
28+
bun src/index.ts
29+
```
30+
31+
## Pre-flight check (do this EVERY time before searching)
32+
33+
1. **Does the question match a trigger pattern below?**
34+
2. **If yes → query the index.** Do NOT use Grep/Glob/SemanticSearch/Read.
35+
3. **If the index result is incomplete → THEN fall back** to other tools, but always try the index first.
36+
37+
Violating this order is wrong even if you get the right answer — it wastes time and ignores the tool purpose-built for this.
38+
39+
## Trigger patterns
40+
41+
If the question looks like any of these → use the index:
42+
43+
| Question shape | Table(s) |
44+
| --- | --- |
45+
| "What/which files import X?" | `imports` (by `source`) or `dependencies` (by `to_path`) |
46+
| "Where is X defined?" | `symbols` |
47+
| "What does file X export?" | `exports` |
48+
| "What hooks does component X use?" / "List React components" | `components` |
49+
| "What are the CSS variables/tokens for X?" | `css_variables` |
50+
| "Find all TODOs/FIXMEs" | `markers` |
51+
| "Who depends on file X?" / "What does file X depend on?" | `dependencies` |
52+
| "How many files/symbols/components are there?" | any table with `COUNT(*)` |
53+
| "What are the CSS classes in X?" | `css_classes` |
54+
| "What keyframe animations exist?" | `css_keyframes` |
55+
56+
## When Grep / Read IS appropriate
57+
58+
- Reading implementation details you need to edit
59+
- Reviewing logic, control flow, or business rules
60+
- Searching for patterns the index doesn't capture (string literals, inline logic, etc.)
61+
62+
## How to query
63+
64+
```bash
65+
bun src/index.ts query "<SQL>"
66+
```
67+
68+
## Quick reference queries
69+
70+
| I need to... | Query |
71+
| --- | --- |
72+
| Find a symbol | `SELECT name, kind, file_path, line_start, line_end, signature FROM symbols WHERE name = '...'` |
73+
| Find a symbol (fuzzy) | `SELECT name, kind, file_path, line_start FROM symbols WHERE name LIKE '%...%'` |
74+
| See file exports | `SELECT name, kind, is_default FROM exports WHERE file_path LIKE '%...'` |
75+
| See file imports | `SELECT source, specifiers, is_type_only FROM imports WHERE file_path LIKE '%...'` |
76+
| Who imports this module? | `SELECT DISTINCT file_path FROM imports WHERE source LIKE '~/some/module%'` |
77+
| Who imports this file? | `SELECT DISTINCT from_path FROM dependencies WHERE to_path LIKE '%...'` |
78+
| What does this depend on? | `SELECT DISTINCT to_path FROM dependencies WHERE from_path LIKE '%...'` |
79+
| Component info | `SELECT name, props_type, hooks_used FROM components WHERE name = '...'` |
80+
| All components | `SELECT name, file_path, props_type, hooks_used FROM components ORDER BY name` |
81+
| TODOs in a file | `SELECT line_number, content FROM markers WHERE file_path LIKE '%...' AND kind = 'TODO'` |
82+
| Most complex files | `SELECT from_path, COUNT(*) as deps FROM dependencies GROUP BY from_path ORDER BY deps DESC LIMIT 10` |
83+
| CSS design tokens | `SELECT name, value, scope FROM css_variables WHERE name LIKE '--%...'` |
84+
| CSS module classes | `SELECT name, file_path FROM css_classes WHERE is_module = 1` |
85+
| CSS keyframes | `SELECT name, file_path FROM css_keyframes` |
86+
87+
**Use `DISTINCT`** on dependency and import queries — a file importing multiple specifiers from the same module produces duplicate rows.
88+
89+
For the full schema, advanced query patterns, and troubleshooting, read the skill at `.agents/skills/codemap/SKILL.md`.
90+
91+
## Keeping it fresh
92+
93+
**After completing a step that modified source files, re-index before making any further queries.** The index only reflects the state at the time it was last built — edits you just made won't appear until you re-index.
94+
95+
```bash
96+
# Targeted — re-index only the files you just touched
97+
bun src/index.ts --files path/to/file1.tsx path/to/file2.ts
98+
99+
# Incremental — auto-detects changed files via git
100+
bun src/index.ts
101+
102+
# Full rebuild — after rebase, branch switch, or stale index
103+
bun src/index.ts --full
104+
```
105+
106+
### When to re-index
107+
108+
- **After editing files** — use `--files` with the paths you modified (fastest). Deleted files are auto-detected and removed from the index
109+
- **After switching branches or rebasing** — run `--full`
110+
- **When unsure which files changed** — run without flags to auto-detect via git

‎.agents/rules/concise-reporting.mdc‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
---
2+
description: AI reporting style — extreme concision
3+
alwaysApply: true
4+
---
5+
6+
# Concise Reporting
7+
8+
When reporting information to me, be extremely concise and sacrifice grammar for the sake of concision.

‎.agents/rules/no-bypass-hooks.mdc‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
---
2+
description: Never bypass or skip pre-commit hooks when committing code
3+
alwaysApply: true
4+
---
5+
6+
# Never Bypass Pre-Commit Hooks
7+
8+
## Rules
9+
10+
1. **Never use `--no-verify`** — Do not pass `--no-verify` (or `-n`) to `git commit`. The pre-commit hooks exist to enforce code quality (formatting, linting, type-checking, tests) and must always run.
11+
12+
2. **Never use `--no-gpg-sign`** or other hook-skipping flags unless the user explicitly requests it.
13+
14+
3. **If hooks fail, fix the code** — When a pre-commit hook rejects a commit, fix the underlying issue (type errors, lint violations, formatting) and commit again. Do not work around the hook.
15+
16+
4. **No exceptions without explicit user consent** — If you believe a hook must be skipped for a legitimate reason, explain why and ask the user for explicit permission before proceeding.

‎.agents/rules/tracer-bullets.mdc‎

Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,67 @@
1+
---
2+
description: Build features in small end-to-end slices, not big horizontal layers
3+
alwaysApply: true
4+
---
5+
6+
# Tracer Bullets
7+
8+
When building features, build a tiny end-to-end slice first, validate it works, then expand.
9+
10+
## Why
11+
12+
AI agents tend to produce complete solutions in one leap — all parsers, all schema, all docs — without ever testing whether the critical path works. This creates massive review burden and rework ("slop").
13+
14+
## Rules
15+
16+
1. **Start with one vertical slice** that touches all relevant layers for the simplest case
17+
2. **Commit and validate** that slice before expanding — the pre-commit hook will run format, lint, typecheck, and tests on staged files (when AI/agent env vars trigger it)
18+
3. **Expand outward** from the working slice in subsequent commits
19+
4. **Never build horizontal layers in isolation** (e.g. all DB helpers before any CLI wiring, or all docs before any working index path)
20+
21+
## Feature layers in this project
22+
23+
A typical vertical slice for Codemap touches these layers top-to-bottom:
24+
25+
1. **CLI / orchestration** — `src/index.ts` (args, incremental vs full, `query` subcommand)
26+
2. **Workers / parsing** — `src/parse-worker.ts`, `src/parser.ts`, `src/css-parser.ts`
27+
3. **Persistence** — `src/db.ts` (schema, inserts, `SCHEMA_VERSION`)
28+
4. **Config / runtime** — `src/config.ts`, `src/runtime.ts`, resolver
29+
5. **Tests** — `src/*.test.ts`
30+
6. **Docs** — `docs/*.md` when behavior is user-visible
31+
32+
## Example 1: Support a new source extension
33+
34+
Bad — building in layers:
35+
36+
- Update every glob, parser, and docs in one giant change
37+
- Hope CI and the index agree
38+
39+
Good — tracer bullet:
40+
41+
1. **`constants` + parser language** — `LANG_MAP` + `extractFileData` / worker `TS_EXTENSIONS` — commit, `bun run check`, small test parsing a one-line file
42+
2. **Resolver** — `resolver` extensions if needed — commit, validate
43+
3. **Docs** — `docs/architecture.md` table row — commit, validate
44+
45+
## Example 2: Add a new SQLite column or table
46+
47+
Bad — schema + all call sites + benchmarks in one unreviewable diff.
48+
49+
Good — tracer bullet:
50+
51+
1. **Schema + insert path** — `db.ts` + one write path exercised by a test or CLI run — commit, validate
52+
2. **Readers / query UX** — expose in `query` or docs — commit, validate
53+
3. **Benchmark / fixtures** — if numbers matter — separate commit
54+
55+
## Example 3: New CLI flag
56+
57+
Bad — flag parsing, help text, config, and tests all speculative.
58+
59+
Good — tracer bullet:
60+
61+
1. **Parse flag + minimal behavior** — e.g. `--dry-run` that only logs — commit, test
62+
2. **Wire to real work** — connect to indexer — commit, validate
63+
3. **Document** — README / `docs/architecture.md` — commit
64+
65+
## Commit cadence
66+
67+
Each commit should represent a functional, describable milestone — not a placeholder. Every tracer bullet is a shippable slice that works end-to-end, even if the feature isn't complete yet. Small commits get validated by the pre-commit hook and are easier to review and revert.

‎.agents/rules/verify-after-each-step.mdc‎

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
---
2+
description: After each working milestone, verify changed files using the same checks lint-staged runs
3+
alwaysApply: true
4+
---
5+
6+
# Verify Changed Files After Each Step
7+
8+
After completing a step, phase, or milestone, verify every file you touched using the project's existing checks. Don't wait for `git commit` — the pre-commit hook is a safety net, not a first line of defense.
9+
10+
## Why
11+
12+
AI agents tend to chain many edits across files and only discover breakage at commit time. By then the failing context is stale and the fix is harder. Running checks after each working milestone keeps the codebase green continuously.
13+
14+
## Discover Project Scripts
15+
16+
1. **Read `package.json` `scripts`** at the start of a task to know available commands.
17+
2. **Read `lint-staged.config.js`** (or equivalent) to know which checks apply to which file patterns.
18+
3. Never assume script names — always verify they exist in `package.json` before running them.
19+
20+
## Current Per-File Checks (from `lint-staged.config.js`)
21+
22+
| File pattern | Checks |
23+
| --- | --- |
24+
| `*.{js,jsx,ts,tsx,mjs,mts,cjs,cts}` | `bun run format:check`, `bun run lint` |
25+
| `*.{css,json,md,mdc,html,yaml,yml}` | `bun run format:check` |
26+
| `*.{ts,tsx}` | `bun run typecheck` with a temporary `tsconfig.lint-staged.json` that includes only **staged files under `src/`** (project-wide types still interconnect — use `bun run typecheck` if you need full-project certainty) |
27+
| `*.test.ts` | `bun test` (on changed test files) |
28+
29+
## What Counts as a Step
30+
31+
A "step" is any self-contained unit of work where you've finished editing and are about to move on:
32+
33+
- Completed a TODO / action item from a plan
34+
- Finished a tracer-bullet slice (see tracer-bullets rule)
35+
- Refactored or moved code across files
36+
- Added or modified a parser, CLI flag, schema, or test
37+
- Fixed a bug or addressed a review comment
38+
39+
## Rules
40+
41+
1. **Verify after every step** — Run the matching checks on every file you touched during that step before moving to the next one.
42+
2. **Fix before moving on** — If any check fails, fix it immediately while context is fresh. Never carry forward known failures.
43+
3. **Use the right scope** — Run `bun run lint` and `bun run format:check` on specific files when possible. Prefer `bun run typecheck` project-wide when types may depend on unstaged files.
44+
4. **Run affected tests** — If you modified or created `*.test.ts` files, run `bun test <file>` on them.
45+
5. **Re-index before querying Codemap** — If you changed indexed source and plan to run SQL against the structural index next, run `bun src/index.ts --files <paths>` with paths **relative to the indexed project root** (set `CODEMAP_TEST_BENCH` / `CODEMAP_ROOT` or `--root` so that root is correct — see [docs/bench-repo.md](../../docs/bench-repo.md)).
46+
6. **Don't duplicate the hook's job** — You don't need to re-verify at commit time; the pre-commit hook (`lint-staged`) handles that automatically when AI/agent env vars trigger it. Your job is to stay green *between* commits.

0 commit comments

Comments
 (0)