Wave 4 / Search: docs say index lives in <root>/.skim but it is in ~/Library/Caches/skim/search/<hash>

[dean0x]

Wave: wave4-search-followups

flowchart LR
  T372["1. #372 lexical AND-intersect"]
  T373["2. #373 FileId skew"]
  T374["3. #374 AST verify gate"]
  T375["4. #375 remove index positional"]
  T377["5. #377 weights notice"]
  T380["6. #380 binarize + positional"]
  T376["7. #376 CRC marker"]
  T379["8. #379 worktree staleness"]
  T378["9. #378 risk volume weighting"]
  T381["10. #381 index-location docs"]
  T372 --> T373 --> T374 --> T375 --> T377 --> T380 --> T376 --> T379 --> T378 --> T381
  T373 -. requires .-> T374
  T373 -. requires .-> T380
  T380 -. requires .-> T376
  T375 -. requires .-> T377
  classDef cur fill:#dcffe4,stroke:#1a7f37,stroke-width:3px;
  class T381 cur;

Loading

Solid arrows = integration order into wave/wave4-search-followups. Dotted = hard dependency. One branch per ticket; this ticket highlighted.

Found via dog-fooding round 2 (2026-06-27, branch wave/wave4-search), tracked under #174 Wave 4. Priority: P2.

Problem: Docs (CLAUDE.md and user-facing) state the search index / search.db live in <root>/.skim/. Actually all artifacts (index.skidx/.skpost/.skfiles, ast_index.*, temporal.db) live in the OS cache dir ~/Library/Caches/skim/search/<hash>/ (hash of the canonical root). This mismatch cost real debugging time during dog-fooding.

Secondary: varying --root path spelling (trailing slash, etc.) hashes to a different dir and silently builds a duplicate index — observed 6 skim-search index dirs.

Best fix: correct the docs to the real location (and the SKIM_CACHE_DIR relationship); canonicalize the root path before hashing so equivalent roots map to one index dir; optionally print the resolved index path under --stats/--debug.

Refs: #174.

[dean0x]

Plan + resolved decisions (dynamic-plan, 2026-06-28)

Durable record of design decisions and acceptance criteria for #381: docs correction (index lives under OS cache dir, not <root>/.skim), accurate stats/JSON path-reporting, and root canonicalize-fallback normalization using pure-lexical algorithm.

Resolved design decisions

• AD-381-1 (run_stats, mod.rs) — Surface the resolved search cache directory in --stats output (text + JSON) under the key cache_dir, including before the "no index found" early-return; path goes INSIDE the single {"error":"no index found","cache_dir":"..."} object, exit FAILURE unchanged. Rationale: the hidden path cost real debugging time; extend --stats rather than add a flag. Governs: ADR-001.
• AD-381-2 (index.rs canonicalize helper) — On canonicalize() failure (non-existent root), fall back to a lexically-normalized path, not the raw spelling, so trailing-slash / dot-segment variants of the same non-existent root map to one index dir. Rationale: closes the residual duplicate-dir fork for the non-existent-root case (common-case fix already in Wave 1d: Index builder (file walker, incremental updates) #182/Wave 1f: CLI integration, auto-index, git hooks, staleness detection #188). Governs: ADR-001.
• AD-381-N (index.rs canonicalize helper — normalization algorithm) — The fallback normalization is PURE-LEXICAL: Path::components to strip trailing separators and collapse . segments, with NO .. resolution and NO filesystem calls — side-effect-free and cross-platform. Rationale: deterministic and OS-agnostic; .. spellings of a non-existent root remain divergent (deliberate; revisit in follow-up only if needed). Governs: ADR-001, ADR-006. Accepted trade documented in rustdoc.

### Acceptance Criteria

1. AC1 (Functionality, NEGATIVE — docs): After the change, grep -rn 'search\.db' CLAUDE.md README.md MUST return zero matches, and grep -rn '<root>/\.skim/search' CLAUDE.md README.md MUST return zero matches. The strings search.db and <root>/.skim/search.db as the rskim-search storage location MUST NOT appear in any documentation file.

2. AC2 (Functionality — docs accuracy): CLAUDE.md (both the line-18 crate description and the line-48 'Search DB' gotcha) and the README Code Search section MUST state ALL of: (a) base = $SKIM_CACHE_DIR when set, else the platform cache dir; (b) the macOS path ~/Library/Caches/skim AND the Linux path ~/.cache/skim; (c) the per-root subpath search/<sha256(canonical_root)[..16]>/; (d) the real SQLite filename is temporal.db (not search.db); (e) the companion artifacts index.skidx, index.skpost, index.skfiles, ast_index.*. The line-48 bullet's PRAGMA user_version/forward-only-migration text MUST be re-attached to temporal.db.

3. AC3 (Functionality — docs, .skim/ clarification): If the docs mention <root>/.skim at all in the search context, they MUST correctly describe it as the advisory build-lock path (cache dir, not project root) — i.e. the docs MUST NOT imply any rskim-search index/db artifact lives under the project-root .skim/.

4. AC4 (Functionality — --stats text output surfaces resolved path): skim search --stats (text mode) against a project WITH a built index MUST print a line containing the exact absolute path returned by resolve_search_cache_dir(root) for that root. The line MUST appear in addition to (not replacing) the existing files indexed, total n-grams, index size, git HEAD, staleness lines.

5. AC5 (Functionality — --stats surfaces path even when NO index exists): skim search --stats (text mode) against a project with NO built index MUST still print the resolved cache-dir path (the 'where would it go?' case) in addition to the existing 'no index found' guidance, and MUST exit non-zero (FAILURE, unchanged). The resolved path MUST equal resolve_search_cache_dir(root).

6. AC6 (API contract — --stats --json additive key, POSITIVE): skim search --stats --json against a built index MUST emit valid JSON containing the new string key cache_dir whose value equals resolve_search_cache_dir(root).display().to_string(), AND MUST retain every pre-existing key unchanged: file_count, total_ngrams, index_size_bytes, last_updated, git_head, staleness.

7. AC7 (API contract — --stats --json no-index branch, decided shape): skim search --stats --json with NO built index MUST emit a SINGLE valid parseable JSON object and MUST exit non-zero. The object MUST be exactly shaped {"error":"no index found","cache_dir":"<resolved path>"} — retaining its error field AND including the cache_dir key with the resolved path.

8. AC8 (API contract — resolve_search_cache_dir signature & determinism, POSITIVE): resolve_search_cache_dir(root: &Path) -> anyhow::Result<PathBuf> MUST remain the public(super) resolver with unchanged signature, and for ANY fixed input it MUST be deterministic. For an EXISTING on-disk root it MUST return exactly base.join("search").join(&sha256_hex(canonical.to_string_lossy().as_bytes())[..16]) where canonical = root.canonicalize().

9. AC9 (Functionality — AD-381-2/AD-381-N canonicalize-fallback equivalence, IN SCOPE): For a NON-existent root resolve_search_cache_dir MUST map trailing-slash and .-segment spellings to the SAME directory: resolve_search_cache_dir("/no/such/root") == resolve_search_cache_dir("/no/such/root/") == resolve_search_cache_dir("/no/such/./root"). This MUST hold WITHOUT performing filesystem I/O for the non-existent root (pure-lexical normalization via Path::components on the canonicalize-error path).

10. AC10 (NEGATIVE — no index-format change / no spurious rebuild): The change MUST NOT alter on-disk index bytes or trigger a rebuild for a pre-existing index. After building an index, then applying ONLY the doc+stats change and running any read command (--stats, a query), the index files (index.skidx/.skpost/.skfiles, ast_index.*) MUST be byte-identical (same mtime/content).

11. AC11 (NEGATIVE — three doc sites agree, PF-008 knowledge-drift): The corrected location/filename wording MUST be consistent across all three edited doc sites (CLAUDE.md:18, CLAUDE.md:48, README Code Search section): same filename (temporal.db), same base resolution, same SKIM_CACHE_DIR relationship. No site may retain the old wording while another is fixed.

12. AC12 (Performance — explicit bound): --stats performance MUST NOT regress beyond O(1) added work: the change adds at most two writeln! calls / one JSON key plus one extra pure-lexical path-normalization (components() walk, no syscalls) per CLI invocation on the cold path, taken ONLY on the canonicalize-error branch. There is NO numeric latency target for this ticket; the requirement is strictly 'no new per-file or per-ngram work, no new I/O, no extra canonicalize syscall for the existing-root path'.

13. AC13 (Functionality, NEGATIVE — AD-381-N pure-lexical algorithm, platform-agnostic): The fallback normalization MUST be pure-lexical and provable WITHOUT Windows CI. The canonical_or_normalized helper MUST satisfy, for relative non-existent inputs that collide on ANY OS: canonical_or_normalized("./foo") == canonical_or_normalized("foo"), canonical_or_normalized("foo/") == canonical_or_normalized("foo"), and canonical_or_normalized("foo/./bar") == canonical_or_normalized("foo/bar"). NEGATIVE bound: .. MUST NOT be resolved -> canonical_or_normalized("foo/../bar") != canonical_or_normalized("bar").

Build context

Wave-4 search follow-ups. Sequence position 10 of 10. Depends on: none. AD-381-N rustdoc at decision sites (ADR-004); falsifiable ACs (PF-007).

[dean0x]

Resolved on wave/wave4-search via PR #386 (Wave-4 dog-food round-2). Residual partial-scope was ticketed as #388–#391 + #392 and resolved. Not yet on main — ships with PR #368 (wave→main).