Optimize Codex cost refresh policy for large history windows

[TheAngryPit]

Context

While investigating local CodexBar menu/CPU lag, I found a narrower cost-history refresh problem that may be contributing to the broader menu performance reports in #1321, #1360, and #1371.

Local setup/signal:

• CodexBar 0.32.5 (80)
• Cost summary enabled
• Cost history window set to 365 days
• ~/.codex/sessions aggregate shape: ~13 GB across ~403 JSONL files
• A sample captured during a high-CPU spike pointed at the background cost path, not the main menu drawing path:
  • UsageStore.refreshTokenUsage
  • CostUsageFetcher.loadTokenSnapshot
  • CostUsageScanner.loadCodexDaily
  • CostUsageScanner.scanCodexFile
  • CostUsageJsonl.scan

This is related to #1016, but I think the shape is different enough to ask separately. #1016 was about raw JSONL scanner throughput / possible Swift vs Rust/SIMD optimization, and was closed as not worth pursuing without a real bottleneck. This issue is not proposing a scanner rewrite. It is about refresh policy for large history windows.

Proposal

For large windows such as 365 days, make the steady-state refresh cheaper by separating:

• the selected display/history window, e.g. "show me 365 days"
• from the normal refresh cost, e.g. "only process new or modified local activity"

Possible behavior:

• Run a full scan when the cost history window is first enabled or expanded.
• Run a full scan when the Codex sessions root changes.
• Run a full scan / rebuild when the cache schema, parser source hash, pricing key, or other correctness-critical invalidator changes.
• During normal background refreshes, use the existing cache and scan only new or modified session files.
• Detect modified sessions by file metadata/cache state, not just by date, so resumed/forked/older sessions are still handled.
• Optionally run a periodic full reconciliation, for example weekly, to catch any drift.

This would keep 365-day history useful without making every normal refresh scale like a large history rebuild.

Questions

• Does this refresh-policy direction fit how you want cost history to behave?
• Should manual "Refresh Now" force a full reconciliation, or should it also stay delta-based unless a setting/root/cache invalidator changed?
• If the direction sounds reasonable, can I claim this and open a small focused PR?

Intended PR scope if accepted

• Keep this to Codex cost-history refresh/cache policy.
• No Rust/SIMD helper.
• No broad menu-rendering refactor.
• No provider/networking changes.
• Add focused tests for:
  • cached steady-state refresh
  • new session file
  • modified existing session file
  • history-window expansion
  • cache invalidation / pricing or parser-key change

The goal would be a conservative performance fix that preserves correctness and is easy to review.

Suggested labels, if maintainers agree: enhancement, priority:medium, area:performance,
area:usage-accuracy, provider:codex, needs-design.

[clawsweeper]

Codex review: keeping this open for maintainer follow-up; there is still a little grit to resolve. Reviewed June 10, 2026, 10:50 AM ET / 14:50 UTC.

Summary
Current main already performs metadata-based cache reuse, incremental append parsing, and full refreshes for expanded windows and correctness invalidators, so the proposed mechanism is largely present. The reporter’s sampled high-CPU path may still identify residual warm-cache work, but a current-main profile must isolate whether directory enumeration, metadata reads, priority lookup, aggregation, or refresh scheduling is actually expensive before a focused change is designed.

Reproducibility: no. not yet at high confidence: the reporter provides a concrete large-history setup and a useful sampled call stack, but a warm-cache current-main trace or timing breakdown is needed to identify the operation that still scales poorly.

Ways to help us reproduce this

• Add a screenshot or short recording showing the behavior.
• Add expected vs actual behavior.

Next step
A feature-history owner should review a warm-cache current-main profile and decide manual-refresh and reconciliation semantics before authorizing a focused implementation.

Review details

Best possible solution:

Measure a warm-cache 365-day refresh on current main, expose focused scan-work counters or a deterministic test seam, and optimize only the dominant repeated operation while preserving the existing parser, pricing, root, window, fork, and modified-file invalidators.

Do we have a high-confidence way to reproduce the issue?

No, not yet at high confidence: the reporter provides a concrete large-history setup and a useful sampled call stack, but a warm-cache current-main trace or timing breakdown is needed to identify the operation that still scales poorly.

Is this the best way to solve the issue?

No, not as a direct implementation of the listed policy, because current main already contains most of those mechanics; the best solution is a narrower optimization based on measured residual warm-cache work.

AGENTS.md: found and applied where relevant.

Remaining risk / open question:

• Implementing the proposal literally could duplicate the existing incremental cache policy without reducing the sampled CPU cost.
• The supplied stack identifies the subsystem but does not distinguish JSONL parsing from directory enumeration, metadata checks, SQLite priority lookup, daily aggregation, pricing, or repeated refresh scheduling.
• Manual “Refresh Now” and periodic reconciliation semantics are product choices that could trade responsiveness against stale or missed cost data.

Codex review notes: reasoning high; reviewed against 4c092ce344c5.

Label changes

Label changes:

• add P2: This is a normal-priority performance improvement affecting users with unusually large local Codex histories, without evidence of a general outage.
• add impact:other: Repeated background cost-refresh work can consume substantial CPU and degrade menu responsiveness.
• add issue-rating: 🐚 platinum hermit: Current issue advisory state selects this label.
• add clawsweeper:needs-live-repro: Current issue advisory state selects this label.
• add clawsweeper:no-new-fix-pr: Current issue advisory state selects this label.
• add clawsweeper:needs-maintainer-review: Current issue advisory state selects this label.
• add clawsweeper:needs-product-decision: Current issue advisory state selects this label.

Label justifications:

• P2: This is a normal-priority performance improvement affecting users with unusually large local Codex histories, without evidence of a general outage.
• impact:other: Repeated background cost-refresh work can consume substantial CPU and degrade menu responsiveness.

Evidence reviewed

What I checked:

• Incremental file processing already exists: The Codex scanner keeps a cached file when its metadata is fresh, parses only appended bytes when safe, and falls back to a full rescan only when required. (Sources/CodexBarCore/Vendored/CostUsage/CostUsageScanner.swift:2126, 4c092ce344c5)
• Requested invalidators already exist: The refresh plan accounts for forced rescans, history-window expansion, session-root changes, cache migrations, pricing changes, priority metadata changes, and elapsed refresh intervals. (Sources/CodexBarCore/Vendored/CostUsage/CostUsageScanner.swift:2155, 4c092ce344c5)
• Cache records file and correctness state: The persisted cache includes parser producer identity, scan range, pricing and priority keys, roots, per-file modification time, size, and parsed byte offsets. (Sources/CodexBarCore/Vendored/CostUsage/CostUsageCache.swift:77, 4c092ce344c5)
• Focused incremental coverage exists: Current tests exercise cached Codex reports, incremental append behavior, fallback rescans, pricing changes, cache migration, and retained history outside a narrower scan window. (Tests/CodexBarTests/CostUsageScannerBreakdownTests.swift:103, 4c092ce344c5)
• Existing performance implementation provenance: The current incremental scanner and refresh-planning behavior originates in the May 18 performance change, which substantially expanded the scanner cache helpers and regression coverage. (Sources/CodexBarCore/Vendored/CostUsage/CostUsageScanner.swift:2126, 69de57b85de7)
• Repository product direction: VISION.md welcomes performance improvements when they remain low-complexity, while new behavior or meaningful data-storage changes require sign-off. (VISION.md:1, 4c092ce344c5)

Likely related people:

• steipete: Peter Steinberger authored the May 18 scanner performance implementation and has the largest history across the central scanner, cache, and tests. (role: feature owner and recent area contributor; confidence: high; commits: 69de57b85de7, 20004f3d2e81; files: Sources/CodexBarCore/Vendored/CostUsage/CostUsageScanner.swift, Sources/CodexBarCore/Vendored/CostUsage/CostUsageCache.swift, Tests/CodexBarTests/CostUsageScannerBreakdownTests.swift)
• ratulsarna: Ratul Sarna has repeated merged work in the cost scanner and related usage tests, including cross-file cost correctness changes. (role: recent adjacent contributor; confidence: medium; commits: 91146a28c308; files: Sources/CodexBarCore/Vendored/CostUsage/CostUsageScanner.swift, Tests/CodexBarTests/CostUsageScannerBreakdownTests.swift)
• hhh2210: Larry Hao contributed recent Codex cost-scanner correctness work and opened the earlier focused scanner-performance discussion referenced by this issue. (role: adjacent cost-scanner contributor; confidence: medium; files: Sources/CodexBarCore/Vendored/CostUsage/CostUsageScanner.swift, Tests/CodexBarTests/CostUsageScannerBreakdownTests.swift)

How this review workflow works

• ClawSweeper keeps one durable marker-backed review comment per issue or PR.
• Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
• A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
• PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
• Maintainers can also comment @clawsweeper review to request a fresh review only.
• Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
• Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
• Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

[TheAngryPit]

Thanks, this is helpful. I re-checked this against current upstream/main (a6bb6772) and agree my original proposal was too broad: the scanner already has the main cache/incremental/invalidator mechanics, so I am not going to jump straight to a behavior PR from that proposal.

I did a local current-main repro pass and kept it aggregate-only: no prompts, no raw JSONL lines, no file paths, no model names, and no cost/token totals.

Setup:

• Cost summary enabled, history window set to 365 days.
• Default Codex sessions root.
• 365-day aggregate shape from the local session tree:
  • 409 JSONL files
  • 12,896,095,558 bytes
  • 1,501,140 lines
  • 605,730 scanner-relevant lines
  • 39,563 lines over 32 KiB; 2,637 lines over 256 KiB
  • 35,178 turn_context lines; model field offset was always below 32 KiB in this dataset
  • 232,814 / 232,814 token_count rows had no explicit model field

After running current main, the cache snapshot shape was:

• 427 cached files
• 87 cached days
• priority metadata present
• priority turn IDs recorded for 367 days

Current-main release CLI timing (CodexBarCLI cost --provider codex --format json --days 365 --json-only; stdout redirected locally and not shared):

• run 1: 26.23s real / 6.63s user / 1.27s sys
• run 2, immediately after: 1.28s real / 0.13s user / 0.05s sys
• run 3, after waiting past the 60s refresh interval: 51.43s real / 26.35s user / 3.59s sys
• run 4, immediately after: 29.76s real / 6.77s user / 1.67s sys
• run 5, immediately after: 0.61s real / 0.13s user / 0.01s sys

All five CLI outputs were valid local Codex cost responses with historyDays=365, 87 daily rows, totals present, and no error.

My read: this supports your point. Cache reuse is definitely working, but there is still a residual warm-cache / refresh-inspection path that can take tens of seconds on this dataset. The next useful step is probably not the original policy change, but focused measurement around:

• file enumeration time and counts
• cached-file fresh hits vs append parses vs full rescans
• priority metadata lookup / priority turn comparison time
• report aggregation / pricing time
• how often the app path schedules this refresh while the menu is being opened

I will wait for maintainer direction before opening any PR. If useful, I can prepare a narrow measurement-only PR or local patch that exposes those counters first, rather than changing behavior immediately.

[steipete]

Fixed by #1430 and landed as dd8cf8b.

Current-main profiling on a real 60,607-file (~43 GB) Codex archive found the dominant expired-refresh cost was repeated per-file Foundation metadata/resource-identifier reads and cached-path validation, not the priority SQLite query. The landed fstatat-based replacement reduced the exact warm expired refresh from 28.79s on current main to 6.74s, preserved normalized output, and keeps the existing append/window/invalidation behavior without new persistent state.

Thanks @TheAngryPit for the reproducible large-history report and @ProspectOre for the related performance investigation.