ci: cut CircleCI credit burn ~25-30% (e2e overhead, master e2e, bit_pr sizing)

[luvkapur]

Why

CircleCI usage is pacing ~4.8M credits/month. Insights (last 30 days, all branches):

Job	Credits/mo	Share
e2e_test	3,413,659	71%
bit_pr	799,564	17%
e2e_test_bbit (nightly)	199,676	4%
bit_merge	152,627	3%

Two measured facts drive the changes: each e2e node spends ~162s attaching the workspace (fixed overhead x 40 nodes x every push), and master-push e2e_test gates nothing (bit_merge doesn't require it; the merge queue's "settled" check watches bit_merge only).

Changes

1. Stop persisting .pnpm-store to the workspace — nothing downstream reads it (all jobs point bit at package-manager.cache). Measured on this PR's run: workspace-attach time is unchanged (165s vs 162s median/node) — the payload is dominated by bit/node_modules, so this is hygiene, not savings.
2. e2e_test no longer runs on master pushes — every PR already ran the full suite to merge; a daily master canary now runs in the nightly workflow instead (~440k → ~185k credits/mo).
3. e2e_test parallelism 40 → 30 — suite time (~507 min total) is constant across node counts; only the ~3 min/node fixed setup scales with nodes. Costs ~4 min wall clock.
4. bit_pr 2xlarge → xlarge, heap 30GB → 12GB — the build is CPU-bound on a single env (see in-file measurement note); 16 cores idled at 80 credits/min. Revert to 2xlarge if the job dies with "Killed".
5. e2e_test_bbit halts on quiet days — when no new @teambit/bit version was published in 24h, the previous nightly already bundled and tested the identical bits (32 nodes x ~30 min saved).

Measured results (this PR's run, paired against the immediately preceding pre-change pipeline)

Job	Before	After	Delta
e2e_test credits/run	6,344 (40 nodes)	6,145 (30 nodes)	−199 (−3.1%), wall +1.6 min
e2e tests run/failed	2,943 / 0	2,936 / 0	correctness intact
bit_pr credits/run	661 (2xlarge)	318 (xlarge)	−52%, wall −3.6%, no OOM
workspace attach s/node	161.9	165.5	no change — (1) is hygiene only

Proven per-run savings extrapolate to ~300k credits/month; the structural changes (2) and (5) — which remove entire runs rather than shrink them — add an estimated ~350k on top (~440k of master e2e runs removed minus ~185k for the nightly canary, plus quiet-day bbit halts). bit_pr on xlarge is verified for small lanes here; a worst-case full-cascade load test (branch ci-pr-loadtest-xlarge) validates heavy PRs before this merges.

🤖 Generated with Claude Code

[qodo-free-for-open-source-projects]

PR Summary by Qodo

CI: reduce CircleCI credits by trimming workspaces, e2e load, and job sizing

⚙️ Configuration changes ✨ Enhancement 🕐 20-40 Minutes

AI Description

• Stop persisting unused .pnpm-store to shrink workspaces and cut attach overhead.
• Reduce e2e cost by lowering parallelism and skipping master-push runs.
• Right-size bit_pr and skip nightly bbit e2e when no new release shipped.

Diagram

graph TD
GH["GitHub pushes"] --> BT["Workflow: build_and_test"] --> SW[("setup_harmony workspace")]
SW --> E2E["Job: e2e_test (30x, non-master)"] & BPR["Job: bit_pr (xlarge)"]
NW["Workflow: nightly (scheduled)"] --> SW --> E2E
NW --> BBIT["Job: e2e_test_bbit (halt if no release)"] --> NPM["npm @teambit/bit"]

Loading

High-Level Assessment

The following are alternative approaches to this PR:

1. Replace workspace payload with cache-first strategy

• ➕ Can further reduce per-node attach time by avoiding large workspaces entirely
• ➕ Cache restore can be narrower-scoped (e.g., pnpm store + bit caches) than full workspace
• ➖ More tuning/iteration required to keep cache keys stable and avoid flakiness
• ➖ Risk of hidden coupling on workspace contents across jobs

2. Trigger released-binary e2e via publish event instead of nightly polling

• ➕ Eliminates redundant nightly runs without relying on npm time heuristics
• ➕ Runs tests closer to the release event, improving signal
• ➖ Requires additional plumbing (webhook/GitHub workflow) and secure token handling
• ➖ Event delivery failures need a fallback schedule anyway

3. Adaptive e2e parallelism (change-based or queue-based)

• ➕ Could reduce spend further on small-change PRs while preserving latency on large ones
• ➕ Can optimize for both cost and PR cycle time
• ➖ Adds complexity and risk of under-parallelizing unexpectedly slow test partitions
• ➖ Requires reliable change-to-test mapping or historical timing data

Recommendation: The PR’s approach is a good low-risk first cut: it removes a clearly unused workspace path, right-sizes the most expensive jobs, and moves master canary coverage to a predictable daily schedule. If additional savings are needed later, consider an event-driven trigger for released-binary e2e and/or a cache-first strategy to reduce workspace attach costs further.

Files changed (1) +53 / -7

Other (1) +53 / -7

config.ymlReduce CI credit burn via workspace, e2e, and job-sizing changes +53/-7

Reduce CI credit burn via workspace, e2e, and job-sizing changes

• Stops persisting .pnpm-store to the CircleCI workspace to reduce attach overhead. Right-sizes bit_pr to xlarge and reduces Node heap to fit 16GB RAM. Lowers e2e_test parallelism, prevents e2e_test on master pushes, adds a nightly master canary run, and halts e2e_test_bbit when no new @teambit/bit release was published in the last 24 hours.

.circleci/config.yml

[qodo-free-for-open-source-projects]

Code Review by Qodo

🐞 Bugs (1) 📘 Rule violations (0) 📎 Requirement gaps (0) 🎨 UX issues (0) 🔗 Cross-repo conflicts (0) 📜 Skill insights (0)

1. Nightly halt fails closed ✓ Resolved 🐞 Bug ☼ Reliability

Description

The new e2e_test_bbit pre-step assumes npm view always succeeds and that time[version] always
exists; if either npm call fails or the version key is missing, the Node snippet can throw or print
NaN, and the subsequent -ge integer comparison can fail and abort the nightly job before any
tests run.

Code

.circleci/config.yml[R981-986]

+            LATEST_AGE_HOURS=$(node -e "
+              const { execSync } = require('child_process');
+              const time = JSON.parse(execSync('npm view @teambit/bit time --json', { encoding: 'utf8' }));
+              const version = execSync('npm view @teambit/bit version', { encoding: 'utf8' }).trim();
+              console.log(Math.floor((Date.now() - new Date(time[version]).getTime()) / 36e5));
+            ")

Evidence

The new step runs first in e2e_test_bbit and does not guard against npm/JSON failures or missing
time[version], yet it uses the result as an integer in a bash -ge test; any of these conditions
can fail the step and abort the nightly job.

.circleci/config.yml[964-991]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The nightly optimization step in `e2e_test_bbit` can fail the job when npm metadata is unavailable/unexpected (network error, JSON parse error, missing `time[version]`, or computed `NaN`). This makes the nightly workflow brittle.
### Issue Context
This check runs before `attach_workspace` and before any e2e execution, so failures here prevent the canary from running at all.
### Fix Focus Areas
- .circleci/config.yml[971-991]
### Suggested fix
- Wrap the Node metadata fetch in a try/catch and explicitly validate the timestamp exists.
- In bash, only run `-ge` if the value is a non-empty integer; otherwise log and **continue with tests** (fail open).
- Example pattern:
- `LATEST_AGE_HOURS=$(node -e '...') || LATEST_AGE_HOURS=""`
- `if ! [[ "$LATEST_AGE_HOURS" =~ ^[0-9]+$ ]]; then echo "could not determine publish age; continuing"; else if [ "$LATEST_AGE_HOURS" -ge 24 ]; then ...; fi; fi`

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

2. e2e_test skipped on tags 🐞 Bug ≡ Correctness

Description

The added filters: branches: ignore: master on e2e_test in build_and_test likely prevents
e2e_test from running on tag pipelines, which can remove e2e coverage from semver tag runs even
though the repo explicitly uses semver tag filters for other jobs.

Code

.circleci/config.yml[R1555-1557]

+          filters:
+            branches:
+              ignore: master

Evidence

The repo config shows explicit semver tag usage (semver_tags_only_filters) and runs at least one
job (generate_docs) only on those tags. The modified e2e_test job in build_and_test now has
only a branches filter, changing how it participates in non-branch pipelines such as tags.

.circleci/config.yml[38-47]
.circleci/config.yml[1543-1557]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`e2e_test` in the `build_and_test` workflow now has a branches-only filter. In CircleCI, jobs with filters commonly require explicit `tags` rules to run on tag pipelines; otherwise they may be skipped for tags.
### Issue Context
This repo explicitly defines semver tag filtering and runs `generate_docs` on version tags; if `build_and_test` also runs on tags, `e2e_test` may no longer run there.
### Fix Focus Areas
- .circleci/config.yml[38-47]
- .circleci/config.yml[1543-1557]
### Suggested fix
Decide whether tag pipelines should run e2e:
- If yes, add a `tags:` rule to `e2e_test` (e.g. `only: /^v[0-9]+(\.[0-9]+)*$/` for semver tags, or `only: /.*/` if all tags).
- Keep `branches: ignore: master` to preserve the master-push cost reduction.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

3. Duplicated npm metadata calls ✓ Resolved 🐞 Bug ➹ Performance

Description

The new e2e_test_bbit quiet-day gate performs two npm view calls per parallel node; with
parallelism: 32 this multiplies identical external registry lookups and increases the chance that
transient npm issues affect the job.

Code

.circleci/config.yml[R982-985]

+              const { execSync } = require('child_process');
+              const time = JSON.parse(execSync('npm view @teambit/bit time --json', { encoding: 'utf8' }));
+              const version = execSync('npm view @teambit/bit version', { encoding: 'utf8' }).trim();
+              console.log(Math.floor((Date.now() - new Date(time[version]).getTime()) / 36e5));

Evidence

The job is configured with parallelism: 32 and the gate step is executed before any conditional
branching, so each node performs the same two npm registry queries.

.circleci/config.yml[964-986]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The quiet-day gate runs `npm view @teambit/bit time --json` and `npm view @teambit/bit version` in every parallel container.
### Issue Context
`e2e_test_bbit` runs with `parallelism: 32`, so this is up to 64 identical npm registry queries per nightly run.
### Fix Focus Areas
- .circleci/config.yml[964-986]
### Suggested fix
- Fetch everything needed in a single command (e.g. `npm view @teambit/bit version time --json`) and compute from that JSON.
- Also consider `npm view --silent` to avoid noisy output affecting parsing.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

Tip of the day

💡 Did you know, you can group findings by type and pick your Finding display, from Minimal to Full

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗