feat(envs): remove core envs from the manifest and their sources from the workspace - #10465
feat(envs): remove core envs from the manifest and their sources from the workspace#10465davidfirst wants to merge 239 commits into
Conversation
PR Summary by QodoLoad former core envs as regular registry envs with legacy version pinning
AI Description
Diagram
High-Level Assessment
Files changed (15)
|
Code Review by Qodo
1. Unfixable NonLoadedEnv remediation
|
![qodo-free-for-open-source-projects[bot]](https://avatars.githubusercontent.com/in/1068000?s=60&v=4){kind=small}
…cope capsules, restore bit-aspect cmd
|
Code review by qodo was updated up to the latest commit c7dd1a7 |
…cted pre-install warning, regen references
Code Review by QodoNew Review StartedThis review has been superseded by a new analysis |
|
Code review by qodo was updated up to the latest commit e6418b9 |
|
Code review by qodo was updated up to the latest commit c9eca3d |
…aspect and env envs to core
…eambit/bit into remove-core-envs-from-manifest
|
Code review by qodo was updated up to the latest commit 3f5c24e |
…nv templates on demand - register legacy core env ids as core extension names so their config entries stay name-only (versionless): prevents env-as-dependency edges that created circular TS project references in lane/tag builds - require the aspect env eslint/prettier configs lazily (saves ~400 file reads per bit command) - bit create: fall back to --env for template lookup, incl. templates registered on the generator slot by envs loaded from the global scope - rewrite e2e node-env fixtures to compose on the core aspect env instead of @teambit/node
|
Code review by qodo was updated up to the latest commit 0607c7d |
- teambit.harmony/aspect and teambit.envs/env become regular envs with pinned legacy versions. components using them get the exact released behavior after bit install (the react-free aspect env rewrite is reverted) - move the bit-aspect template and harmony starters to the generator aspect so 'bit create bit-aspect' and 'bit new' work without loading the env - bind manifest deps of legacy envs to pinned versions in scope context (models built when these envs were core don't list them as dependencies) - load the full manifest graph when loading aspects from the global scope - keep legacy core env ids versionless when configured via bit create/env set
|
Code review by qodo was updated up to the latest commit b23b273 |
…ion when available - fixes the ci snap failure: pinned-version copies of workspace components leaked into the load groups and into the snap list - review fixes: index-based BFS queue in getDependentsIds, guard the typescript require in the fallback compiler, suppress legacy-env load failures only when the env package itself is missing, match both quote styles when detecting fixture env packages
|
Code review by qodo was updated up to the latest commit 94eddce |
…s an install that re-keys them (#10595) Extracted from #10465 so the fix can land on its own. ## Problem pnpm keys a virtual-store directory by the package's peer-resolution hash, so an install that changes the dependency set gives the same `name@version` a **new** directory and deletes the one this process loaded its modules from. Node keeps the loaded module objects, but not the files — so any `require`/`import` the loaded code defers past load time resolves against the deleted directory and throws `MODULE_NOT_FOUND`. Every package the process loaded out of the workspace's own virtual store is exposed to this, and an env is the worst case: `@teambit/aspect` defers `require('./babel/babel-config')` until `getCompiler()` is called — which the install flow itself does right after the package-manager run, when it compiles components and reloads envs. The install then dies with `Cannot find module './babel/babel-config'`. Replacing the in-memory instances instead does not work (verified by trying): every reload path — `reloadMovedEnvs`, loading components as aspects — has to consult the registered env to do its work, and consulting it is exactly what throws. `reloadMovedEnvs` is additionally a no-op for these envs: it filters on `env.__path`/`env.id`, which only plugin-loaded envs carry. ## Fix Follow the rule an OS applies to a running binary's deleted files: what the process has loaded stays available for the process's lifetime. - Before the package-manager run, snapshot which virtual-store directories back loaded modules; afterwards, restore any that vanished from their re-keyed twin — same `name@version`, different peer hash — whose package content is identical (same tarball; the peer set only affects the sibling dependency symlinks, which are relative and stay valid from the restored location). - `pnpmPruneModules` skips directories backing loaded modules so it does not re-delete a restored one. A later command's process, which has nothing loaded from it, prunes it. - CJS modules are found via `require.cache`. ESM modules live in node's ESM module map, which has no enumeration API, so `aspect-loader` records every file it loads through dynamic `import()` (the env-plugin loader and `loadEsm()`, the only two such call sites) in a `Symbol.for`-keyed global set that the pnpm side reads. A global symbol rather than a shared import because the reader lives in the pnpm package manager, which `aspect-loader` must not depend on — the dependency runs the other way — and `Symbol.for` resolves to the same key even when a module is duplicated in `node_modules`. Both sides document the contract and point at each other. - Restores run sequentially: this sits right after every install, where the engine has just saturated the disk, and each restore is a recursive copy. The common case is zero removed directories and the checks stay cheap. ## Commits Cherry-picked from #10465 (`-x` trailers reference the originals): - `fix(deps): keep packages the running process loaded requireable across an install that re-keys them` - `fix(deps): extend loaded-package preservation across installs to ESM modules` - `fix(deps): restore removed loaded virtual-store dirs sequentially` #10465 also carries `fix(generator): stop reporting an env-load failure as "template not found"`, which is **not** included here: it fixes `getTemplateWithIdOrEnvFallback`, a method that only exists on that branch, so there is nothing on master for it to apply to. It stays with #10465. ## Verification - `bit test teambit.harmony/aspect-loader teambit.dependencies/pnpm` — 55/55 passing (4 new specs for the ESM recorder, 6 new for the snapshot/donor/prune logic). - `npm run lint` (tsc + oxlint) green. - Behavior verified on #10465 with `deps-in-capsules.e2e.ts`: the second install re-keys a dozen loaded `@teambit/*` slots; without the fix the suite fails on the babel-config require, with it all tests pass and the debug log shows each removed slot restored. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
…from-manifest The loaded-package preservation work this branch carried was upstreamed as #10595, so master and the branch both hold it and every conflict is between the branch's original and the reviewed version that landed. Master's version wins throughout: it is the same work with the realpath spellings, the patch-hash donor check, the slot-owner attribution and the prune early-return added on top. plugins.ts and aspect-loader's loadEsm merged cleanly into a double recordLoadedEsmFile - the branch recorded before the load, master after it, on the grounds that only a load that succeeded leaves something in memory worth keeping files for. Kept master's single post-load call in both. The branch-only aspect-loader changes (the versionless loaded-aspect lookup, the core-aspect manifest guard, the requested-id-preserving def dedup) are untouched - master never had them. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
| const envPackageName = getLegacyCoreEnvPackageName(failedIdWithoutVersion); | ||
| const isEnvPackageInstalled = fs.existsSync(path.join(this.path, 'node_modules', envPackageName)); | ||
| // the missing module may be reported by its package name or by its absolute path in | ||
| // the workspace node_modules (when the require used a resolved path). |
There was a problem hiding this comment.
1. Legacy env failures suppressed 🐞 Bug ◔ Observability
Workspace.getWorkspaceIssues() suppresses legacy-core env MODULE_NOT_FOUND failures by checking only <workspace>/node_modules/<envPkg>, but roots-based env resolution installs them under node_modules/.bit_roots/<envId>/node_modules/<envPkg>. This can hide real env load failures from bit status even when the env package is actually installed (just not hoisted to root node_modules).
Agent Prompt
### Issue description
`Workspace.getWorkspaceIssues()` suppresses certain legacy-core env load failures when it believes the env package is "not installed yet". The installation check only looks for `<workspace>/node_modules/<envPackageName>`, but with `resolveEnvsFromRoots` the env package may be installed under the roots layout (`node_modules/.bit_roots/...`). This can incorrectly suppress real failures from `bit status`.
### Issue Context
The suppression intends to hide the expected pre-`bit install` state, not hide failures when the env is actually present.
### Fix Focus Areas
- scopes/workspace/workspace/workspace.ts[1807-1818]
### What to change
- When deciding `isEnvPackageInstalled`, check both:
- `<workspace>/node_modules/<envPackageName>` (current)
- `<workspace>/node_modules/.bit_roots/<failedIdWithoutVersion>/node_modules/<envPackageName>` (roots layout, note that roots dir is keyed by versionless env id per comments in the resolver)
- Only suppress when **neither** location exists.
- Keep the existing `isEnvModuleNotFound` guard, but broaden it if needed to match the roots-path form as well.
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
|
Code review by qodo was updated up to the latest commit 854eed1 |
…s nested copy
An install that decides a hoisted copy satisfies what was nested deletes the
nested directory - and if this process loaded modules out of it, node keeps the
module objects but not the files, so any require the loaded code deferred past
load time throws MODULE_NOT_FOUND. `@teambit/aspect` defers
`require('./babel/babel-config')` until `getCompiler()`, which the install flow
itself calls right after the package-manager run, so the install dies with
`Cannot find module './babel/babel-config'`.
This is the same hazard #10595 fixed for the virtual store, reached by a
different route. That snapshot only scans `node_modules/.pnpm`, so a package
loaded from a root component's own node_modules was invisible to it and the
preservation was inert. root-components' hoisted-linker suite hit exactly this:
17 loaded directories under `.bit_roots/teambit.harmony_aspect/node_modules`
were removed by the second install.
`reloadMovedEnvs` does not cover it either - it skips any env without a
`__path`, which is only set for plugin-registered envs, and `@teambit/aspect`'s
AspectEnv is registered by its aspect provider. Reloading is no substitute
anyway: consulting the registered env is what triggers the deferred require.
So preserve these the same way: snapshot the package directories under the
workspace's node_modules that back loaded modules, and afterwards restore any
that vanished from a same-version copy found by walking the node_modules chain
up from where it used to be - what node itself would resolve now - bounded at
the workspace root. A donor must match on version, since the point is to keep
serving the files belonging to the modules already in memory. The loaded-files
plumbing shared with the virtual-store module moves to loaded-module-files.ts.
Verified with root-components.e2e.ts's hoisted-linker suite: 0 passing/1 failing
before (the before-all hook died on the babel-config require), 12 passing after,
with the debug log showing 17 of 17 removed directories restored.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Code review by qodo was updated up to the latest commit 62ff0c2 |
… of a nested capsule's A nested capsule install is rooted at the capsule, but it was handed the root links keyed by the capsules root - the directory one level *above* its root. That describes an importer outside the workspace, and the package manager then has to give the root project itself a directory dep path relative to itself. That path is empty, so it comes out as `<name>@file:`, which its own parser rejects: Failed to build lockfile from resolved dependency graph: Resolved dependency path "@teambit/react@file:(...)" keys no lockfile entry: Failed to parse suffix: Empty path after `file:` scheme Only the first nested capsule got the links (`index === 0`), so this needed that capsule to be a named package - which is exactly what an env capsule is. The mis-keying dates to #10453/#10457; it stayed latent while envs were core aspects and no env capsule was installed this way. Give the links the install they describe: one rooted at the capsules root. It runs before the capsules rather than alongside them, so they can resolve the core aspects it links while they install and nothing races it over the root's node_modules. When a cyclic group exists it is already rooted there and keeps owning them, unchanged. Verified against optional-dependencies.e2e.ts, whose "before all" runs `bit create react button --env teambit.react/react`: 0 passing/1 failing before, 10 passing after. The failure needs a cold capsule cache - once the env capsule is cached the install is rooted at the capsules root instead and the bug is skipped - so reproduce with `rm -rf ~/Library/Caches/Bit/capsules/*` first. On CI this cluster accounts for 12 failing e2e tests. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
| // envs that used to be core aspects are old-style envs, they are not of type env by design | ||
| if (this.envs.isLegacyCoreEnv(envId)) return; |
There was a problem hiding this comment.
1. Core env check ignores versions 🐞 Bug ≡ Correctness
Workspace.warnAboutMisconfiguredEnv() checks core envs via getCoreEnvsIds().includes(envId) without stripping a version, even though EnvsMain.isCoreEnv() was updated to ignore versions for core env IDs. If a core env ID ever reaches this method in a versioned form (the scenario is explicitly documented in EnvsMain.isCoreEnv), the workspace may perform unnecessary resolution and emit an incorrect misconfiguration warning.
Agent Prompt
### Issue description
`Workspace.warnAboutMisconfiguredEnv()` uses a raw string `includes()` check against `getCoreEnvsIds()`, but core env IDs are now explicitly treated as version-insensitive elsewhere (`EnvsMain.isCoreEnv()` strips `@version`). This creates inconsistent core-env detection and can mis-handle versioned core-env IDs.
### Issue Context
This method is on a hot path during component load and is used to warn about envs that are “not of type env”. With versioned core-env IDs, it may attempt resolution for a core env (which may not exist as a component) and/or warn incorrectly.
### Fix Focus Areas
- scopes/workspace/workspace/workspace.ts[1722-1726]
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
| getLoadedAspectIdIgnoringVersion(idWithoutVersion: string): string | undefined { | ||
| return this.harmony.extensionsIds.find( | ||
| (extId) => extId.split('@')[0] === idWithoutVersion && Boolean(this.harmony.extensions.get(extId)?.loaded) | ||
| ); |
There was a problem hiding this comment.
2. Legacy env canonical id unstable 🐞 Bug ☼ Reliability
AspectLoaderMain.getLoadedAspectIdIgnoringVersion() returns the first loaded aspect ID matching the versionless ID, making legacy-core env canonicalization dependent on Harmony extension ordering rather than a deterministic policy (e.g. highest version). DependencyResolverMain.getCanonicalLegacyCoreEnvId() uses this result to enforce legacy-core env single-instance semantics, so when multiple versions are loaded it can bind to an unintended version despite EnvsMain explicitly sorting/warning for multi-version legacy envs.
Agent Prompt
### Issue description
`getLoadedAspectIdIgnoringVersion()` uses `Array.find()` over `harmony.extensionsIds`, so when multiple versions of the same aspect/env are loaded it picks whichever was registered first. This is inconsistent with EnvsMain’s deterministic “pick highest version and warn” behavior for multi-version legacy-core envs.
### Issue Context
This function is used by dependency resolution to rewrite legacy-core env dependency IDs to a single canonical instance. If multiple loaded versions exist, order-dependent selection can lead to confusing/unstable behavior.
### Fix Focus Areas
- scopes/harmony/aspect-loader/aspect-loader.main.runtime.ts[237-251]
- scopes/dependencies/dependency-resolver/dependency-resolver.main.runtime.ts[1578-1585]
- scopes/envs/envs/environments.main.runtime.ts[304-327]
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
|
Code review by qodo was updated up to the latest commit 825cfeb |
e2e status after merging master: 20 → 8 failuresMerged The merge itself is cleanSet comparison of failing test names, merge commit vs. the run before it: 0 new failures, 3 fixed. Every conflict was this branch's copy of the loaded-package-preservation work that master picked up as #10595, so master's reviewed version won throughout. One thing worth flagging because git did not: Fixed1. The second install deletes
Fix: a sibling module Verified: hoisted-linker suite 0 passing/1 failing → 12 passing, debug log showing 17 of 17 removed directories restored. On CI this signature went 1 → 0. 2. Every This is not the deps-graph converter — The actual cause is in if (index === 0 && !cyclicCapsules.length) {
linkedDependencies[capsulesDir] = rootLinks; // capsulesDir is the PARENT
}
await this.installInCapsules(capsule.path, ...) // rootDir is the CHILDThat describes an importer one level above the install root, so the package manager has to give the root project a directory dep path relative to itself — empty, hence The mis-keying dates to #10453/#10457 and is latent on master — it only bites here because envs are no longer core aspects, so an env capsule gets installed this way. Fix: give those links the install they describe, rooted at the capsules root, run before the capsules so they can resolve the core aspects it links and nothing races the root's Verified:
Needs a decision:
|
| Count | Failure | Notes |
|---|---|---|
| 2 | global-virtual-store — comp1/dist/index.js not written to the package dir; TSCompiler failure in bit tag --build |
not investigated |
| 1 | root-components — react-dom nested in comp4 |
pnpm regression, filed as pnpm/pnpm#13775 |
| 1 | bit-import-on-lanes — failed loading env |
same family as the bit_pr failure below |
| 1 | build-cmd — Cannot read properties of undefined (reading 'files') |
likely a cascade from an earlier failed step |
pnpm/pnpm#13775 (the react-dom one): with dedupeDirectDeps, a project's symlink that becomes redundant is not removed on a later install. Reduced to a plain-pnpm fixture, and it is a regression in 12 — 10.34.5 and 11.21.0 both remove it, 12.0.0-rc.3 does not. A clean install of the identical final state dedupes correctly, so only the incremental path is affected. Nothing to change in bit; this branch's two-phase install just exposes it.
Unrelated to these fixes
bit_pr has failed on every pipeline of this branch (12+ runs, well before the merge), currently:
teambit.mdx/aspect-docs/mdx@…
failed loading env (run "bit install"): bitdev.general/envs/js-env@0.0.8
Also note one CI run showed 63 failures rather than 20 — 44 of those were a registry incident (No matching version found … node-registry.bit.cloud), not regressions.
🤖 Generated with Claude Code
…as a loadable env again The suite asserts that an old-format env's dependency policy is missing after the first install and applied after the second. It skips setCustomEnv's install because it measures what each install does - which used to be fine, since the envs the fixture needs were core aspects that were always present. They are packages now, so with nothing installed the env cannot load at *any* point and the policy is never applied: the second-install and recurring-install assertions all fail. The chain fails one link at a time - @teambit/node first, then teambit.react/react, then teambit.harmony/aspect - so the fixture's own imports are only the first of what loading it needs. State the chain as a workspace policy rather than installing it up front. The first install then fetches it while the env is still unloadable - which is exactly what makes it an "old env" for the first-install assertions - and the second install finds it loadable. Installing the packages beforehand instead makes the env load on the first install and applies the policy there, which inverts the two first-install assertions. Not env.jsonc, though it would sidestep the chain: `calculateEnvManifest` reads the policy without loading the env, and `setOldNonLoadedEnvs` intersects with `envsWithoutManifest`, so a manifest would statically apply the deps on the first install and silence the warning - turning this suite into a duplicate of env-jsonc-policies.e2e.ts and dropping the only coverage of the legacy path. install.e2e.ts's old-envs suite: 2 passing/3 failing before, 5 passing after. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Update: the "needs a decision" item is resolved — 8 → 5 failures@zkochan asked whether env.jsonc could solve the old-envs suite. It can, and checking why led to a fix that keeps the original assertions intact — pushed in Why env.jsonc works but isn't the right lever here. What it did point at. The suite's premise is fine; the env simply had nothing to load from. The fix is to seed the legacy-core-env chain as a workspace policy instead of installing it up front:
My first attempt failed only because I seeded two packages (
Remaining 5
🤖 Generated with Claude Code |
…compiler (#10592) Closes #10591. ## The reported failure is not a global-virtual-store failure `global-virtual-store.e2e.ts` → *"should compile the component into its package directory"* failed on `remove-core-envs-from-manifest` (#10465). That branch sets `DEFAULT_ENV` to `teambit.harmony/empty-env` — an env that deliberately provides no compiler — and the test's component takes the default env, so nothing compiles and no `dist` is written. The store plays no part in it. Reproduced on master by pinning that same env, with the store both on and off — identical outcome: ``` COMPILE-OUTPUT-GVS: ✔ 0/0 components compiled successfully. dist exists: false COMPILE-OUTPUT-NO-GVS: ✔ 0/0 components compiled successfully. dist exists: false ``` For the other direction: build 436414 (#10589, a master-based branch that extends this same e2e file) is green across all 2933 tests, this one included. The build the report came from had 14 of 40 shards red, the rest of them on that branch's own env changes (`bit create react` failing, `Cannot find module './babel/babel-config'`). ## What is worth fixing: the run reports success `bit compile` counted only the components it ran a compiler on. A run where every component's env provides no compiler therefore ended in `✔ 0/0 components compiled successfully.`, exit code 0, with the skip recorded nowhere but the debug log. A 0/0 ratio counted as a success is indistinguishable from an empty workspace, so a missing `dist` looks like it was produced and lost rather than never produced — which is how this came to be filed against the store. Such a component is now carried in the results with a `skipped` reason, kept out of the compiled ratio, and named on its own line: ``` ⚠ my.scope/comp1 ... not compiled, the env "teambit.harmony/empty-env" provides no compiler ⚠ nothing was compiled. 1 component(s) have an env that provides no compiler. ``` Mixed with components that do compile: ``` ⚠ my.scope/comp1 ... not compiled, the env "teambit.harmony/empty-env" provides no compiler ✔ 1/1 components compiled successfully. 1 component(s) skipped, no compiler. ``` The ratio's denominator counts what was attempted, so it no longer stands in for what exists. Exit code is unchanged — a component with no compiler is not a failure. `bit watch`'s per-change output ignores skipped entries, as before. ## Also here The global-virtual-store e2e now names the env it needs (`teambit.harmony/node`) rather than taking whatever the default env is, so what that file covers stays the store layout and a default-env change cannot present itself as a store bug again. ## Testing - new e2e in `compile.e2e.ts`: a component on `empty-env` — output names the component and the env, and does not claim success - `global-virtual-store.e2e.ts` and the new describe run locally: 7 passing - `npm run lint` green - output checked by hand for all three shapes: all-skipped, mixed, and the unchanged all-compiled case 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
… compiler
The suite's last assertion is that the component compiles into its package
directory, but the component takes the default env - which on this branch is the
empty env, with no compiler - so no dist is ever written. The file landed on
master after this branch named an env explicitly in the e2e suites that assert a
dist, so it never got that treatment.
Use the workspace-local ts env rather than a published one: it installs nothing,
so what the global store does or does not serve is still exactly what the
assertions measure. `setBitdevNodeEnv` would install its env package and fails
to load under the global virtual store ("The requested module '@teambit/component'
does not provide an export named 'ComponentMap'"), which is a separate matter
from this suite's subject.
The "building an aspect" block still fails and is untouched: its TSCompiler
type-checks bit's own repo sources through the linked core aspects, against the
`@types/react@17` the pinned `@teambit/aspect@1.0.1042` env brings, so react 18
APIs (`startTransition`, `useSyncExternalStore`) come back as missing exports.
global-virtual-store.e2e.ts: 4 passing/2 failing before, 6 passing/1 failing after.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The suite deletes the component's dist artifact from the remote and asserts on the error that follows, so the tag has to record one. It did not: the ts env this branch put here compiles in the workspace but emits nothing in a capsule build - its TSCompiler task runs and reports success while the capsule ends up with no dist directory, so builder records only the schema and package-tar artifacts and `artifacts.find(a => a.name === 'dist')` is undefined. A workspace compile is what made that look fine elsewhere: it copies the files it cannot compile into the dist, so a dist appears there regardless. A build does not copy, so nothing appears. Use the bitdev node env instead, the same env this file already relies on for the --loose build below, and install it after the re-import rather than importing it - it is a published package, not a scope component. build-cmd.e2e.ts "dist file is deleted from the remote": the before hook threw on `undefined.files` before, 2 passing after. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…'s so the bridge is the only variable The aspect built here uses the pinned legacy aspect env, which brings older typings than this repo does - @types/react@17 against its 19, @types/mime@1 against its 2. The type-paths bridge maps the program's `react` to whatever the workspace hoisted, which is correct for the workspace's own packages; but bit's own sources share that program, because a core aspect in the dev repo exposes `types: index.ts` rather than a built `.d.ts`, and they need the repo's typings. So the compile failed on `startTransition` and `useSyncExternalStore` missing from react, and `getType` missing from mime - none of which is about the reachability this suite measures. The mismatch is broad (13 of the 19 typings packages the workspace and the repo share differ), so pinning per package as errors appear would be endless. Take the versions from the running repo instead, at runtime, so a repo-side bump cannot silently reintroduce the skew. Filtering the errors by file was the other option and would have been wrong: with the bridge stubbed out to return no paths, all 47 resulting errors land in bit's own sources - the same files the version skew shows up in. Ignoring them would leave a suite that passes with the bridge fully disabled. That the guard still bites was verified both ways: bridge on with these versions pinned is 0 errors, bridge off with them pinned is still 47. global-virtual-store.e2e.ts: 6 passing/1 failing before, 7 passing after. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… an aspect `createAspect` configures the component with the aspect env, which used to be a core aspect and is now a package. This suite is the one createAspect caller that also configures the aspect in workspace.jsonc, so bit has to load it - and with the env's package absent the load fails, the component is reported with a "failed loading env" issue, and the snap in the before hook throws before the import under test ever runs. The other callers scaffold an aspect without the workspace using it, so nothing forces the env to load and they are unaffected. bit-import-on-lanes.e2e.ts: the before hook threw before, 19 passing after. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ests after fixing the oom issue in pnpm side
… has not landed e7cffe7 dropped e2e_test and e2e_test_bbit back to medium on the grounds that the OOM was fixed on the pnpm side. The run it produced (#437962) says otherwise: 15 failures, 12 of them a bare `Killed`, 11 of those on the `bit install` of an env package - @teambit/react.react-env@1.3.5, @teambit/env + @teambit/node. 2793 tests reported instead of 2948, the missing 155 being suites whose before-hooks were killed before they could run. The premise has not been met yet. pnpm/pnpm#13681 - the engine still peaking about 2x pnpm v10, 4.4GB for a 3.8k-package graph - is open, so the peak does not fit in medium's 4GB whichever way the fix lands. This branch is also pinned to @pnpm/napi 12.0.0-rc.1, behind the rc.2/rc.3 that any such fix would ship in, so it could not have picked one up. Restored with the reasoning in the file so the next attempt starts from what to re-measure: once #13681 closes and the engine is bumped here, drop this and watch for `Killed`. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…aited on has not landed" This reverts commit 60b209a.
…tion spec The default-export describe builds its harmony without TypescriptAspect. On this branch the mock component gets the default empty-env, which provides no schema extractor; the fallback extractor is registered by the typescript aspect, so without it getSchema swallows the no-extractor error, falls back to a non-existent artifact and returns an empty schema - failing the test with "a `default` export should exist: expected undefined to exist". The two sibling describes in this file were already patched exactly this way (eedfdcd). This describe landed on master after that (#10378) and arrived via the 886f6a2 merge with no textual conflict, so the semantic conflict went unnoticed. Also assert schema availability in the before hook, so a future regression fails fast with the actual reason (e.g. NO_EXTRACTOR) instead of a puzzling empty-exports assertion. Verified: bit test teambit.semantics/schema - 39 passing / 1 failing before, 40/40 passing after.
The e2e OOM is not fixable on the pnpm side — measurementsAfter I went looking for the pnpm-side fix and could not find one. The measurements below are why. The engine improvements already landed, and are already in use
( The OOM predates the Rust engine entirelyFrom CircleCI history: job 430568, 2026-07-24, this same branch, commit Rate across the v12 merge: pre-v12 49/511 (9.6%), post-v12 7/48 (14.6%), Fisher exact p = 0.31. And 0 of 56 OOM-affected jobs were on master — it is a PR-branch phenomenon, with this branch accounting for 44 of them, well before v12 existed. The Rust engine is not a regression here. It uses less than the TypeScript one did. The install is not where the memory goesThe decisive measurement. Same command, same packages, same warm store:
Fetching and linking that closure costs ~400 MB. The other ~2.7 GB is what bit does around it, because the e2e workspace has components and a custom env — per-component dependency resolution, capsule isolation, aspect loading, compile. This matches an earlier profile on this branch where the peak was Reproduced under CI's exact budget ( Things that do not move it
What is actually leftThe workload is the driver, and it is one this branch creates: removing core envs turns envs into installed components, so suites that previously needed no install now install and process multi-thousand-package closures. That is why this branch dominates the OOM history both before and after v12. Levers, in the order I would take them:
CaveatsThe single-run comparisons in the 4 GB context are noisy — peak anon ranged 3452–3985 MB across arms, so no one pair of runs proves an effect. The conclusions above rest on things outside that noise: the source reading for 🤖 Generated with Claude Code |
Follow-up: why master doesn't OOM on the same 4 GBThe obvious objection to the above is "master runs the same suite and is fine". It does — on the same budget. Master's What changed is what a single helper call costs. // master
if (!options.skipInstall) this.command.install();
// this branch
this.command.install([ENVS_ENV_PACKAGE, ...this.getFixtureEnvBasePackages(extensionsBaseFolder)].join(' '));On master that The multiplier:
So the cost is not the ten extra explicit installs. It is that ~65 pre-existing call sites, untouched in the test files, silently became multi-thousand-package installs. Every suite that scaffolds a custom env now carries the ~3.1 GB profile measured above, where on master it carried almost none. That is consistent with everything else in this thread:
Worth stating the milder reading too: e2e may simply be reporting a genuine product change faithfully. A user upgrading to a bit without core envs pays this install cost once per workspace. This suite pays it ~65 times per run, inside 4 GB, alongside mocha and verdaccio. Which is the argument for fixing it in the helper or the resource class rather than in pnpm: the engine is being asked to make a workload fit that master never asked it to run. 🤖 Generated with Claude Code |
…waited on has not landed" This reverts commit 5967a30.
BabelMain.createCompiler() has zero callers in shipped app code - only in docs examples and e2e fixtures (babel-env, multiple-compilers-env), same shape as the earlier mocha removal. Fixed the two e2e fixtures to install @teambit/babel as a regular package (it's no longer a core aspect), matching how react/aspect/node were already handled.
setCustomEnv installs the env packages and then compiles. `bit install` compiles the workspace itself, so the workspace was compiled twice per call, and a workspace compile is the memory peak of an env-scaffolding test. Pass --skip-compile when the explicit compile is going to run anyway; when the caller asked for no compile, leave the install's compile alone as the only one. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The engine-side memory fix now exists: pnpm/pnpm#13844Following up on the OOM thread, and on Root causePeer resolution realizes one tree node per distinct root-to-package path. Replaying the exact options bit passes to Why so many occurrences: Children are only realized for a node that misses the peers cache. 91% of full walks end in a cycle, and a cycle-resolved walk caches nothing by design (pnpm#5108 — its truncated subtree makes its peer sets non-authoritative). So only 10,393 walks ever populate the cache,
What landed in the PRpnpm/pnpm#13844 cuts the per-occurrence cost without touching the algorithm:
3670 -> 3224 MB, -12% (n=3 each, non-overlapping ranges), no wall-time change, 314 resolver tests passing. What this means for
|
4 participants
Removes the env aspects (
teambit.react/react,teambit.harmony/node,teambit.harmony/aspect,teambit.envs/env,teambit.mdx/mdx,teambit.mdx/readme) from the core manifest to slim Bit. They now act like any other env, installed from the registry.New default env:
teambit.harmony/empty-env(core). A totally empty env - no compiler, no tester, no preview, no dependency policy. Components with no env configured use it and work fully offline out of the box (add → compile no-op → tag/snap → export). Since it has no behavior, it has nothing to drift when bit itself changes - the one env that is safe to keep core (and versionless in models) forever. To get a dev experience, users configure a real env (bit createflows already do).teambit.harmony/aspectandteambit.envs/envare removed like the rest, with zero behavior change. Their implementation is untouched (react-based, preview and all) - users get the exact released behavior afterbit install(the pinned-version machinery auto-installs them). New envs are created from the bitdev env packages (bit create react-envetc.), so these built-in envs are legacy surface. Thebit-aspecttemplate and the harmony starters moved to the core generator aspect, sobit create bit-aspectandbit newkeep working out of the box (the created aspect needsbit installbefore it loads, like any env).Versionless by design. Config entries for the removed env ids are persisted by name, without a version - exactly as they were when core (registered as core-extension names). Keeping them versionless is deliberate on two counts. First, it keeps the env from becoming a dependency edge of its own components; otherwise an env such as react, whose dependency closure includes components that use it as their env, creates circular TS project references and breaks lane/tag builds. Second, it preserves forward compatibility: a re-tag under the new bit keeps the env id versionless, so a teammate who has not upgraded yet (whose bit still ships these as core) can import the re-tagged component and resolve the env - instead of receiving a versioned id their bit has no component for. The alternative (showing the component as modified and pinning the env on the next tag) would silently break not-yet-upgraded consumers.
Backward compatibility. Old components have the removed envs saved without a version.
legacy-core-envs.tsmaps them to pinned versions, applied only at the resolution/loading/install level - stored objects are never mutated. Versionless legacy ids match the env slot ignoring version,bit installauto-adds their packages, and single-instance semantics are enforced (a loaded version is reused rather than loading another copy). Not-installed legacy envs fail fast with aNonLoadedEnvissue suggestingbit install- no scope-capsule isolation in workspace context (which used to take minutes). Old components load without being reported as modified, and re-tagging keeps the env versionless - covered end-to-end bye2e/harmony/legacy-core-env-back-compat.e2e.ts, which imports a component exported by a pre-removal bit (env saved versionless) and asserts it is not modified and stays versionless after a re-tag.Relocated core wiring: the
bit aspectCLI command moved toteambit.workspace/workspace;validateBeforePersistHookmoved toteambit.dependencies/dependency-resolver; the dead@teambit/legacylink is now skipped instead of crashing.Also fixes latent issues this path exposed: versionless seeders filtering out all manifests in
loadExtensionsByManifests, circular env chains causing infinite component-load recursion, versioned core-aspect ids escaping core filters anddoRequiremutating shared core manifests, stack overflows from recursive graph traversal, and a spuriousMissingDistsissue for compiler-less envs.Verified locally: fresh workspace (JS and TS components) - clean status in ~1s, tag/snap/export offline,
bit envs/bit testgraceful; this repo's workspace - status/insights/list-core clean; the seven repo components that relied on the default env are now explicitly set to the node env.bit create <template> --env <removed-env>loads the env's templates on demand from the global scope (pinned version); this path also loads the full manifest graph, and binds manifest deps of legacy envs to their pinned versions (models built when these envs were core don't list them as dependencies). The e2esetCustomEnvhelper installs the env package the fixture imports (e.g.@teambit/node).Also removes the former-core env sources from this repo's workspace (
scopes/harmony/node,scopes/react/react,scopes/harmony/aspect,scopes/envs/env,scopes/mdx/mdx,scopes/docs/readme) - bit now dogfoods them as installed packages like any consumer, and the source-vs-installed duality is gone. Making this pass end-to-end surfaced several general fixes that ride along:.docs.mdximports are detected even when the mdx aspect isn't loaded (latent gap once mdx is no longer core - without it, docs deps silently drop from dependency computation and preview bundling fails).Module._extensionsrequire hooks are restored after each build task. An in-process tester leaves@babel/register's pirates hook installed; the hook claims all.jsfiles (including node_modules, regardless of babelignoreconfig) and breaksrequire()of ESM-only packages in every later task in the process (pirates drops theformatarg node >=22.12 uses to routerequire(esm)).import()instead of a top-level require, immune to the same stale-hook hazard.@bit-no-check; timings manifest covers the split spec files so shard balancing accounts for the heavier env-install suites.