Skip to content

IC: share CacheCounter state between nim m processes via a file lock … - #26271

Open
Araq wants to merge 10 commits into
develfrom
araq-ic-lockfiles
Open

Araq wants to merge 10 commits into
develfrom
araq-ic-lockfiles

Conversation

@Araq

@Araq Araq commented Sep 24, 2026

Copy link
Copy Markdown
Member

…(#26201)

Sibling modules are compiled by separate, possibly parallel nim m processes, so an in-memory counter let both allocate from the same initial state and embed duplicate values. The counters now live in <nimcache>/ic.counters, guarded by an OS file lock that a process holds from its first counter operation until it is done with its module.

The file records, per counter, the high-water mark and the numbers each module was handed: a re-semmed module gets the same numbers again (no-op rebuilds stay content-stable), a module needing more gets fresh ones above the high-water mark and never collides with values cached modules embed.

Araq and others added 10 commits September 24, 2026 09:07
…26201)

Sibling modules are compiled by separate, possibly parallel `nim m`
processes, so an in-memory counter let both allocate from the same initial
state and embed duplicate values. The counters now live in
`<nimcache>/ic.counters`, guarded by an OS file lock that a process holds
from its first counter operation until it is done with its module.

The file records, per counter, the high-water mark and the numbers each
module was handed: a re-semmed module gets the same numbers again (no-op
rebuilds stay content-stable), a module needing more gets fresh ones above
the high-water mark and never collides with values cached modules embed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Field uses resolve against their owning type via one `loadFieldUse`
(a local `lookupInRecord`/`lookupFieldAgain`, since ast2nif cannot import
astalgo/types) and fall back to the stub instead of raising. The field of
a constructor's `f: v` now resolves against the constructor's type too, so
`nkObjConstr`/`nkTupleConstr` share PSym identity with the reclist.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`writeBackendActions`/`applyBackendActions` take the module's `.c` path and
derive the `.cflags` sidecar name themselves, so `localpassc` needs no path
field at all: it applies to the C file the sidecar belongs to. Removes the
suffix-stripping of the sidecar path.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Emit the wrapper for every type definition, even without sons, instead of
only for six structural kinds: 4 bytes per type give every BIF consumer an
unambiguous arity. The loader now requires the section; the format stamp
already rules out older caches, so the raw-tail fallback is gone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- sigmatch: a converter returning `var T` from a loaded module flagged its
  sealed return type with tfVarIsPtr; flag a private copy instead.
- transf: treat nkReplayAction as a leaf; its payload (e.g. a macrocache
  value holding an untyped proc AST) is data, not code.
- cgen: a routine owned by a module with `{.localPassC: "-m...".}` that is
  emitted into another module's TU (generic instance, emit-everywhere copy)
  is compiled for the owner's target via `#pragma GCC target` (clang:
  `#pragma clang attribute`). ast2nif.loadedReplayActions reads the owner's
  replay actions lazily.
- vm: getImpl of a loaded routine rebuilds its params as symbols, not
  identifiers, so macros can query their types (shady's toShader).
- icconfig: the precompiled config now carries lazyPaths/nimblePaths, so
  `nim ic` resolves nimble packages (IcConfigVersion 3).
- ast: skipGenericOwner tolerates a backend-lifted hook without owner.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The iface cookie hashes signatures only, so a body edit re-sems no importer.
But the backend inlines foreign bodies into other modules: an inline proc
into every using TU (`cg`), an inline iterator into the using module's
lowered NIF (`lower`). Those rules listed only their own module's NIF, so
the users kept the old body and `nim ic` diverged from `nim c`.

Each `lower`/`cg` run now records the modules whose routine bodies it read
(`ModuleGraph.icBodyDeps` via `getBody`; `BModule.icImplMods`, now including
inline procs) in a `.bodydeps` sidecar, and deps.nim lists their `.s.bif`
resp. `.t.bif` as inputs of the rule on the next run, like a `-MD` depfile.

The metamorphic harness treats the backend build file as provenance, like
the frontend one: it carries the discovered dependencies a clean build has
not seen yet.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants