fix: run the compiler pipeline on a worker thread with a larger stack#13
Open
Flechazoie wants to merge 1 commit into
Open
fix: run the compiler pipeline on a worker thread with a larger stack#13Flechazoie wants to merge 1 commit into
Flechazoie wants to merge 1 commit into
Conversation
The compiler is recursive-descent throughout (parsing, AST->IR lowering, type inference), so stack usage grows linearly with expression length. tests/long_code2 (~4000 terms) already needs >5 MiB of the default 8 MiB main-thread stack; an 8000-term expression overflows it outright. Run the pipeline on a worker thread with a 256 MiB stack instead -- the same technique rustc uses to sidestep deep-recursion stack overflows. The reservation is virtual memory only; pages are committed on demand.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
teacis recursive-descent throughout: pest pair → AST conversion, AST → IR lowering, and type inference all recurse once per nesting level of the source expression. A sum of N terms parses into a left-leaning tree of depth ≈ N, so stack usage grows linearly with expression length. With the default 8 MiB main-thread stack the compiler crashes on long (but perfectly legal) inputs:Reproduction (current
main, aarch64 macOS, debug build)The in-tree stress test is already close to the cliff:
tests/long_code2(a ~4000-term sum) needs more than 5 MiB of the 8 MiB budget — lowering the limit makes it crash:Doubling the same pattern crosses it outright:
Since the assignments (asmt-2/3/4) all add code to these recursive paths (extra match arms, type coercion calls, float lowering), student forks consume even more stack per level and are likely to hit this on the stock test suite.
Fix
Run the whole pipeline on a worker thread with a generous (256 MiB) stack — the same technique rustc itself uses to sidestep deep-recursion stack overflows. The reservation is virtual memory only; pages are committed on demand, so the cost is one extra thread at startup and no measurable change in compile time.
Alternatives considered
stacker::maybe_grow) — adds a dependency and requires guarding every recursion site; easy to miss one.main.rs, no behavioral change otherwise, and the constant is trivially adjustable.Testing
cargo test: 30/30 pass on aarch64 macOS (no behavior change).