diff --git a/CHANGELIST.md b/CHANGELIST.md index 07e9338e46..20569fd33c 100644 --- a/CHANGELIST.md +++ b/CHANGELIST.md @@ -158,6 +158,7 @@ A public site and service for the `[tune]` ecosystem: upload a tuned box's sidec - **Position-based liveness** (#3744, #3746) - every expression gets a preorder frame position and locals carry integer liveness intervals, replacing the source-range gate that made eight families of scopes GC-invisible (class-ctor `self`, comprehension accumulators, inliner tails, one-line blocks - locals were swept while alive); `frame_position()` is new das surface - **Attributable frames only** (#3739) - inline splices stamp host call-site positions, functions reaching `heap_collect` are statically denied fastcall, the collector refuses chains it cannot attribute, and `Context::fastCallDepth` is deleted (C++ ABI break; the hot dispatch path gets lighter) - **Scratch opt-out** (#3745) - `very_safe_context` gains per-container `set_scratch` so trusted internal buffers free eagerly on growth instead of deferring to GC +- **Opt-in fastcall depth guard** - `options max_fast_call_depth = N` / `CodeOfPolicies::max_fast_call_depth` turns unbounded fastcall recursion from a native stack overflow into a `recover`-able panic: a second `FastCallChecked` node family (fused one- and two-argument shapes included) is emitted only when the cap is set, so unprotected programs run the same nodes as before; a recovered panic restores the counter at every catch point (C++ ABI: `Context` gains two trailing members, `fastCallDepth` - the name #3739 deleted, back as the guard's counter - and `maxFastCallDepth`; `DAS_POLICIES_VERSION` moves to 2 because the new policy field lands in tail padding); found on the way: AOT `TTable::moveT` skipped `tombstones`, so a table moved by value could rehash mid-iteration on stack garbage - it moves the whole header now #### Automatic Inlining (#3389, #3393, #3396, #3441, #3445, #3462) diff --git a/daslib/ARCHITECTURE.md b/daslib/ARCHITECTURE.md index 821f0a7e7d..bc4ed65a38 100644 --- a/daslib/ARCHITECTURE.md +++ b/daslib/ARCHITECTURE.md @@ -4,7 +4,7 @@ Design rationale not evident in code; numbered module sections are anchored to s Companion documents carry separate concerns. - `ARCHITECTURE_LINT.md` - sec. 1-4: perf_lint, lint_config, lint, style_lint. -- `ARCHITECTURE_EMIT.md` - sec. 5-7, 28-29: aot_cpp, aot_standalone, flatten, the shader rails. +- `ARCHITECTURE_EMIT.md` - sec. 5-6: aot_cpp, aot_standalone; `ARCHITECTURE_SHADER.md` - sec. 7, 28-29: flatten, the shader rails. - `ARCHITECTURE_CAPI.md` - sec. 30: c_api_header, the C surface both backends emit. - `ARCHITECTURE_LINQ.md` - sec. 11-17, 33, 37: the linq family, sql_linq, sql_migrate. - `ARCHITECTURE_CURSOR.md` - sec. 40: ast_cursor, the cursor module the LSP and MCP navigation tools share. diff --git a/daslib/ARCHITECTURE_EMIT.md b/daslib/ARCHITECTURE_EMIT.md index 3acd9108f9..7f809ea630 100644 --- a/daslib/ARCHITECTURE_EMIT.md +++ b/daslib/ARCHITECTURE_EMIT.md @@ -1,4 +1,4 @@ -# daslib architecture notes - emission: AOT C++, standalone contexts, shaders +# daslib architecture notes - emission: AOT C++, standalone contexts Companion to `ARCHITECTURE.md` in this folder; section numbers are unique across the family. @@ -16,6 +16,12 @@ Companion to `ARCHITECTURE.md` in this folder; section numbers are unique across nothing when its destructor fires during an unwind (`include/daScript/simulate/ARCHITECTURE.md#aot-finally-unwind`). The pair moves together; only `test_aot` on a `DAS_ENABLE_EXCEPTIONS` build fails on a mismatch. +- **A `try`/`recover` restores what the interpreter's handler restores**: the emitter writes + the pair as `das_try_recover(__context__, [&](){...}, [&](){...})`, whose catch path puts + back `fastCallDepth` before the recover body runs, then `abiArg`, `abiCMRES` and the stack + watermark after it - the same state `SimNode_TryCatch` and `jit_try_recover` restore + (`include/daScript/simulate/ARCHITECTURE.md#fastcall-depth-guard`). The three move together; + only `test_aot` over a capped program whose recursion stays interpreted fails on a mismatch. - **C++ identifier mangling**: `aotSuffixNameEx` prepends `_S`/`_E`/`_V`/`_f_` when a das name is a C++ keyword, holds a non-alnum char, or is `DELETE` (winnt.h). Structs and enums share ONE C++ namespace while daslang keeps separate tables, so `struct X` + @@ -69,11 +75,10 @@ Companion to `ARCHITECTURE.md` in this folder; section numbers are unique across module constructors `Module::require` earlier ones by name (`fio_core` takes `strings`, dasHV takes `rtti_core`). A module missing from the daslib list still registers, only in the dependencies-first pass that follows; a module added to the C++ side joins the list. - - **The AnnotationInfo table resets at the START of the debug-info dump, not its end.** The globals' `VarInfo`s are written after that dump and a handled global's info refers to an - `AnnotationInfo` by the name the dump minted, so clearing on the way out left `&` with nothing - after it. Only that walk reaches a global's annotation - `writeHandledAnnotations` iterates + `AnnotationInfo` by the name the dump minted, so clearing on the way out would leave `&` with + nothing after it. Only that walk reaches a global's annotation - `writeHandledAnnotations` iterates types, structs and functions. - **A member pointer is qualified with `aotModuleName`, never the raw module name.** The main module is unnamed, so `_module.name` is empty for every type a script declares itself, while @@ -160,7 +165,7 @@ Companion to `ARCHITECTURE.md` in this folder; section numbers are unique across `preVisitExprAddr` never asks. The dependency dump's second `CppAot` emits no expression and needs no table. - **Every used `[init]` is called from the ctor, whatever module declares it.** The TU holds - every used function of every module (below), so a required module's `[init]` has an AOT body + every used function of every module (above), so a required module's `[init]` has an AOT body like any other and needs no special case; the call order is the simulated context's own, read back through rtti. A `[no_aot]` one stays a collected emit error, because there is no body to call. An engine that registers itself from its modules - dasLLAMA's architecture registry is @@ -191,7 +196,7 @@ Companion to `ARCHITECTURE.md` in this folder; section numbers are unique across the set, because module constructors `Module::require` those by name (dasHV takes `rtti_core` this way). A default C++ module the program never reaches stays out, however the compiler loaded it: a macro module's `daslib/ast` brings `rtti_core` and `ast_core` - into the compiler, and a context that registered them ran two constructors and carried + into the compiler, and a context that registered them would run two constructors and carry their code for nothing. The pruned modules (`compile time only, not linked`) get no `aotRequire` include and no registration. `standaloneModuleRegistration` orders the C++ subset and ranks it: `DEFAULT_MODULE_ORDER` @@ -228,72 +233,3 @@ Companion to `ARCHITECTURE.md` in this folder; section numbers are unique across `#pragma once` and the required modules' `aotRequire` includes so it stands alone in an embedder TU; two DIFFERENT contexts' headers sharing a das dependency still cannot be included in one TU (the shared types have no per-type guards). - -## 7. flatten - -- **Predicated lowering carries one live-mask per exit flavor** - `__flat_live` for - return, a per-loop break mask (persists across unrolled copies) and continue mask - (re-minted per copy). A write's predicate ANDs every active mask plus the structural - predicate; a narrow term excludes its own mask so it self-cancels. An inlined callee - gets a fresh live mask and lowers with `ctx.loopMasks` moved OUT, so its break/continue - can never reach the caller's loops. -- **`flatten_preshade_cse` is a joint fixpoint, not a pipeline** - extraction, regroup, - CSE and alias elimination mutually enable each other; the `_preshader_`/`_cse_` counters - are owned by that loop and re-seeded from surviving suffixes (per-call numbering - re-mints a live name). -- **A CSE/regroup tally counts exactly the regions its rewrite can change** - a duplicate - counted where the rewrite cannot reach never drops below 2 and runs the fixpoint to its - iteration cap. -- **`__flat_ret` carries `safeWhenUninitialized` only while every write is a - self-referential select** - a lowering change that makes the bare-decl read observable - turns the flag into a real uninitialized read. -- **CSE is local value numbering over one converged basic block, and it is complete** - - pure subtrees keyed by `describe()`; value-stability = reads no reassigned name; a store - through index/field/swizzle destabilizes its base; an unrecognized node fails closed as - mutable-reading. Uniform duplicates route to the preshader. -- **The copy-prop/CSE walks stay O(size)** - one name-to-statement index, one structural - walk. A `string` materialized per `ExprVar` in a visitor callback breaks that: each - `describe()` allocates a string that lives to the end of the pass, so the walk goes - quadratic in heap bytes, not only in time. -- **`MutCollect` is what CSE trusts to say whether a name is stable, so it counts every - store spelling, not the one the lowering emits.** CSE treats a name outside its set as - constant for the whole block; a missed store is a shared subexpression across a mutation. - Copies are only the visible half - `<-` also zeroes its SOURCE, `:=` lowers to a - `builtin`clone`(dst, src)` CALL rather than an `ExprClone`, `++`/`+=` are their own - nodes, and a by-reference - argument writes with no assignment node anywhere. Hence the argument arm keys on the - callee's parameter type (non-const and `ref` or a ref type), not on a node kind. -- **`delete` on a container of `ExpressionPtr` frees the BUFFER, never the nodes** - - `delete array` frees the pointees only for das-heap `T`, and `Expression` is a - handled C++ type whose instances are not heap chunks at all (the - measurement: an `array` of das structs returns its pointees to `heap_bytes_allocated`, - an `array` returns only the buffer and the nodes surface in the exit GC - report). That is why `make_float_ctor`'s const-fold early return may leave its lanes - un-consumed while the ctor path `emplace`s them away, why every `unsafe { delete args }` - after it is sound over borrowed tree nodes, and why a struct field holding a borrowed - node needs no `@do_not_delete`. Node lifetime belongs to the AST GC: a lane the const - fold drops is unreachable and collected at the enclosing `ast_gc_guard`. -- **The whitelist admits value-returning primitives only.** `lower_stmt`'s fall-through arm - lowers an unrecognized statement for its lifted sub-lets and drops the statement itself, - which is correct exactly while every surviving call is pure - so `lift_expr` refuses a - whitelisted call that writes through a by-reference argument (`sincos`) rather than let - the drop delete the store. Predicating such a write would need per-out-param temps the - lowering does not own. - -## 28. shader_block_layout - -- **Two rails, deliberately separate** - the LAYOUT rail admits int64/uint64 as block - members (`compute_block_layout` special-cases them) while the ARITHMETIC rail rejects - 64-bit INT (`arith_width_ok` allows width 64 only for floats); `cpu_only_lattice_width` - keys both emitters' fail-closed diagnostic. - -## 29. shader_lingua_franca {#shader-lingua-franca} - -- **Every symbol is either an exact CPU mirror of its GPU semantics or a `[sideeffects]` - dummy every rail lowers by name** - the dummies return zero on the host, so a CPU replay - reproduces GPU semantics only for the real-bodied set. Unsigned overloads never fold into - signed twins (glslang picks the unsigned opcode). -- **A width-variant of a lowered-by-name symbol is one more overload here, never an emitter - arm.** `unpack8` carries `int16 -> byte2` and `uint16 -> ubyte2` beside the 32-bit pair; every - overload is the same `reinterpret` on the host and the same single `OpBitcast` on the SPIR-V - rail, so the emitter matches the name and reads the width off the operand type. diff --git a/daslib/ARCHITECTURE_SHADER.md b/daslib/ARCHITECTURE_SHADER.md new file mode 100644 index 0000000000..d14642fa15 --- /dev/null +++ b/daslib/ARCHITECTURE_SHADER.md @@ -0,0 +1,72 @@ +# daslib architecture notes - the shader rails: flatten, block layout, lingua franca + +Companion to `ARCHITECTURE.md` in this folder; section numbers are unique across the family. + +## 7. flatten + +- **Predicated lowering carries one live-mask per exit flavor** - `__flat_live` for + return, a per-loop break mask (persists across unrolled copies) and continue mask + (re-minted per copy). A write's predicate ANDs every active mask plus the structural + predicate; a narrow term excludes its own mask so it self-cancels. An inlined callee + gets a fresh live mask and lowers with `ctx.loopMasks` moved OUT, so its break/continue + can never reach the caller's loops. +- **`flatten_preshade_cse` is a joint fixpoint, not a pipeline** - extraction, regroup, + CSE and alias elimination mutually enable each other; the `_preshader_`/`_cse_` counters + are owned by that loop and re-seeded from surviving suffixes (per-call numbering + re-mints a live name). +- **A CSE/regroup tally counts exactly the regions its rewrite can change** - a duplicate + counted where the rewrite cannot reach never drops below 2 and runs the fixpoint to its + iteration cap. +- **`__flat_ret` carries `safeWhenUninitialized` only while every write is a + self-referential select** - a lowering change that makes the bare-decl read observable + turns the flag into a real uninitialized read. +- **CSE is local value numbering over one converged basic block, and it is complete** - + pure subtrees keyed by `describe()`; value-stability = reads no reassigned name; a store + through index/field/swizzle destabilizes its base; an unrecognized node fails closed as + mutable-reading. Uniform duplicates route to the preshader. +- **The copy-prop/CSE walks stay O(size)** - one name-to-statement index, one structural + walk. A `string` materialized per `ExprVar` in a visitor callback breaks that: each + `describe()` allocates a string that lives to the end of the pass, so the walk goes + quadratic in heap bytes, not only in time. +- **`MutCollect` is what CSE trusts to say whether a name is stable, so it counts every + store spelling, not the one the lowering emits.** CSE treats a name outside its set as + constant for the whole block; a missed store is a shared subexpression across a mutation. + Copies are only the visible half - `<-` also zeroes its SOURCE, `:=` lowers to a + `builtin`clone`(dst, src)` CALL rather than an `ExprClone`, `++`/`+=` are their own + nodes, and a by-reference + argument writes with no assignment node anywhere. Hence the argument arm keys on the + callee's parameter type (non-const and `ref` or a ref type), not on a node kind. +- **`delete` on a container of `ExpressionPtr` frees the BUFFER, never the nodes** - + `delete array` frees the pointees only for das-heap `T`, and `Expression` is a + handled C++ type whose instances are not heap chunks at all (the + measurement: an `array` of das structs returns its pointees to `heap_bytes_allocated`, + an `array` returns only the buffer and the nodes surface in the exit GC + report). That is why `make_float_ctor`'s const-fold early return may leave its lanes + un-consumed while the ctor path `emplace`s them away, why every `unsafe { delete args }` + after it is sound over borrowed tree nodes, and why a struct field holding a borrowed + node needs no `@do_not_delete`. Node lifetime belongs to the AST GC: a lane the const + fold drops is unreachable and collected at the enclosing `ast_gc_guard`. +- **The whitelist admits value-returning primitives only.** `lower_stmt`'s fall-through arm + lowers an unrecognized statement for its lifted sub-lets and drops the statement itself, + which is correct exactly while every surviving call is pure - so `lift_expr` refuses a + whitelisted call that writes through a by-reference argument (`sincos`) rather than let + the drop delete the store. Predicating such a write would need per-out-param temps the + lowering does not own. + +## 28. shader_block_layout + +- **Two rails, deliberately separate** - the LAYOUT rail admits int64/uint64 as block + members (`compute_block_layout` special-cases them) while the ARITHMETIC rail rejects + 64-bit INT (`arith_width_ok` allows width 64 only for floats); `cpu_only_lattice_width` + keys both emitters' fail-closed diagnostic. + +## 29. shader_lingua_franca {#shader-lingua-franca} + +- **Every symbol is either an exact CPU mirror of its GPU semantics or a `[sideeffects]` + dummy every rail lowers by name** - the dummies return zero on the host, so a CPU replay + reproduces GPU semantics only for the real-bodied set. Unsigned overloads never fold into + signed twins (glslang picks the unsigned opcode). +- **A width-variant of a lowered-by-name symbol is one more overload here, never an emitter + arm.** `unpack8` carries `int16 -> byte2` and `uint16 -> ubyte2` beside the 32-bit pair; every + overload is the same `reinterpret` on the host and the same single `OpBitcast` on the SPIR-V + rail, so the emitter matches the name and reads the width off the operand type. diff --git a/daslib/shader_lingua_franca.das b/daslib/shader_lingua_franca.das index 52370c01b7..eb72e05503 100644 --- a/daslib/shader_lingua_franca.das +++ b/daslib/shader_lingua_franca.das @@ -231,10 +231,10 @@ def public unpack8(x : int) : byte4 => unsafe(reinterpret(x)) def public unpack8(x : uint) : ubyte4 => unsafe(reinterpret(x)) -[arch(at="ARCHITECTURE_EMIT.md#shader-lingua-franca")] +[arch(at="ARCHITECTURE_SHADER.md#shader-lingua-franca")] def public unpack8(x : int16) : byte2 => unsafe(reinterpret(x)) -[arch(at="ARCHITECTURE_EMIT.md#shader-lingua-franca")] +[arch(at="ARCHITECTURE_SHADER.md#shader-lingua-franca")] def public unpack8(x : uint16) : ubyte2 => unsafe(reinterpret(x)) def public pack32(v : byte4) : int => unsafe(reinterpret(v)) diff --git a/doc/source/reference/language/options.rst b/doc/source/reference/language/options.rst index 60d62a6694..3b5616d088 100644 --- a/doc/source/reference/language/options.rst +++ b/doc/source/reference/language/options.rst @@ -175,6 +175,14 @@ Optimization - bool - false - Disables the fastcall optimization. + * - ``max_fast_call_depth`` + - int + - 0 + - Interpreter only. Caps how deep fastcall (frameless) calls may nest; past the cap the + call panics with ``stack overflow, max_fast_call_depth exceeded while calling + ``, which ``recover`` can catch, instead of exhausting the native stack. ``0`` + leaves fastcall unchecked and costs nothing. Calls through function pointers, lambdas + and class methods push a regular frame and are already bounded by ``stack``. -------------------- Memory diff --git a/doc/source/stdlib/handmade/structure_annotation-rtti-CodeOfPolicies.rst b/doc/source/stdlib/handmade/structure_annotation-rtti-CodeOfPolicies.rst index 6e7d704aef..d13ce38ef2 100644 --- a/doc/source/stdlib/handmade/structure_annotation-rtti-CodeOfPolicies.rst +++ b/doc/source/stdlib/handmade/structure_annotation-rtti-CodeOfPolicies.rst @@ -111,3 +111,4 @@ JIT size optimization level for compiled code (0-3). Path to shared library, which is used in JIT. Path to linker, which is used in JIT. compile_file from a script reads and refreshes the default module cache around this compile (keyed by the file, the running binary, the host arguments and these policies). +Interpreter only: caps how deep fastcall (frameless) calls may nest, panicking past the cap instead of exhausting the native stack; 0 leaves fastcall unchecked. Host-side counterpart of ``options max_fast_call_depth`` (the option overrides the policy). diff --git a/include/daScript/ast/ast_serializer.h b/include/daScript/ast/ast_serializer.h index 7b99fcf58a..d0d3ba405a 100644 --- a/include/daScript/ast/ast_serializer.h +++ b/include/daScript/ast/ast_serializer.h @@ -365,7 +365,7 @@ namespace das { AstSerializer & serializeModule ( Module & module, bool already_exists ); static constexpr uint32_t getVersion () { - return 220; // 220: (a type back-reference stays inside its module's record vs annotation numeric payloads retain 64 bits) + return 221; // 221: max_fast_call_depth joins the module-cache policy stream } void serializeProgram ( ProgramPtr program, ModuleGroup & libGroup ) noexcept; diff --git a/include/daScript/simulate/ARCHITECTURE.md b/include/daScript/simulate/ARCHITECTURE.md index 3219558858..295d2796bb 100644 --- a/include/daScript/simulate/ARCHITECTURE.md +++ b/include/daScript/simulate/ARCHITECTURE.md @@ -98,6 +98,38 @@ to have finished - a capture macro's check that a captured `JobStatus` was relea again inside a destructor, which is `noexcept`, and the process dies in `std::terminate`. The guard costs nothing where exceptions are off. +## The fastcall depth guard {#fastcall-depth-guard} + +A fastcall function (`Function::fastCall`, decided in `src/ast/ast_allocate_stack.cpp`, repo +root) pushes no das stack frame: its body is a single expression evaluated straight from the +caller's node, so a call chain of fastcall functions grows only the native C++ stack. A +regular call fails cleanly at `stack.push` with `stack overflow while calling` once the das +stack is spent; unbounded fastcall recursion runs to the native guard page instead, which is a +process crash no `recover` sees. The guard is `CodeOfPolicies::max_fast_call_depth` (also +`options max_fast_call_depth`), mirrored into `Context::maxFastCallDepth` by +`Program::simulate` and `Context::setup`, and copied into every clone. When it is nonzero, +`Function::makeSimNode` (`src/ast/ast_simulate.cpp`, repo root) emits +`SimNode_FastCallChecked` in place of `SimNode_FastCall`: the same node with +`Context::enterCheckedFastCall` before the body - increment `fastCallDepth`, panic past the cap +- and a decrement after. The fused one- and two-argument shapes are a second family registered +under `"FastCallChecked"` (`src/simulate/simulate_fusion_call1.cpp` / `call2.cpp`, repo root), +so a checked call keeps its superinstructions; the unchecked `"FastCall"` family never gains +the counter, and a program with the cap at zero emits only the unchecked `FastCall` nodes - +the guard costs the unprotected call path nothing. + +The counter is a plain integer the panic path does not unwind: a panic is a longjmp in a +build without C++ exceptions, so every frame between the throw and the handler skips its +decrement. Each handler that restores `abiArg` after a caught panic - `SimNode_TryCatch`, its +debugger twin, `das_try_recover` for AOT, `jit_try_recover` (`src/builtin/jit_runtime.cpp`, +repo root) and the `evalWithCatch` / `runWithCatch` family +(`src/simulate/simulate_exceptions.cpp`, repo root) - restores `fastCallDepth` to the value it +held at the `try`, and `Context::restart` zeroes it, so a recovered overflow leaves no drift. +Calls that resolve at runtime - function pointers, lambdas, class methods through +`SimNode_InvokeFn` and its kin - push a regular frame and are bounded by the das stack +already; `Context::callOrFastcall`, the entry AOT and JIT code use to call back into an +interpreted function, stays frameless and uncounted, and the native code around it has no +guard of its own either. + ## Sanctioned hot-path additions The ledger the checklist's hot-path rules route to. Each entry: what was added, where, why @@ -138,6 +170,17 @@ correctness required it, and the alternative that was rejected. the odd bit of `trunc(y)` shifted to bit 31, anded with x. `pow_est` is the same without the xor, which is `GLSLstd450.Pow` - undefined for a negative base, as GLSL leaves it. +- **The fastcall depth counter's save and restore at every panic handler** - one 32-bit load + of `Context::fastCallDepth` at the entry of `SimNode_TryCatch::eval`, its debugger twin, + `das_try_recover` (`src/simulate/simulate_exceptions.cpp`, repo root), `jit_try_recover` + (`src/builtin/jit_runtime.cpp`, repo root) and the `evalWithCatch` / `runWithCatch` family, + held across the `setjmp` or `try`, and one store on the catch path, cap set or not. + Correctness requires it because a panic is a longjmp that runs no frame's epilogue, so the + checked node's decrement is skipped for every frame between the throw and the handler, and + the handler is the only place the counter can be put back (`ARCHITECTURE.md#fastcall-depth-guard`). + Rejected alternatives: a save guarded on `maxFastCallDepth != 0` is a branch on the same + path for the same load; an RAII guard runs no destructor under longjmp. + - **`das_ordered2`** (`aot.h`) - a function the AOT emitter wraps around any binary op whose operands are not both side-effect-free. It takes the op and one thunk per operand, and runs the thunks in two declarations, which C++ sequences left-to-right; a plain call argument diff --git a/include/daScript/simulate/aot.h b/include/daScript/simulate/aot.h index 503153d690..73684e65e2 100644 --- a/include/daScript/simulate/aot.h +++ b/include/daScript/simulate/aot.h @@ -1361,6 +1361,7 @@ namespace das { flags = arr.flags; arr.flags = 0; keys = arr.keys; arr.keys = 0; hashes = arr.hashes; arr.hashes = 0; + tombstones = arr.tombstones; arr.tombstones = 0; } __forceinline TV & operator () ( const TK & key, Context * __context__ ) { TableHash thh(__context__,sizeof(TV)); @@ -1402,6 +1403,7 @@ namespace das { flags = arr.flags; arr.flags = 0; keys = arr.keys; arr.keys = 0; hashes = arr.hashes; arr.hashes = 0; + tombstones = arr.tombstones; arr.tombstones = 0; } }; diff --git a/include/daScript/simulate/code_of_policies.h b/include/daScript/simulate/code_of_policies.h index 5f3d2539ae..a0c4684ef0 100644 --- a/include/daScript/simulate/code_of_policies.h +++ b/include/daScript/simulate/code_of_policies.h @@ -7,7 +7,7 @@ namespace das { // bump when CodeOfPolicies changes layout or meaning without changing size — a field // reorder/repurpose, or an insert that lands in a padding pocket (sizeof can't see those) - #define DAS_POLICIES_VERSION 1 + #define DAS_POLICIES_VERSION 2 // per-binary linkage for the ABI-stamp functions: at -O0 gcc/clang emit them as weak // default-visibility symbols, and the dynamic linker would bind every binary to the FIRST @@ -171,7 +171,8 @@ namespace das { vector dll_search_paths; // additional search paths for dll loading // one-liners /*option*/ bool temp_table_lint_warning = false; - bool module_cache = false; // a script's compile_file runs under the default module cache; last: a cached JIT DLL binds earlier fields by offset + bool module_cache = false; // a script's compile_file runs under the default module cache + /*option*/ uint32_t max_fast_call_depth = 0; // interpreter only; 0 = unchecked. last: a cached JIT DLL binds earlier fields by offset // the abi_stamp truth (a member so the NSDMI above can call it - complete-class context). // Low byte 0 keeps a pre-stamp libDaScript reading this word seeing aot == false - diff --git a/include/daScript/simulate/simulate.h b/include/daScript/simulate/simulate.h index a09ea13f48..f24eba5db0 100644 --- a/include/daScript/simulate/simulate.h +++ b/include/daScript/simulate/simulate.h @@ -468,6 +468,7 @@ namespace das stopFlags = 0; exception = nullptr; last_exception = nullptr; + fastCallDepth = 0; } __forceinline void restartHeaps() { @@ -619,6 +620,12 @@ namespace das return result; } + __forceinline void enterCheckedFastCall ( const LineInfo & at, const SimFunction * fn ) { + if ( ++fastCallDepth > maxFastCallDepth ) { + throw_error_at(at, "stack overflow, max_fast_call_depth %u exceeded while calling %s", maxFastCallDepth, fn->mangledName); + } + } + DAS_EVAL_ABI __forceinline vec4f callOrFastcall(const SimFunction * fn, vec4f * args, LineInfo * line) { if ( fn->fastcall ) { auto aa = abiArg; @@ -945,6 +952,10 @@ namespace das vector forkContextPool; mutex forkContextPoolMutex; atomic forkContextsBorrowed{0}; // acquired and not yet released; the destructor waits for zero + public: + // include/daScript/simulate/ARCHITECTURE.md#fastcall-depth-guard + uint32_t fastCallDepth = 0; + uint32_t maxFastCallDepth = 0; //! fixed at simulate: 0 exactly when the program carries no guarded fastcall nodes }; struct DebugAgentInstance { diff --git a/include/daScript/simulate/simulate_nodes.h b/include/daScript/simulate/simulate_nodes.h index 0d58f9ad2e..00e3a56919 100644 --- a/include/daScript/simulate/simulate_nodes.h +++ b/include/daScript/simulate/simulate_nodes.h @@ -1379,6 +1379,82 @@ SIM_NODE_AT_VECTOR(Float, float) #undef EVAL_NODE }; + // FUNCTION CALL via FASTCALL convention, checked + + // include/daScript/simulate/ARCHITECTURE.md#fastcall-depth-guard + struct SimNode_FastCallCheckedAny : SimNode_CallBase { + SimNode_FastCallCheckedAny(const LineInfo& at) : SimNode_CallBase(at,"") {} + virtual SimNode* visit(SimVisitor& vis) override; + }; + + template + struct SimNode_FastCallChecked : SimNode_FastCallCheckedAny { + SimNode_FastCallChecked ( const LineInfo & at ) : SimNode_FastCallCheckedAny(at) {} + DAS_EVAL_ABI virtual vec4f eval ( Context & context ) override { + DAS_PROFILE_NODE + vec4f argValues[argCount ? argCount : 1]; + EvalBlock::eval(context, arguments, argValues); + context.enterCheckedFastCall(debugInfo, fnPtr); + auto aa = context.abiArg; + context.abiArg = argValues; + auto res = fnPtr->code->eval(context); + context.stopFlags &= ~(EvalFlags::stopForReturn | EvalFlags::stopForBreak | EvalFlags::stopForContinue); + context.abiArg = aa; + context.fastCallDepth --; + return res; + } +#define EVAL_NODE(TYPE,CTYPE)\ + virtual CTYPE eval##TYPE ( Context & context ) override { \ + DAS_PROFILE_NODE \ + vec4f argValues[argCount ? argCount : 1]; \ + EvalBlock::eval(context, arguments, argValues); \ + context.enterCheckedFastCall(debugInfo, fnPtr); \ + auto aa = context.abiArg; \ + context.abiArg = argValues; \ + auto res = EvalTT::eval(context, fnPtr->code); \ + context.stopFlags &= ~(EvalFlags::stopForReturn | EvalFlags::stopForBreak | EvalFlags::stopForContinue); \ + context.abiArg = aa; \ + context.fastCallDepth --; \ + return res; \ + } + DAS_EVAL_NODE +#undef EVAL_NODE + }; + + template <> + struct SimNode_FastCallChecked<-1> : SimNode_FastCallCheckedAny { + SimNode_FastCallChecked(const LineInfo& at) : SimNode_FastCallCheckedAny(at) {} + DAS_EVAL_ABI virtual vec4f eval(Context& context) override { + DAS_PROFILE_NODE + vec4f argValues[DAS_MAX_FUNCTION_ARGUMENTS]; + evalArgs(context, argValues); + context.enterCheckedFastCall(debugInfo, fnPtr); + auto aa = context.abiArg; + context.abiArg = argValues; + auto res = fnPtr->code->eval(context); + context.stopFlags &= ~(EvalFlags::stopForReturn | EvalFlags::stopForBreak | EvalFlags::stopForContinue); + context.abiArg = aa; + context.fastCallDepth --; + return res; + } +#define EVAL_NODE(TYPE,CTYPE)\ + virtual CTYPE eval##TYPE ( Context & context ) override { \ + DAS_PROFILE_NODE \ + vec4f argValues[DAS_MAX_FUNCTION_ARGUMENTS]; \ + evalArgs(context, argValues); \ + context.enterCheckedFastCall(debugInfo, fnPtr); \ + auto aa = context.abiArg; \ + context.abiArg = argValues; \ + auto res = EvalTT::eval(context, fnPtr->code); \ + context.stopFlags &= ~(EvalFlags::stopForReturn | EvalFlags::stopForBreak | EvalFlags::stopForContinue); \ + context.abiArg = aa; \ + context.fastCallDepth --; \ + return res; \ + } + DAS_EVAL_NODE +#undef EVAL_NODE + }; + // FUNCTION CALL struct SimNode_CallAny : SimNode_CallBase { diff --git a/skills/internal/daslang_internals.md b/skills/internal/daslang_internals.md index 57bf0408b9..f7fa67e197 100644 --- a/skills/internal/daslang_internals.md +++ b/skills/internal/daslang_internals.md @@ -25,8 +25,11 @@ asymmetry is the design. ## Options that are not really options A `CodeOfPolicies` field is settable from `options` ONLY if its declaration in -`include/daScript/ast/ast.h` carries the `/*option*/` marker. Without it the field exists, is -bound to rtti, and reads plausibly in docs, but `options that_field = true` in a `.das` silently +`include/daScript/simulate/code_of_policies.h` carries the `/*option*/` marker - the marker +records that the C++ use site reads `options.get*Option("", policies.)`. Without +it the field exists, is bound to rtti (which is what makes the option NAME valid: +`getCodeOfPolicyOptions` in `src/builtin/module_builtin_rtti.cpp` lists every bound workhorse +field), and reads plausibly in docs, but `options that_field = true` in a `.das` silently does nothing - only a C++ embedder setting the policy can reach it. Check the marker before believing an option is live: a policy-gated pass with no marker has effectively never run. diff --git a/src/ast/ast_simulate.cpp b/src/ast/ast_simulate.cpp index 69fcc73b09..bae9efb7a7 100644 --- a/src/ast/ast_simulate.cpp +++ b/src/ast/ast_simulate.cpp @@ -628,6 +628,10 @@ namespace das if ( copyOnReturn || moveOnReturn ) { return context.code->makeNodeUnrollAny(int(arguments.size()), at); } else if ( fastCall ) { + // include/daScript/simulate/ARCHITECTURE.md#fastcall-depth-guard + if ( context.maxFastCallDepth ) { + return context.code->makeNodeUnrollAny(int(arguments.size()), at); + } return context.code->makeNodeUnrollAny(int(arguments.size()), at); } else { return context.code->makeNodeUnrollAny(int(arguments.size()), at); @@ -3689,6 +3693,7 @@ namespace das context.failed = true; context.verySafeContext = options.getBoolOption("very_safe_context",policies.very_safe_context); context.maxUnreservedSize = options.getUInt64Option("max_unreserved_size", policies.max_unreserved_size); + context.maxFastCallDepth = uint32_t(options.getIntOption("max_fast_call_depth", int32_t(policies.max_fast_call_depth))); astTypeInfo.clear(); // this is to be filled via typeinfo(ast_typedecl and such) auto disableInit = options.getBoolOption("no_init", policies.no_init); context.thisProgram = this; diff --git a/src/builtin/jit_runtime.cpp b/src/builtin/jit_runtime.cpp index 0b1f8e2d7e..8545f3bf90 100644 --- a/src/builtin/jit_runtime.cpp +++ b/src/builtin/jit_runtime.cpp @@ -703,7 +703,7 @@ extern "C" { DAS_API void WIN_EH_NO_ASAN jit_try_recover ( Block * try_block, Block * catch_block, void * lineInfo, Context * context ) { auto at = (LineInfoArg *) lineInfo; - auto aa = context->abiArg; auto acm = context->abiCMRES; + auto aa = context->abiArg; auto acm = context->abiCMRES; auto fcd = context->fastCallDepth; char * EP, * SP; context->stack.watermark(EP,SP); #if DAS_ENABLE_EXCEPTIONS @@ -712,6 +712,7 @@ extern "C" { } catch ( const dasException & ) { context->abiArg = aa; context->abiCMRES = acm; + context->fastCallDepth = fcd; context->stack.pop(EP,SP); context->stopFlags = 0; context->last_exception = context->exception; @@ -728,6 +729,7 @@ extern "C" { context->throwBuf = JB; context->abiArg = aa; context->abiCMRES = acm; + context->fastCallDepth = fcd; context->stack.pop(EP,SP); context->stopFlags = 0; context->last_exception = context->exception; diff --git a/src/builtin/module_builtin_ast_serialize.cpp b/src/builtin/module_builtin_ast_serialize.cpp index 6a14a30495..579cacf4eb 100644 --- a/src/builtin/module_builtin_ast_serialize.cpp +++ b/src/builtin/module_builtin_ast_serialize.cpp @@ -3254,7 +3254,7 @@ namespace das { X(strict_properties) X(no_writing_to_nameless) X(no_optimizations) X(no_fold_unsafe) X(fast_math) X(disable_dse) \ X(disable_cse) X(disable_temp_string_reclaim) X(disable_inline) X(disable_auto_inline) \ X(auto_inline_functions) X(auto_inline_cost) X(disable_run) X(no_infer_time_folding) \ - X(fail_on_no_aot) X(fail_on_lack_of_aot_export) X(no_fast_call) X(fusion) X(scoped_stack_allocator) \ + X(fail_on_no_aot) X(fail_on_lack_of_aot_export) X(no_fast_call) X(max_fast_call_depth) X(fusion) X(scoped_stack_allocator) \ X(force_inscope_pod) X(log_inscope_pod) X(debugger) X(profiler) X(jit_enabled) \ X(jit_jit_all_functions) X(jit_debug_info) X(jit_opt_level) X(jit_size_level) X(jit_dll_mode) \ X(jit_output_path) X(jit_path_to_shared_lib) X(jit_path_to_linker) X(threadlock_context) \ diff --git a/src/builtin/module_builtin_rtti.cpp b/src/builtin/module_builtin_rtti.cpp index 83b55c2e5b..96783116d9 100644 --- a/src/builtin/module_builtin_rtti.cpp +++ b/src/builtin/module_builtin_rtti.cpp @@ -1089,6 +1089,7 @@ namespace das { addField("jit_path_to_shared_lib"); addField("jit_path_to_linker"); addField("module_cache"); + addField("max_fast_call_depth"); } virtual bool isLocal() const override { return true; } }; diff --git a/src/runtime/context.cpp b/src/runtime/context.cpp index 2d27b753fb..52d799a449 100644 --- a/src/runtime/context.cpp +++ b/src/runtime/context.cpp @@ -91,6 +91,7 @@ namespace das } verySafeContext = options.getBoolOption("very_safe_context",policies.very_safe_context); maxUnreservedSize = options.getUInt64Option("max_unreserved_size", policies.max_unreserved_size); + maxFastCallDepth = uint32_t(options.getIntOption("max_fast_call_depth", int32_t(policies.max_fast_call_depth))); breakOnException |= policies.debugger; gcEnabled = options.getBoolOption("gc", false); gcLogTime = options.getBoolOption("log_gc_time", policies.log_gc_time); @@ -299,6 +300,7 @@ namespace das ref_count_magic = TRACK_PTR_CONTEXT; verySafeContext = ctx.verySafeContext; maxUnreservedSize = ctx.maxUnreservedSize; + maxFastCallDepth = ctx.maxFastCallDepth; persistent = ctx.persistent; gcEnabled = ctx.gcEnabled; code = ctx.code; diff --git a/src/simulate/simulate_exceptions.cpp b/src/simulate/simulate_exceptions.cpp index 8954a768d0..ab2d8fdbeb 100644 --- a/src/simulate/simulate_exceptions.cpp +++ b/src/simulate/simulate_exceptions.cpp @@ -92,6 +92,7 @@ namespace das { auto aa = abiArg; auto acm = abiCMRES; auto atba = abiThisBlockArg; + auto fcd = fastCallDepth; char * EP, * SP; stack.watermark(EP,SP); vec4f vres = v_zero(); @@ -102,6 +103,7 @@ namespace das { abiArg = aa; abiCMRES = acm; abiThisBlockArg = atba; + fastCallDepth = fcd; stack.pop(EP,SP); exceptionMessage = ex.what(); exception = exceptionMessage.c_str(); @@ -117,6 +119,7 @@ namespace das { abiArg = aa; abiCMRES = acm; abiThisBlockArg = atba; + fastCallDepth = fcd; stack.pop(EP,SP); } throwBuf = JB; @@ -128,6 +131,7 @@ namespace das { auto aa = abiArg; auto acm = abiCMRES; auto atba = abiThisBlockArg; + auto fcd = fastCallDepth; char * EP, * SP; stack.watermark(EP,SP); bool bres = false; @@ -139,6 +143,7 @@ namespace das { abiArg = aa; abiCMRES = acm; abiThisBlockArg = atba; + fastCallDepth = fcd; stack.pop(EP,SP); exceptionMessage = ex.what(); exception = exceptionMessage.c_str(); @@ -155,6 +160,7 @@ namespace das { abiArg = aa; abiCMRES = acm; abiThisBlockArg = atba; + fastCallDepth = fcd; stack.pop(EP,SP); } throwBuf = JB; @@ -178,6 +184,7 @@ namespace das { auto aa = abiArg; auto acm = abiCMRES; auto atba = abiThisBlockArg; + auto fcd = fastCallDepth; char * EP, * SP; stack.watermark(EP,SP); vec4f vres = v_zero(); @@ -188,6 +195,7 @@ namespace das { abiArg = aa; abiCMRES = acm; abiThisBlockArg = atba; + fastCallDepth = fcd; stack.pop(EP,SP); exceptionMessage = ex.what(); exception = exceptionMessage.c_str(); @@ -203,6 +211,7 @@ namespace das { abiArg = aa; abiCMRES = acm; abiThisBlockArg = atba; + fastCallDepth = fcd; stack.pop(EP,SP); } throwBuf = JB; @@ -214,7 +223,7 @@ namespace das { vec4f WIN_EH_NO_ASAN SimNode_TryCatch::eval ( Context & context ) { DAS_PROFILE_NODE - auto aa = context.abiArg; auto acm = context.abiCMRES; + auto aa = context.abiArg; auto acm = context.abiCMRES; auto fcd = context.fastCallDepth; char * EP, * SP; context.stack.watermark(EP,SP); #if DAS_ENABLE_EXCEPTIONS @@ -223,6 +232,7 @@ namespace das { } catch ( const dasException & ) { context.abiArg = aa; context.abiCMRES = acm; + context.fastCallDepth = fcd; context.stack.pop(EP,SP); context.stopFlags = 0; context.last_exception = context.exception; @@ -239,6 +249,7 @@ namespace das { context.throwBuf = JB; context.abiArg = aa; context.abiCMRES = acm; + context.fastCallDepth = fcd; context.stack.pop(EP,SP); context.stopFlags = 0; context.last_exception = context.exception; @@ -253,7 +264,7 @@ namespace das { #if DAS_DEBUGGER vec4f WIN_EH_NO_ASAN SimNodeDebug_TryCatch::eval ( Context & context ) { DAS_PROFILE_NODE - auto aa = context.abiArg; auto acm = context.abiCMRES; + auto aa = context.abiArg; auto acm = context.abiCMRES; auto fcd = context.fastCallDepth; char * EP, * SP; context.stack.watermark(EP,SP); #if DAS_ENABLE_EXCEPTIONS @@ -263,6 +274,7 @@ namespace das { } catch ( const dasException & ) { context.abiArg = aa; context.abiCMRES = acm; + context.fastCallDepth = fcd; context.stack.pop(EP,SP); context.stopFlags = 0; context.last_exception = context.exception; @@ -281,6 +293,7 @@ namespace das { context.throwBuf = JB; context.abiArg = aa; context.abiCMRES = acm; + context.fastCallDepth = fcd; context.stack.pop(EP,SP); context.stopFlags = 0; context.last_exception = context.exception; @@ -295,13 +308,14 @@ namespace das { #endif void WIN_EH_NO_ASAN das_try_recover ( Context * __context__, const callable & try_block, const callable & catch_block ) { - auto aa = __context__->abiArg; auto acm = __context__->abiCMRES; + auto aa = __context__->abiArg; auto acm = __context__->abiCMRES; auto fcd = __context__->fastCallDepth; char * EP, * SP; __context__->stack.watermark(EP,SP); #if DAS_ENABLE_EXCEPTIONS try { try_block(); } catch ( const dasException & ) { + __context__->fastCallDepth = fcd; catch_block(); __context__->abiArg = aa; __context__->abiCMRES = acm; @@ -317,6 +331,7 @@ namespace das { if ( !setjmp(ev) ) { try_block(); } else { + __context__->fastCallDepth = fcd; catch_block(); __context__->throwBuf = JB; __context__->abiArg = aa; diff --git a/src/simulate/simulate_fusion_call1.cpp b/src/simulate/simulate_fusion_call1.cpp index ae0e2f5277..3237b9a2aa 100644 --- a/src/simulate/simulate_fusion_call1.cpp +++ b/src/simulate/simulate_fusion_call1.cpp @@ -122,6 +122,36 @@ __forceinline SimNode * safeArg1 ( SimNode * node, int index ) { IMPLEMENT_ANY_OP1_FUSION_POINT(__forceinline,FastCall,,vec4f,vec4f) +/* FastCallChecked op1 */ + +#undef IMPLEMENT_ANY_OP1_NODE +#define IMPLEMENT_ANY_OP1_NODE(INLINE,OPNAME,TYPE,CTYPE,RCTYPE,COMPUTE) \ + struct SimNode_Op1##COMPUTE : SimNode_Op1Call1 { \ + NO_ASAN_INLINE vec4f compute(Context & context) { \ + DAS_PROFILE_NODE \ + vec4f argValues[1]; \ + argValues[0] = v_zero(); \ + memcpy(&argValues[0], subexpr.compute##COMPUTE(context), loadSize); \ + context.enterCheckedFastCall(debugInfo, fnPtr); \ + auto aa = context.abiArg; \ + context.abiArg = argValues; \ + auto res = fnPtr->code->eval(context); \ + context.stopFlags &= ~(EvalFlags::stopForReturn | EvalFlags::stopForBreak | EvalFlags::stopForContinue); \ + context.abiArg = aa; \ + context.fastCallDepth --; \ + return res; \ + } \ + DAS_EVAL_ABI virtual vec4f eval ( Context & context ) override { \ + return compute(context); \ + } \ + DAS_EVAL_NODE \ + }; + +#include "daScript/simulate/simulate_fusion_op1_impl.h" +#include "daScript/simulate/simulate_fusion_op1_perm.h" + + IMPLEMENT_ANY_OP1_FUSION_POINT(__forceinline,FastCallChecked,,vec4f,vec4f) + /* Call op1 */ #undef IMPLEMENT_ANY_OP1_NODE @@ -148,6 +178,7 @@ __forceinline SimNode * safeArg1 ( SimNode * node, int index ) { void createFusionEngine_call1() { (*getFusionEngine())["FastCall"].emplace_back(new Op1FusionPoint_FastCall_vec4f()); + (*getFusionEngine())["FastCallChecked"].emplace_back(new Op1FusionPoint_FastCallChecked_vec4f()); (*getFusionEngine())["Call"].emplace_back(new Op1FusionPoint_Call_vec4f()); } } diff --git a/src/simulate/simulate_fusion_call2.cpp b/src/simulate/simulate_fusion_call2.cpp index 149af6121e..fcc4d47a85 100644 --- a/src/simulate/simulate_fusion_call2.cpp +++ b/src/simulate/simulate_fusion_call2.cpp @@ -287,10 +287,93 @@ IMPLEMENT_ANY_OP2(__forceinline, CallAndCopyOrMove, Ptr, StringPtr) IMPLEMENT_ANY_OP2(__forceinline, FastCall, Ptr, StringPtr) +/* FastCallChecked */ + +// OP(COMPUTEL,*) +#undef IMPLEMENT_OP2_NODE_ANYR +#define IMPLEMENT_OP2_NODE_ANYR(INLINE,OPNAME,TYPE,CTYPE,COMPUTEL) \ + struct SimNode_##OPNAME##_Any_##COMPUTEL : SimNode_Op2Call2 { \ + NO_ASAN_INLINE auto compute ( Context & context ) { \ + DAS_PROFILE_NODE \ + vec4f argValues[2]; \ + argValues[0] = l.subexpr->eval(context); \ + argValues[1] = v_zero(); \ + memcpy(&argValues[1], r.compute##COMPUTEL(context), rightLoadSize); \ + context.enterCheckedFastCall(debugInfo, fnPtr); \ + auto aa = context.abiArg; \ + context.abiArg = argValues; \ + auto res = fnPtr->code->eval(context); \ + context.stopFlags &= ~(EvalFlags::stopForReturn | EvalFlags::stopForBreak | EvalFlags::stopForContinue); \ + context.abiArg = aa; \ + context.fastCallDepth --; \ + return res; \ + } \ + DAS_EVAL_ABI virtual vec4f eval ( Context & context ) override { \ + return compute(context); \ + } \ + DAS_EVAL_NODE \ + }; + +// OP(*,COMPUTER) +#undef IMPLEMENT_OP2_NODE_ANYL +#define IMPLEMENT_OP2_NODE_ANYL(INLINE,OPNAME,TYPE,CTYPE,COMPUTER) \ + struct SimNode_##OPNAME##_##COMPUTER##_Any : SimNode_Op2Call2 { \ + NO_ASAN_INLINE auto compute ( Context & context ) { \ + DAS_PROFILE_NODE \ + vec4f argValues[2]; \ + argValues[0] = v_zero(); \ + memcpy(&argValues[0], l.compute##COMPUTER(context), leftLoadSize); \ + argValues[1] = r.subexpr->eval(context); \ + context.enterCheckedFastCall(debugInfo, fnPtr); \ + auto aa = context.abiArg; \ + context.abiArg = argValues; \ + auto res = fnPtr->code->eval(context); \ + context.stopFlags &= ~(EvalFlags::stopForReturn | EvalFlags::stopForBreak | EvalFlags::stopForContinue); \ + context.abiArg = aa; \ + context.fastCallDepth --; \ + return res; \ + } \ + DAS_EVAL_ABI virtual vec4f eval ( Context & context ) override { \ + return compute(context); \ + } \ + DAS_EVAL_NODE \ + }; + +// OP(COMPUTEL,COMPUTER) +#undef IMPLEMENT_OP2_NODE +#define IMPLEMENT_OP2_NODE(INLINE,OPNAME,TYPE,CTYPE,COMPUTEL,COMPUTER) \ + struct SimNode_##OPNAME##_##COMPUTEL##_##COMPUTER : SimNode_Op2Call2 { \ + NO_ASAN_INLINE auto compute ( Context & context ) { \ + DAS_PROFILE_NODE \ + vec4f argValues[2]; \ + argValues[0] = v_zero(); \ + memcpy(&argValues[0], l.compute##COMPUTEL(context), leftLoadSize); \ + argValues[1] = v_zero(); \ + memcpy(&argValues[1], r.compute##COMPUTER(context), rightLoadSize); \ + context.enterCheckedFastCall(debugInfo, fnPtr); \ + auto aa = context.abiArg; \ + context.abiArg = argValues; \ + auto res = fnPtr->code->eval(context); \ + context.stopFlags &= ~(EvalFlags::stopForReturn | EvalFlags::stopForBreak | EvalFlags::stopForContinue); \ + context.abiArg = aa; \ + context.fastCallDepth --; \ + return res; \ + } \ + DAS_EVAL_ABI virtual vec4f eval ( Context & context ) override { \ + return compute(context); \ + } \ + DAS_EVAL_NODE \ + }; + +#include "daScript/simulate/simulate_fusion_op2_impl.h" + +IMPLEMENT_ANY_OP2(__forceinline, FastCallChecked, Ptr, StringPtr) + void createFusionEngine_call2() { (*getFusionEngine())["Call"].emplace_back(new FusionPoint_Call_StringPtr()); (*getFusionEngine())["CallAndCopyOrMove"].emplace_back(new FusionPoint_CallAndCopyOrMove_StringPtr()); (*getFusionEngine())["FastCall"].emplace_back(new FusionPoint_FastCall_StringPtr()); + (*getFusionEngine())["FastCallChecked"].emplace_back(new FusionPoint_FastCallChecked_StringPtr()); } } diff --git a/src/simulate/simulate_visit.cpp b/src/simulate/simulate_visit.cpp index cf3c5c69a9..f0950a9965 100644 --- a/src/simulate/simulate_visit.cpp +++ b/src/simulate/simulate_visit.cpp @@ -120,6 +120,13 @@ namespace das { V_END(); } + SimNode* SimNode_FastCallCheckedAny::visit(SimVisitor& vis) { + V_BEGIN(); + V_OP(FastCallChecked); + V_CALL(); + V_END(); + } + SimNode * SimNode_CallAny::visit(SimVisitor& vis) { V_BEGIN(); V_OP(Call); diff --git a/tests-cpp/small/test_aot_table_move_tombstones.cpp b/tests-cpp/small/test_aot_table_move_tombstones.cpp new file mode 100644 index 0000000000..8e9f543268 --- /dev/null +++ b/tests-cpp/small/test_aot_table_move_tombstones.cpp @@ -0,0 +1,20 @@ +#include +#include "daScript/daScript.h" +#include "daScript/simulate/aot.h" + +using namespace das; + +TEST_CASE("an AOT table move carries tombstones, so a moved table never rehashes on a count it never had") { + TTable a; das_zero(a); + a.tombstones = 7; + TTable b; das_zero(b); + b = a; + CHECK(b.tombstones == 7u); + CHECK(a.tombstones == 0u); + TTable sa; das_zero(sa); + sa.tombstones = 5; + TTable sb; das_zero(sb); + sb = sa; + CHECK(sb.tombstones == 5u); + CHECK(sa.tombstones == 0u); +} diff --git a/tests-cpp/small/test_fast_call_depth_context.cpp b/tests-cpp/small/test_fast_call_depth_context.cpp new file mode 100644 index 0000000000..cecb93c876 --- /dev/null +++ b/tests-cpp/small/test_fast_call_depth_context.cpp @@ -0,0 +1,19 @@ +#include +#include "daScript/daScript.h" + +using namespace das; + +TEST_CASE("the fastcall depth cap reaches a context through setup, a clone copies it, restart zeroes the counter") { + Context ctx(16 * 1024, false); + CodeOfPolicies policies; + policies.max_fast_call_depth = 7; + ctx.setup(0, 0, policies, AnnotationArgumentList()); + CHECK(ctx.maxFastCallDepth == 7u); + ctx.fastCallDepth = 5; + ctx.restart(); + CHECK(ctx.fastCallDepth == 0u); + ctx.fastCallDepth = 3; + Context clone(ctx, 0u); + CHECK(clone.maxFastCallDepth == 7u); + CHECK(clone.fastCallDepth == 0u); +} diff --git a/tests/README.md b/tests/README.md index 21a5c23ba7..7e2023fbde 100644 --- a/tests/README.md +++ b/tests/README.md @@ -693,6 +693,8 @@ JIT compilation and code-generation tests. None have `expect` directives. The sl | invalid_type_ref_in_table_value.das | Ref type as table value | **expect** `30106` | | invalid_types.das | Oversized types and arguments - declarations, `new`, ascend, `default` | **expect** `30500:3` `30508` `30510` `30512:3` `30513` | | failed_jit_abi.das | JIT ABI correctness - `test_abi_mad` for float2/3/4, function pointers | | +| _fast_call_depth_recursers.das | *(helper)* `no_aot` module of `[no_jit]` fastcall recursers (one, two, any-left, any-right, nine arguments) and their framed entries for `fast_call_depth.das` | | +| fast_call_depth.das | `options max_fast_call_depth` - the message names the option, the cap is exact per fused shape, a thousand recovered overflows leave no drift, a host policy caps a program with no options line and a host catch restores the counter | | | labels.das | Labels and goto - control flow, nested loops, labeled break | | | lambda_basic.das | Lambda capture, invoke, null check, addX returning lambda | | | lambda_capture.das | Lambda capturing const values, finalizer behavior | | diff --git a/tests/language/_fast_call_depth_recursers.das b/tests/language/_fast_call_depth_recursers.das new file mode 100644 index 0000000000..be6de0a0df --- /dev/null +++ b/tests/language/_fast_call_depth_recursers.das @@ -0,0 +1,73 @@ +options gen2 +options no_aot // the guard counts interpreted fastcall nodes - an AOT body recurses natively into the stack guard page +module _fast_call_depth_recursers shared + +//! Every recurser is [no_jit] for the same reason the module keeps out of AOT: a jitted body is native. + +var g_sink = 0 //! every probe result lands here, so no call is dropped as a pure call with an unused result +var g_no_fold = 0 //! added to every argument, so no probe folds to a constant at compile time + +[no_jit] +def rec_forever(n : int) : int { + return rec_forever(n + 1) +} + +[no_jit] +def rec_sum(n : int) : int { + return n == 0 ? 0 : n + rec_sum(n - 1) +} + +//! both arguments simple: the fused ArgConst shape +[no_jit] +def rec_sum2(n : int, acc : int) : int { + return n == 0 ? acc : rec_sum2(n - 1, acc + n) +} + +//! a computed left argument beside a plain one: the fused Any-left shape +[no_jit] +def rec_any_left(n : int, acc : int) : int { + return n == 0 ? acc : rec_any_left(n - 1, acc) +} + +//! a plain left argument beside a computed one: the fused Any-right shape +[no_jit] +def rec_any_right(acc : int, n : int) : int { + return n == 0 ? acc : rec_any_right(acc, n - 1) +} + +//! nine arguments: the unrolled node family stops at eight, past it the variadic specialization +[no_jit] +def rec_nine(a, b, c, d, e, f, g, h : int; n : int) : int { + return n == 0 ? a + b + c + d + e + f + g + h : rec_nine(a, b, c, d, e, f, g, h, n - 1) +} + +//! A framed, interpreted entry: the counted chain starts at the recursion itself, so the depth +//! is n + 1 whether the caller is interpreted, jitted or AOT. +[export, never_inline, no_jit] +def framed_rec_sum(n : int) : int { + return rec_sum(n) +} + +[export, never_inline, no_jit] +def framed_rec_sum2(n : int) : int { + return rec_sum2(n, 0) +} + +[export, never_inline, no_jit] +def framed_rec_any_left(n : int) : int { + return rec_any_left(n, 7) +} + +[export, never_inline, no_jit] +def framed_rec_any_right(n : int) : int { + return rec_any_right(7, n) +} + +[export, never_inline, no_jit] +def framed_rec_nine(n : int) : int { + return rec_nine(1, 2, 3, 4, 5, 6, 7, 8, n) +} + +def triangular(n : int) : int { + return n * (n + 1) / 2 +} diff --git a/tests/language/fast_call_depth.das b/tests/language/fast_call_depth.das new file mode 100644 index 0000000000..9465c01a13 --- /dev/null +++ b/tests/language/fast_call_depth.das @@ -0,0 +1,115 @@ +options gen2 +options max_fast_call_depth = 256 + +require dastest/testing_boost public +require daslib/ast +require daslib/rtti +require daslib/debugger +require daslib/fio +require strings +require _fast_call_depth_recursers + +//! The recursers live in a no_aot module, so this file runs under AOT and JIT as well: each +//! sweep exercises its own recover handler (the try node, jit_try_recover, das_try_recover). +//! An instrumented run (dastest --cov-path, the debugger) rewrites every body, so nothing is +//! fastcall and the guard counts nothing: the exactness arms skip there, by the probe below. + +let DEPTH_LIMIT = 256 + +let UNARMED = "fastcall is off in this run (instrumented bodies), the guard counts nothing" + +//! The guard's own message from an unbounded recursion; any other panic means fastcall is off. +def guard_armed : bool { + let why = panic_text() $ { + g_sink = rec_forever(g_no_fold) + } + return find(why, "max_fast_call_depth") >= 0 +} + +[test] +def test_fast_call_depth_panics(t : T?) { + g_no_fold = 0 + if (!guard_armed()) { + t |> skip(UNARMED) + return + } + t |> run("unbounded fastcall recursion panics naming the option") @(t : T?) { + let why = panic_text() $ { + g_sink = rec_forever(g_no_fold) + } + t |> success(find(why, "stack overflow, max_fast_call_depth 256 exceeded while calling") >= 0, why) + } + t |> run("one argument: at the cap passes, one call past it panics") @(t : T?) { + t |> equal(framed_rec_sum(DEPTH_LIMIT - 1 + g_no_fold), triangular(DEPTH_LIMIT - 1)) + t |> success(panic_text() $ { g_sink = framed_rec_sum(DEPTH_LIMIT + g_no_fold) } != "", "one call past the cap must panic") + } + t |> run("two plain arguments, the fused shape") @(t : T?) { + t |> equal(framed_rec_sum2(DEPTH_LIMIT - 1 + g_no_fold), triangular(DEPTH_LIMIT - 1)) + t |> success(panic_text() $ { g_sink = framed_rec_sum2(DEPTH_LIMIT + g_no_fold) } != "", "one call past the cap must panic") + } + t |> run("a computed left argument, the fused Any-left shape") @(t : T?) { + t |> equal(framed_rec_any_left(DEPTH_LIMIT - 1 + g_no_fold), 7) + t |> success(panic_text() $ { g_sink = framed_rec_any_left(DEPTH_LIMIT + g_no_fold) } != "", "one call past the cap must panic") + } + t |> run("a computed right argument, the fused Any-right shape") @(t : T?) { + t |> equal(framed_rec_any_right(DEPTH_LIMIT - 1 + g_no_fold), 7) + t |> success(panic_text() $ { g_sink = framed_rec_any_right(DEPTH_LIMIT + g_no_fold) } != "", "one call past the cap must panic") + } + t |> run("nine arguments, the variadic node") @(t : T?) { + t |> equal(framed_rec_nine(DEPTH_LIMIT - 1 + g_no_fold), 36) + t |> success(panic_text() $ { g_sink = framed_rec_nine(DEPTH_LIMIT + g_no_fold) } != "", "one call past the cap must panic") + } +} + +[test] +def test_recovered_overflow_leaves_no_drift(t : T?) { + g_no_fold = 0 + if (!guard_armed()) { + t |> skip(UNARMED) + return + } + t |> run("a thousand recovered overflows, then a recursion at the cap still passes") @(t : T?) { + for (_i in range(1000)) { + try { + g_sink = rec_forever(g_no_fold) + } recover { + } + } + t |> equal(framed_rec_sum(DEPTH_LIMIT - 1 + g_no_fold), triangular(DEPTH_LIMIT - 1)) + } +} + +//! A program with no options line, capped at 16 by the host's CodeOfPolicies: at_cap nests 16 +//! calls, past_cap 17, and overflow never returns. +let HOST_CAPPED_SOURCE = "options gen2\nvar g_no_fold = 0\nvar g_sink = 0\ndef rec_sum(n : int) : int \{\n return n == 0 ? 0 : n + rec_sum(n - 1)\n\}\ndef rec_forever(n : int) : int \{\n return rec_forever(n + 1)\n\}\n[export]\ndef at_cap \{\n g_sink = rec_sum(15 + g_no_fold)\n\}\n[export]\ndef past_cap \{\n g_sink = rec_sum(16 + g_no_fold)\n\}\n[export]\ndef overflow \{\n g_sink = rec_forever(g_no_fold)\n\}\n" + +[test] +def test_host_policy_and_host_catch(t : T?) { + t |> run("the policy caps a program with no options line, and a host catch restores the counter") @(t : T?) { + let path = path_join(test_temp_dir(), "host_capped.das") + fwrite(path, HOST_CAPPED_SOURCE) + var inscope access <- make_file_access("") + using() $(var mg : ModuleGroup) { + using() $(var cop : CodeOfPolicies) { + cop.max_fast_call_depth = 16u + cop.threadlock_context = true + compile_file(path, access, unsafe(addr(mg)), cop) $(ok; program; issues) { + t |> success(ok, "the capped program compiles: {issues}") + return if (!ok) + simulate(program) $(sok; ctx; serrors) { + t |> success(sok, "the capped program simulates: {serrors}") + return if (!sok) + unsafe { + t |> equal(panic_text() $ { invoke_in_context(ctx, "at_cap") }, "") + let past = panic_text() $ { invoke_in_context(ctx, "past_cap") } + t |> success(find(past, "max_fast_call_depth 16 exceeded") >= 0, "the host policy is the cap: {past}") + let escaped = panic_text() $ { invoke_in_context(ctx, "overflow") } + t |> success(find(escaped, "max_fast_call_depth 16 exceeded") >= 0, "the overflow escapes to the caller: {escaped}") + t |> equal(panic_text() $ { invoke_in_context(ctx, "at_cap") }, "") + } + } + } + } + } + } +} diff --git a/tests/module_cache/_fixtures/mc_cf_drv.das b/tests/module_cache/_fixtures/mc_cf_drv.das index 257d5fff9e..f3e333f53a 100644 --- a/tests/module_cache/_fixtures/mc_cf_drv.das +++ b/tests/module_cache/_fixtures/mc_cf_drv.das @@ -5,28 +5,32 @@ require daslib/rtti require daslib/ast //! compiles the file named after `--` through compile_file, under the module cache when an -//! argument is `cache` and unoptimized when one is `noopt`; the per-module compile-time log (a -//! served module's line reads `cache read took`) is printed for the test to read +//! argument is `cache`, unoptimized when one is `noopt` and fastcall-capped when one is `depth`; +//! the per-module compile-time log (a served module's line reads `cache read took`) is printed +//! for the test to read [export] def main { let args <- get_command_line_arguments() let sep = find_index(args, "--") if (sep < 0 || sep + 1 >= length(args)) { - print("USAGE: mc_cf_drv.das -- [cache] [noopt]\n") + print("USAGE: mc_cf_drv.das -- [cache] [noopt] [depth]\n") return } let target = args[sep + 1] var cached = false var noopt = false + var depth = false for (i in range(sep + 2, length(args))) { cached ||= args[i] == "cache" noopt ||= args[i] == "noopt" + depth ||= args[i] == "depth" } var inscope access <- make_file_access("") using() $(var mg : ModuleGroup) { using() $(var cop : CodeOfPolicies) { cop.module_cache = cached cop.no_optimizations = noopt + cop.max_fast_call_depth = depth ? 64u : 0u cop.log_module_compile_time = true compile_file(target, access, unsafe(addr(mg)), cop) $(ok; program; issues) { print("{issues}\n") diff --git a/tests/module_cache/test_compile_file_cache.das b/tests/module_cache/test_compile_file_cache.das index 6cd21de3c5..f578e7d2ed 100644 --- a/tests/module_cache/test_compile_file_cache.das +++ b/tests/module_cache/test_compile_file_cache.das @@ -101,6 +101,11 @@ def test_compile_file_module_cache(t : T?) { targetNooptOk = targetNooptOk && !served(targetNoopt, leafFile) && !served(targetNoopt, rootFile) t |> success(targetNooptOk, "other policies beside a --jit-target still compile cold:\n{targetNoopt}") t |> equal(cache_files(cacheDir, "mc_cf_root"), 5, "the policies hash survives the jit target in the key") + var depth : string + var depthOk = run_child_reported(t, "depth", "{drv} cache depth", depth, "MC_CF_COMPILED") + depthOk = depthOk && !served(depth, leafFile) && !served(depth, rootFile) + t |> success(depthOk, "a fastcall depth cap is a policy the key folds, so it compiles cold:\n{depth}") + t |> equal(cache_files(cacheDir, "mc_cf_root"), 6, "max_fast_call_depth keys a sixth record") var err : string rmdir_rec(tmp, err) }