Internal working roadmap. Public-facing docs: guide.md (user guide), design.md (rationale). Deep dives: task-internals.md, deferred-versioned-state.md, command-buffer-design.md, task-systems-comparison.md.
Legend: [ ] open · [~] partial/WIP · [S] shelved (designed, not scheduled).
Priority: P0 = pre-public · P1 = soon · P2 = later · P3 = someday.
Fuller notes for the larger items are in § Detail; exhaustive research
survives in git history (this file was condensed 2026-07).
All layers built, tested (501 checks as of 2026-08, TSan/ASan clean, subprocess death tests), documented, and CI-gated.
- Task core (coroutine-first, 2026-08) —
Task<R>is the one completion primitive: awaitable from any task,sync()/take()legal ONLY outside tasks (in-task blocking is fatal underTS_SAFETY_CHECKS). Composition isco_await;ts::launch(free-running, inherits NO grant — detached, so a touch of the launcher's guarded data faults deterministically) is the one bare-task verb; grant-inheriting fan-out ists::parallel_for(helpers snapshot the grant, join synchronously). Thets::nested/Task_scope/join_nestedscoped-launch verbs were REMOVED (2026-08, an unsafe concurrent grant-inheriting child — coroutine-first.md §4.3);detail::add_nestedsurvives as graph-internal completion-gating. Inheritance is gated-only (docs/internals/coroutine-first.md §2). Cooperative cancellation (+Cancel_callback+ trailing-Cancellation_tokenbody early-out),Signal(+reset()) as the awaitable event. Deleted with the transformation:then,when_all, thets::taskbuilder +after, retraction/deep-retraction, executable task reuse, the inline-dispatch trampoline (set_inline/run_inline) — see coroutine-first.md §3/§8. - Guarded / access — awaitable
co_await obj.access(fn)(inline when the pipe is free or the caller already owns the write grant, else suspends) + eagerasync(returnsTask<R>, callable from any color), const-ness deduction, multi-objectts::access/ts::async(canonical-order cascade, deadlock-free), per-object reader/writer pipe. - Pipe (evolved, 2026-08) — one mutex per pipe with rebuilt internals: intrusive FIFO of
embedded
Pipe_links (noJob/std::deque/closures), pipe turns asnum_locksprerequisites, one unified multi-object cascade serving dynamic multi-objectasyncAND graph nodes (deletingMulti_async_state,acquire_next,preheld, the explicit graph handoff), always-onwriter_ownergrant ownership. Design of record: pipe-rebase.md §0. - Static graph — access-conflict-derived edges + explicit
after/before, per-node mode-aware acquire (objects free in the gaps), coroutine node bodies, per-node implicit scope, allocation-free re-runs. - Scheduler (M2 stages 1–4) — per-priority lock-free MPMC queues, Vyukov eventcount,
per-worker Chase-Lev deques + stealing, low-starvation valve + spinner cap; idle policies
(
spin/spin_then_blockdefault /handoff). - Coroutines — mandatory, not optional: awaitable
Task,co_await ts::read_only/ read_writeguards, promise-carried access context reinstalled at every resumption, coroutine graph nodes, frame/block fusion (one allocation — the promise embeds the block), the bounded resume trampoline, the circular-wait detector for suspended-ABBA. - Deferred / Versioned — staged-write layer (
Journal/Recorder/Parallel_recorder), ONE auto-dispatchingcommit()(inline under a held write grant, else an enqueued write —commit_asyncremoved), three-phase publish, replay/copy/overwrite resync, single-publisher enforcement. - Allocation —
launch1 ·async1 · coroutine task 1 (frame/block fusion) allocs/op (intrusive refcount, per-dispatchsubmit_closurekill). Thethen/when_allmulti-alloc offenders are gone with the verbs themselves; re-baselinemem_profileon the coroutine shapes (10.14 territory). - Build / public-prep — MIT license, whole lib in
ts::, CMake + presets, CI (MSVC / clang-cl / Linux-TSan), Shipping config (TS_SAFETY_CHECKS=0), v0.1.0 (+ CHANGELOG.md; the library goes public as 0.1.0),ts.humbrella,.clang-format,CONTRIBUTING+ issue/PR templates.
Reference items as area.item (e.g. 3.1 = ambient scheduler). Numbers are stable
IDs — when an item is done, mark it, don't renumber.
-
Guarded / access
Standing priority (author, 2026-07): anything that strengthens the harness is P1 by default — the runtime oracle is the library's signature safety property and its coverage gaps (§13.6 of research-deepdive.md) are where every comparable system's real-world failures concentrated.
-
[~](P1) Zero-alloc inlineaccess— void→done-sentinel, small-R→SBO handle, large/deferred→heap. [§D1] -
[x](P1) Multi-objectts::accessinline — DONE (2026-08, multi-access-op-design.md).ts::access(fn, objs...)returns one caller-ownedAccess_op<Objects..., Body>at every arity (the type became variadic, objects first and body last, so the single-object spellingAccess_op<T, Body>is unchanged — asserted, not just claimed). Dispatch matches the single-object verb: objects the calling task already holds are lent (no turn taken, the graph'sbind_links_for_runprotocol — this subsumes the reentrant arm, and a read holder where the body writes is fatal), and the remaining pipes are probed all or nothing under their mutexes at once, admitting every one or none.ts::asyncis unchanged. -
[x](P1) Generic-lambda /auto&deduction — DONE (A3+A4, 2026-07). Introspecting a generic lambda's parameter const-ness is impossible (templatedoperator()), but classification is not: the rvalue-bindability probe classifies each bare-arg position at declaration level —const auto&/auto&&binds an rvalue → read,auto&cannot → write — so generic lambdas deduce under the SAME "const = read" spelling rule as non-generic ones, no tags required. A3: every read position (deduced, probed, or tagged) invokes the body withconst T&(mode_ref), so mutating under a read classification is a compile error — and the constraint/return-type machinery evaluates the mode gate FIRST (accessor_mode,Accessor_result) because probing a generic lambda's invocability againstconst T&deduces its return type, which instantiates the BODY — a hard error for a mutating body, not SFINAE. Q1: by-value / rvalue-ref resource params arestatic_assert-rejected wherever introspectable (a by-value copy silently discards writes); the genericautoby-value spelling is undetectable at declaration level — documented residual. Tagsts::as_read_only(g)/as_read_write(g)(names ratified; match theAccessenum and coroutine guards) remain as the explicit-declaration style, the escape for anauto&&that must write, and permit conservative write-over-declaration (write tag over aconst T¶m serializes, legally). One unified builder per surface (async_build_modes/fill_node_modes) serves the deduced/probed/tagged tiers. Tests: probed single-object read-overlap/write-serialization + classification static_asserts, probed multi-object (incl. mixed spellings) + over-declared write, probed graph nodes;#if 0blocks document the compile-time rejections. 356→369 checks; clean MSVC + clang-cl + MSBuild, TSan clean. -
[ ](P1, raised 2026-07 — harness item) Access-check completeness — clang-tidy "every public method callsTS_CHECK_ACCESS()" + aGuarded_value<T>for POD/no-method types. [§D2] -
[ ](P2) AdoptedGuarded— wrap an existing instance (borrow) via anadoptctor tag; access key = borrowed address. Ownership-transfer note (2026-07, T22): an adopted object handed between owners needs an explicit re-bind ritual (Chromium'sDETACH_FROM_SEQUENCE— the first access after detach re-binds the checker), NOT implicit adoption. Frame it as an escape hatch, discouraged for general use; the owned-Guardedpath (in-place construction, harness-instrumented methods) stays the recommended default. -
[ ](P2) Sub-object / range harness — grant onScovers[&S, &S+sizeof), or a declarableRegion<T>. -
[ ](P3) Standalone publicPipe— expose the reader/writer pipe (UEFPipe); defer until a non-single-object use case appears. -
[x](P1)Access_optionsvsDispatch_options- DONE (2026-08-23, API-stability M2(b)). Split, not shared:Launch_optionswas renamedDispatch_options{token, priority, name}and taken by every verb that always schedules (ts::launch,Guarded::async, the freets::async,Deferred::commit,Versioned::publish), whileAccess_options= the same three plusqueuedand is taken only by the opportunistic verbs (Guarded::access, the freets::access,Versioned::read, theAccess_opctors), which are the only ones with an inline arm to skip. No inheritance: a base subobject cannot be named by a designator, soAccess_options{.priority = p}would stop compiling (verified against MSVC 19.51:error C7559).Versioned::publishalso stopped droppingopts.name. Was: — revisit whether the two option aggregates share a base or stay split (flagged intask.h); resolve in the code-review API-stability pass. -
[S]Checked accessor wrapper (Granted<T>) — designed, declined for now (author, 2026-07: "maybe never"). Functors could receive a pointer-likeGranted<T>/Granted<const T>whoseoperator->/*runsaccess_checkthen hands outT&— zero-intrusion diagnostics for types withoutTS_CHECK_ACCESS()instrumentation (incl. PODs/third-party types; would subsume 1.4'sGuarded_value). Semantics: checks are tautological inside the sanctioned body; the signal is stashed/escaped handles (post-release, cross-thread, across-suspension use faults at deref); raw-ref extraction (T& r = *g) still launders the check away, so method instrumentation stays strictly stronger for owned types. Trust tiers: 0 = none (scheduling stays correct — the harness is diagnostics, not correctness), 1 = wrapper, 2 = instrumented methods (+ clang-tidy). Implementation is cheap (access_check(T*)/(const T*)overloads already resolve the mode;TS_SAFETY_CHECKS=0→ raw passthrough); the work is the third signature tier in the deduction machinery (by-valueGrantedexempt from the Q1 static_assert). Revisit only if instrumentation burden proves a real adoption blocker. -
[ ](P2) Whole-object granularity serialization — think through the mitigation menu beyond current guidance (sharding/Versionedplacement, sub-object grants via 1.6, split-along-the-seams doc patterns); research-deepdive.md §13/§19 maps how Bevy (per-component + filters) and Unity (per-container, escape attributes) evolved under the same pressure. Flagged by the 2026-07 research pass. -
[x](P1) Grant-generation validity check — DONE (2026-07). The stale inherited-grant hole (research-deepdive.md §13.6.5) is closed via a per-pipe write-epoch with seqlock parity (Pipe::write_epoch): bumped at write-grant acquire + release (under the pipe mutex, relaxed) and by +2 on a graph write handoff (which elides both pipe ops); reader traffic never bumps. EveryAccess_contextentry declared under a pipe grant captures the epoch (all declaration sites threaded: single- object access/async, multi-object, graph nodes, coroutine pipe guards,Versionedfront scopes);Access_context::checkreturns granted/stale/none andaccess_checkfatals on stale with a dedicated diagnostic naming the fix (acquire fresh viaobj.async/co_await obj.access, or fan out withts::parallel_for). One rule serves both modes: a write entry is valid while its window is open, a read entry until a writer acquires. Compiled out underTS_SAFETY_CHECKS=0(bumps, capture, compare all gated; the 8-byte pipe field stays for layout stability). Chosen over per-owner generations (refcount traffic, per-owner plumbing) and inherit-into-nested-only (arming-after-launch races the child's first check). Tests: epoch unit verdicts, read-era semantics,stale_inherited_grantdeath scenario; suite 470 checks green, Shipping compiles, Release+Shipping stress clean (no false positives across handoff /parallel_for/Versionedpaths). -
[x](P1) Blocking-sync()-under-grant check — DONE (2026-07). Two pieces, fullyTS_SAFETY_CHECKS-gated: (a)TS_ENSURE(expr, message)(fatal.{h,cpp}) — the UE-ensureshape per author: evaluatesexpronce in both configs, yields its bool (if (!TS_ENSURE(...))recovers); on failure bumpsts::ensure_failure_count()on EVERY occurrence but reports ONCE PER CALL SITE (captureless-lambda function-local static claim — per-frame recurrences stay one stack trace, exact counts). Presentation via a swappable handler (ts::set_ensure_handler,std::set_terminateshape; counting stays outside it, so a host dialog can't hide failures); default handler =ENSURE FAILED:+ stacktrace + debugger-break when attached (detail::is_debugger_present/debug_break, the C++26 P2546 pair polyfilled —IsDebuggerPresent/__debugbreak, LinuxTracerPid+int3/brk, macOS sysctlP_TRACED; feeds the future platform layer 3.6).ts::fatalalso breaks pre-abortat the failure site. Cleanliness is enforced: the harness fails on failures not consumed viats::test::consume_ensure_failures(n),--bench/--stressfail their exit on any (full no-arg run covered bysummary()). (b) the check atretract_or_wait(the chokepoint every blocking task wait passes): fires only when genuinely about to park (post-retraction, post-serial-drain, target unsettled) under an access scope on non-retractable work; sharp same-object message when the target is a single-object pipe job on a pipe the context holds (certain deadlock —Flags::pipe_job+dispatch_argstamped with thePipe*at creation, matched viaAccess_context::holds_epoch;blocking_sync_diagnoseseam task.h→guarded.cpp), general never-block warning otherwise (multi-object targets land on the general message — theirdispatch_argcarries the reuse generation, not a pipe). Structurally exempt:parallel_for(joins via its own counter, neverretract_or_wait), retractable targets (the waiter runs them), grant-free threads. Tests: ensure facility (once-per-site: same site twice → +2 count, one report), handler hook (install/restore, custom handler hit), deterministic warns-but-completes (blocker spins until the failure lands),parallel_for-in-node zero failures,sync_own_object_deadlockdeath scenario (child aborts on observing the sharp report; bounded poll so a miss fails instead of hanging). 503 checks green; full run (samples + benchmarks) zero failures; Shipping compiles. -
[S]Escaped-reference hardening (T21) — analyzed, SHELVED (author, 2026-07). Decision document: escaped-refs-hardening.md. Findings: (a)Granted<T>confirmed in the noise quadrant — five-system precedent survey (Chromium raw_ptr conceding.get()and winning via quarantine not deref checks; folly stating the launder hole verbatim; MSVC's checked-in-release default reversed) shows checked handles succeed only as the sole access path, tool-enforced, or with guarantees that survive extraction — 1.9 stays shelved with receipts. (b) Option A (interval-containment grants = 1.6's implementation; unlocks instrumented member types under a parent grant) and Option B (grant-scoped ASan poisoning off the pipe's acquire/release edges — catches stashed rawT&on Windows where TSan doesn't exist) both viable; author verdict: B "interesting but too limited, maybe in the future"; A shelved with it. Page protection rejected on numbers. Escaped-ref coverage remains TSan's job per limits.md; revisit on a real adoption-blocking incident. -
[x]DONE, but NOT as designed (2026-08) — the lock-free chain was implemented, stress-tested, and RETIRED; what shipped is the evolved mutex pipe. The R10 gate came back negative (22 producers on one pipe: 798 ns/op vs ~1050 ns/op uncontended — the mutex was never the bottleneck), and every reader-group piece of the chain burned (walk double-claim, three custody/lifetime UAFs, claim-without-fire, the tenure/era ABA that exists only because the chain is lock-free), while every serial UE-verbatim piece worked immediately. What shipped instead: one mutex per pipe with rebuilt internals — intrusive FIFO of embeddedPipe_links, pipe turns asnum_locksprerequisites, the unified multi-object cascade (one path for dynamic multi-objectasyncand graph nodes;Multi_async_state/acquire_next/preheld/explicit handoff deleted), always-onwriter_owner, and thewait_until_idledrain kept as a CV notified under the mutex (verified immune to the UEFPipeteardown UAF, which is a lock-free-notify artifact). Design of record + the retirement evidence: pipe-rebase.md §0; tests pipe-rebase-tests.md. The addenda below are kept as the engineering record — note that theDeferred::last_commit_race they describe IS dissolved (the enqueue-and-record seam under the pipe mutex;commit_mutex_deleted), and the retraction addendum is moot (retraction is gone with the coroutine-first transformation, 6.4). Original design text follows. UE's pipe is lock-free not by clever atomics but by not being a scheduler structure: one atomicLastTask; push = exchange +AddSubsequenton the previous task — serialization compiled into dependency edges, with the existing task machinery doing all dispatch (verified in Pipe.cpp). Our generalization: writers chain FPipe-style (atomic tail, prerequisite edge on the previous holder); a reader batch becomes a reader-group sentinel — a bare block whose nested-task counter counts active readers (existingnum_locksmachinery verbatim); a subsequent writer takes one edge from the sentinel; last reader out completes it. Reservations (graph acquire) become bodyless blocks whose dispatch fireson_acquiredand whose completion IS the release.wait_until_idle= task-count + eventcount (copy UE's pattern incl. its last-decrement use-after-free caution). Gains: structurally inline-safe (nothing dispatches under a lock), lock-free producers, reader join/leave =fetch_addnot mutex, and unification — subsumes the pending graph/when_allrebase thread; 1.7's standalonePipefalls out. Costs: the tail-push protocol (writer arrival closes the group to joiners; a reader's failed join-CAS re-reads the tail — a known queued-RW-lock construction, ~100 careful lines), one pooled bare block per reader batch (rides 4.1's free-list), write-epoch bump migration, a serious TSan campaign. GATE: measure pipe-mutex contention on a reader-heavy fixture first — no rewrite without evidence. (Ashared_mutexis NOT the interim answer: every pipe op mutates admission state, so all would take it exclusive.) Addendum (2026-07, from theDeferred::last_commit_race): the pipe does not expose queue facts (last-enqueued write, subset completion), so a wrapper that fire-and-forgets jobs and later needs "are my jobs done" must reconstruct pipe ordering externally — and the reconstruction is subtle: handle-store order need not match enqueue order, so recording "my last job" requires a lock spanning enqueue + store (a plain or even atomic store of the newest handle can leave an earlier job's handle recorded, and a settled-check on it then misses a pending job). Incident:Deferred::commit_asyncraced exactly this way (caught by the pre-push TSan hook; fixed withcommit_mutex_spanning enqueue + store). Inventory:Versioned::chain_is sound by construction (theseq_mutex_'d chain, not the pipe, is its ordering source). This rebase dissolves the class: the atomic tail IS the last-job handle — expose it (e.g.Pipe::last_write()completion, or a settled-check) and deleteDeferred's external mutex; any future fire-and-forget wrapper gets ordering for free. Until then the rule stands: never record pipe ordering without a lock spanning enqueue + record. Addendum (2026-07, retraction): the split-structure pipe is why pipe/asyncjobs are non-retractable. Pipe ordering lives in a separate structure (pipe.jobsdeque- reader/writer admission in
dispatch), NOT in the block'sprerequisites.retract(task.h) walks onlyprerequisites, so it is blind to pipe admission order — a pipe job flaggedretractablecould be run out of turn while still queued behind a conflicting writer (the exact race task-internals.md §6 cites). So pipe blocks are marked non-retractable (run_pipe_job,guarded.cpp: "pipe blocks are not retractable and never inline-dispatched"), and a blockingsync()on one parks a worker — caught only by the blocking-sync diagnostic, never made safe. UE gets pipe retraction for free precisely because it does NOT keep a separate structure: the previous piped task IS registered as a prerequisite (FTaskBase::Prerequisitesis populated "by piping, when the previous piped task is added as a prerequisite"), soTryRetractAndExecutecovers pipe chains through the same recursive prerequisite walk and respects order structurally (predecessor retracted first — never out of turn). The catch: UE'sFPipeis strictly serial, so each piped task has exactly ONE pipe-predecessor, which maps to a single prerequisite edge. This rebase restores retraction the same way — but only cleanly for writers: a writer chains via one prerequisite edge on the previous holder, so it becomes retractable like any prerequisite (walk the edge, run inline once it clears). Readers do not map: the reader-group sentinel has N predecessors and a reader block carries no single ordering edge to walk, so reader retraction stays a separate design question even post-rebase. Net: the rebase reframes §6's parked "Retraction of pipe/async tasks" blocker from "admission-ordering plumbing" to "pipe ordering is not a prerequisite chain," and delivers writer retraction as a side effect — reader retraction not.
- reader/writer admission in
-
[ ](P1, author 2026-07) Audit the two access routes (sync/async): defaults, low-level launches, zero-cost. A systematic review of how a two-way data access is performed across the whole surface:access(opportunistic-inline) vsasync(always-scheduled), and the same choice for coroutineco_awaitaccesses, with three goals. (a) Review the DEFAULTS, especially for coroutines: is the default route (inline-when-free vs schedule) the right one at each entry, and consistent across single-object / multi-object / graph / coroutine access? Coroutine defaults are the least-examined and most likely wrong. (b) Minimise task launches at LOWER levels of abstraction: the design stance is that parallelism concentrates at the HIGHER levels (the frame graph,parallel_forover real work), so a low-level access that silently spawns a scheduled task where an inline body would serve is a cost with no parallelism win. Find and cut those, and consider stating this as a design principle (design.md / the CLAUDE.md principles list). (c) Driveaccessand inline dispatch toward a zero-cost abstraction: where the object is free, anaccess/ awaited access should approach the cost of a direct call. Subsumes and should be done WITH 1.1 (zero-alloc inlineaccess) and 6.1 (inline-when-free for awaited accesses); pairs with 10.11. Deliverable first: a route-by-route table (entry x default x inline-eligibility x alloc/dispatch cost) before changing anything. Partly answered (2026-08, 6.4). (a) is settled for the dynamic surface: there is one awaitable access verb whose default IS inline-when-free (plus reentrant-owner inline), and one eagerasync— the split is now by color, not by policy, so there is no inconsistent default left to pick between. What survives of this item: multi-objectaccessstill has no inline arm (1.2), (b) the low-level-launch minimisation pass has not been done (and should now count coroutine frames, not just task blocks — 6.2), and (c) the zero-cost target is unmeasured on the new verbs. Redo the route table against the coroutine surface before acting. -
[ ](P1, author 2026-07) NestedGuarded: investigate, then sample or hazard doc. The case: a guarded resource whose member is itself guarded (aGuarded<T>whereTholds aGuarded<U>), and accessing the innerGuardedfrom inside the outer access scope. Work through whether it composes cleanly: does the per-objectAccess_contextnest correctly (an inneraccess/asyncunder an outer grant), do the pipes compose without deadlock (inner acquire while holding the outer), and do mode deduction + the write-epoch stale-grant machinery behave? If it works smoothly, add a sample or test demonstrating the pattern; if it does not, document the hazards (which nestings deadlock, defeat the harness, or serialize unexpectedly) AND add tests pinning the failure modes so they cannot silently regress. Either outcome ships coverage. Relates to 1.6 (sub-object grants) and the completeness-hazard story. -
[ ](P2, author 2026-08-18 - harness strengthening) Detect awaiting a detached graph run from a caller holding one of its objects. The deadlock shape: a detached run (execute({.detach = true})) receives no lend, so its nodes queue behind the caller's holds; the caller then awaiting the run waits on work that waits for the caller's own release. Design (rides existing detectors, no new machinery): give the detached run'sdoneblock pipe METADATA - a per-graphPipe_linkarray carryingdistinct_pipes_(never entered;num_locksuntouched) - so (a)co_awaitwhile holding a declared object hitscircular_wait_record's self-cycle branch ("holding X awaits X", immediate- named), (b) an in-task
sync()getsblocking_sync_diagnose's sharp same-object message, (c) an awaiter holding none of the objects records edges and proceeds - which is the correct, legal case. Cost: one lazy allocation per graph + two stores per detached execute; zero on attached runs and ordinary tasks. Caveat: the awaiter's collection array caps atAccess_context::max_entries(8), so graphs touching more objects get best-effort coverage of the first 8 in canonical order. Related: 2.19 (the lend-await hole on ATTACHED runs - same "containment assumes the await" family).
- named), (b) an in-task
-
[ ](P1, evidence 2026-08-18) Pipe turn tail-chaining on the releasing worker. Exposed by an exploratory mutex-vs-Guardedbench (2026-08-18, since removed; numbers preserved here): at a 1 us critical section on one hot store,Guardeddead-heats the mutex (0.43 vs 0.42 M/s, both at ~42% of the serial bound) because once the pool has drained everything except the hot pipe's chain, every admitted turn is dispatched viasubmitand wakes a parked worker - the pipe pays the same ~1.4 us wake-per-element a mutex convoy pays. A busy-pool variant of the same experiment showed the opposite (pipe beats the mutex hot chain) - the "completing worker dispatches the next turn directly" advantage exists only while the pool stays busy. Fix shape: whenreleaseadmits the next turn and the releasing thread is a worker, run the admitted command inline on it (tail-chain) instead of submitting - bounded by the usual trampoline discipline; interacts with fairness (a long hot chain would monopolize the worker - cap chained turns, then submit) and with priorities. Expected effect: the 1 us column roughly doubles toward the serial bound; the 200 ns column's remaining gap is per-touch block allocation, i.e. theAccess_opwork. -
[ ](P1, planned 2026-08-18)Access_op- caller-owned operation state foraccess. Design of record: access-op-design.md. The one-sentence model: the block allocation is the price of detachment, not ofR-accessreturns the operation state itself (eager, pinned once queued, dtor-checked),asynckeeps the heap handle for the leavers; zero-alloc attended access for anyR, no SBO tiers. Pipe layer unchanged (verified: entries already refcount the owner block andfire_grantedis pipe-free). Five phases in the doc (floor metric, the type + return-type change, guard awaiters embedding their links - killspipe_acquire's hold-node alloc, multi-object, rearm + async rebase). Supersedes 1.1's tiering, delivers 1.15(c), closes 1.2 at phase 3. Progress (2026-08-18): phases 0-1 + the §10 lifecycle landed (37070f9/707b7b0/26afa54); §9 questions all decided; custody as landed deviates from the doc's §4 claim -Flags::caller_ownedborrowed-pointer path, see the doc's §4 correction. REVIEW - perf still short of target (author flag): floor moved 125.5 -> 54.9 ns/op but that is ~4.4x a bare mutex vs the §7 target of ~2-3x; the malloc was only ~13 ns of the original gap. Residual ~40 ns = two pipe-mutex passes (admit + release) + claim CAS + context install - revisit after phases 2-3 and 1.18 (tail-chaining is the mechanism aimed at exactly the mutex-pass cost); if still short, profile the admission path itself.
-
-
Static task graph
-
[ ](P1) Typed graph chaining — a node consumes prerequisite-node results (nodes are void-only now); aGraph_nodemay then mint a per-runTask<R>. -
[ ](P2, raised within-band) Ambiguity detection —compile({.ambiguity = Warn|Error|Ignore})determinism diagnostic; needs edge provenance; feeds profiler-guided reorder. Research validation (2026-07, research-static-vs-dynamic.md): ordering ambiguity is the top user-facing failure of access-derived schedules — Bevy shipped exactly this diagnostic (ambiguity_detection) after its stageless rework because users hit nondeterministic system order in practice. PARKED (author, 2026-07). Full analysis in ordering-ambiguity.md: our declaration-index orientation is deterministic, so we lack Bevy's per-frame-nondeterminism bug class — the residual is hidden, unratified orientation (a refactor that swaps twoadd_nodelines silently flips gameplay). The proposed feature (conflict provenance + a fragile-orientation lint + a commutativity annotation feeding the optimizer) is rescoped as optimizer infrastructure, not a safety feature — but the annotation-cost question (pairwise = combinatorial; object-level = the mitigation, unproven) is unresolved. Do nothing until real usage data (start with the tiebreak-only pair count ongame_frame). Provenance itself is still needed by 2.4/2.5 and the DOT dump regardless. -
[ ](P3, pulled up 2026-08) Pipelined execution — more than oneexecute()in flight (frame overlap). Now also the relaxation path for nested graph runs (6.9a): the v1 rule there is one instance per concurrent user, because a pre-compiled inner graph invoked from two concurrently-running parents collides with one-run-at-a-time. Run-queueing / pipelined runs is the general fix, so a demonstrated shared-inner-graph case promotes this item rather than adding new machinery. -
[ ](P2) Profiler-guided optimization — reorder/rebucket from measured durations. Manual dry-run done (2026-07, game_frame); the headline finding is about measurement, not levers. Five configurations were tried (placement of staleness-tolerantVersionedreaders early-vs-tail × priority ranking none/high-low/spine-high): all cells measured 4.9–5.6 ms within one session window, while the SAME baseline configuration measured 4.15 ms in an earlier window and 5.3 ms later — cross-window ambient drift (~25%) dwarfed every within-window delta (~5–10%), so no lever's sign is established at single-run resolution. Everything below is therefore prerequisites + hypotheses to re-test under proper methodology:- Measurement methodology is prerequisite #1: interleaved runs (A/B/A/B to cancel drift), medians over N, controlled load, and the per-lane timeline capture from the graph-viz session (docs/graph-viz-handoff.md) — without this, optimization conclusions are noise. Ties to 10.1 (benchmark regression infra). Capture landed (2026-07):
Graph_tracestreams per-node medians/variance, per-edge binding gaps, measured critical-path frequency, dispatch waits, and the critical-dead-time headline (makespan − critical work). First validated findings, from the permanent 4-worker starved run: dead time 22.5% vs 2.4% on 12 workers (scheduling- vs dependency-bound, now a one-line diagnosis); the priority lever is REFUTED for the occupancy case, and the ordering lever half-confirmed (gap closed, makespan −10% regression —afteris completion-to-start). Full experiment record, lessons (makespan is the objective, dead time the diagnostic; the chain-extension guard; the missing yield/start-ordering primitive), and the three-tier tuner design in profiler-guided-optimization.md. Remaining prerequisite: interleaved A/B methodology. - Mechanistic hypotheses recorded (plausible, magnitudes unproven): (1) naive early gap-filling delays the serial spine — a ready critical-path node waits for a worker to finish its current slice, nothing evicts a runner → keep-out zones / finer filler granularity / reservation may be needed; (2) the current
Priorityenum cannot serve as rank:lowis valve-gated background (a mislabeled dependency — hand-labeling got nav→ai and ui→submit wrong, which automated upward-rank would not — stalls its dependents), andhighdispatches via the global MPMC queues instead of the per-worker deques (M2: onlynormalgoes local) — a structural cost. Prerequisite either way: a rank mechanism native to the local-deque path (rank-ordered pop withinnormal, or M2 stage 5), plus slice priority inheritance — done (2026-07):parallel_for/parallel_for_asynchelpers now inherit the calling task's priority by default (Parallel_options::priorityoverrides), so a node's slices dispatch at the node's class rather than alwaysnormal. - Info the optimizer needs: per-node wall duration as (total_work, parallel_width) — a sliced node's wall depends on free workers, a scalar misleads; medians + variance; the edge list WITH provenance (conflict-derived = semantic, immovable; explicit = intent; version-choice reader→flip / flip→reader = movable IFF the user annotates the read staleness-OK); per-lane timelines; worker count.
- Sanctioned lever taxonomy: (a) dispatch order among ready nodes — HEFT-style list scheduling by upward rank (blocked on the rank mechanism); (b) DAG shape via user-annotated semantic relaxations only — staleness-OK
Versionedreaders become a binary placement choice per reader, a small discrete search evaluable OFFLINE by simulating the captured cost model; (c) commutative-pair reordering (needs 2.2's provenance); (d) node fusion — merge nodes below the dispatch-cost floor that share an edge without losing parallelism (a PGO-time lever; see also 2.10's per-node cost numbers). The optimizer proposes; the user ratifies semantic moves. - Conflict-edge orientation as the core static-graph lever (author 2026-07, T18). The framing that makes this a headline selling point: a W-W or R-W conflict edge is a disjunctive arc (job-shop scheduling) — the pair must be ordered, either direction is race-free, and the direction is a scheduling choice. A pipes-only / dynamic system decides that direction online by arrival (no lookahead, nondeterministic) and can never plan around it because it never sees the arc before contention. A materialized graph sees every arc up front, so it can (1) compute critical-path / upward ranks and orient/dispatch to shorten the makespan, and (2) even for edges it must not flip, report "edge X (R->W on
physics) adds Y µs to the critical path — considerVersionedor deferral" (the restructuring-suggestion mode). This is the strongest concrete answer to "what does a static graph buy that scheduling can't." Prerequisite = edge provenance (which arcs are semantic/immovable vs orientation-choices), which is needed here regardless of 2.2's parked annotation work — do provenance with this track; the flip lever for non-commutative pairs stays gated on the (parked) commutativity annotation. Names for nodes/objects already exist, so provenance labeling is cheap — but review the naming approach first (see Inconsistency 6:ts::Namednot yet ratified). - Validation fixture: game_frame (hand-derived chain ≈2.6 ms, 26.9 ms work); a simulator must reproduce measured timelines before its proposals are trusted.
- Measurement methodology is prerequisite #1: interleaved runs (A/B/A/B to cancel drift), medians over N, controlled load, and the per-lane timeline capture from the graph-viz session (docs/graph-viz-handoff.md) — without this, optimization conclusions are noise. Ties to 10.1 (benchmark regression infra). Capture landed (2026-07):
-
[ ](P2) Compile-time rank → native dispatch shaping. Compute upward rank per node atcompile()(longest path to sink; weights = node priority now, measured durations once 2.4's capture exists; reuses the Kahn order, O(V+E) once). At run the graph shapes dispatch WITHOUT the scheduler's priority classes — avoidinghigh's global-queue detour andlow's valve (2.4's blockers): (a) rank-aware successor submission order when one completion releases several ready nodes (owner deques pop LIFO — submit lowest-rank first so the settling worker pops the highest; thieves take the other end); (b) inline-successor selection by rank (set_inlinecurrently picks without regard to importance). ConvertsGraph_node::set_priorityfrom a queue-class request into a rank weight. Related: D6's static effective-priority pass should ride this mechanism (not the Priority classes) until M2 stage 5; priority-as-conflict-tiebreak belongs behind 2.2's opt-in (it changes observed values). Sequence AFTER the graph-viz timing capture — 2.4's dry-run showed dispatch-shaping effects are unmeasurable without it. -
[ ](P3, downgraded 2026-07 — author unconvinced; revisit on a demonstrated disabled-writer-stalls-readers case) Per-run node enable predicate (conditional execution). The research pass's one design challenge (research-static-vs-dynamic.md — every mature static-graph system grew a dynamic escape; pure DAGs fail on data-dependent control flow first): the graph has no in-graph skip/branch. The cheap 80%: a per-run predicate on a node, evaluated before acquisition — a disabled node acquires nothing, completes immediately, releases successors (render graphs' conditional-pass-execution pattern). Preserves acyclicity, the derived-edge story, and the safety model; deliberately NOT Taskflow-style condition tasks (their weak/strong-dependency semantics carry documented race/deadlock pitfalls). Full loops/branches stay out of scope — dynamic tasks andco_awaitare the escape for data-dependent shape, as designed. -
[x](P1) Serial execution mode — DONE as GLOBAL worker-less mode (2026-07). Rescoped from graph-onlyexecute({.serial=true})to the UE shape after reading UE's source (zero workers when threads unavailable;LaunchInternalexecutes inline at launch, looping the continuation chain):Scheduler_config{.single_threaded = true}= no workers, every submit executes inline on the submitting thread via a bounded FIFO trampoline (per-thread pending vector + shared head, reentrant); blocking waits drain the thread's pending entries before parking (drain_serial_pendingseam in task.h → scheduler.cpp), so a body that admits work thensync()s it doesn't deadlock. Serves debugging/bisection, deterministic tests, thread-less platforms, low-end fallback. Enabler shipped with it (Tier 0): pipe job submission moved OUTSIDEpipe.mutex(dispatchcollects the admitted batch under the lock;submit_admittedafter unlock) — inline-at-submit would deadlock on a body releasing its own pipe under the held mutex. Worker-less graph runs skip the trace fold (their timings describe the trampoline, not scheduling).executenow takesExecution_options{.token}(no bare-token overload — author decision); 2 call sites migrated. Graph-scoped serial (.serialper run, compiled-topo order, rest of app stays parallel) DEFERRED — revisit only if serialize-one-graph-in-a-live-app materializes; it would reuse this routing core plus a TLS gate. Tests: inline-at-submit, FIFO chain order, end-to-end (launch/then/async/parallel_for/graph+nested all on the caller thread), sync-inside-body drain, deterministic run-twice order; 486 checks green, Shipping + stress clean. -
[x](P1) Graph/Guardedlifetime fatals — DONE (2026-07). Two fatals, fullyTS_SAFETY_CHECKS-gated (field included —Pipe::graph_refscompiles out entirely, per author): (1)~Static_task_graph(and a move-assign overwrite, via destroy+placement-new move-assign so member moves stay maintenance-free; andcompile()on recompile) fatals while a run is in flight (run_->remaining_nodes != 0— inits to 0, so pre-first-run destruction is clean); (2)compile()registers the graph on everydistinct_pipes_pipe, the same paths release, and~Guardedfatals while any compiled graph still references it, naming the object viadebug_name. Moves are balanced by construction (registrations ride thedistinct_pipes_vector; moved-from is empty) (research-deepdive.md §10.2; Taskflow #82 precedent). Tests:guarded_outlived_by_graph+graph_destroyed_mid_rundeath scenarios, and a recompile/move/move-assign-overwrite balance test; 490 checks green, Shipping + stress clean. -
[ ](P3) Over-declaration diagnostic — report objects a node declared but never accessed (silent lost parallelism; RDG's unused-declaration warnings are the precedent — research-deepdive.md §7.4). Caveat (author, 2026-07): conditional early-out bodies legitimately skip declared accesses — aggregate across runs ("never accessed in any of N runs") and provide a per-node "may-skip" annotation before warning. -
[ ](P2) Publish cost numbers — measuredcompile()cost, per-node dispatch cost (queued /set_inline/ handoff), and a "merge nodes below X µs" guideline; makes per-frame rebuild a measured option instead of an anti-pattern and answers the first sophisticated-evaluator question (research-deepdive.md §9.4, §12.2). Ties to 10.1. -
[x](P2, author 2026-07 — DONE 2026-09) Yield points inside long-running nodes. The 4-worker game_frame trace made the failure concrete: a ready critical-path node (economy) waited ~0.9 ms behind a long off-path runner (audio) — nothing evicts a runner, and priority cannot help work that is already running (measured: audio atlowchanged neither its placement nor the gap; occupancy, not arbitration). A cooperative yield point inside a long node body — splitting it into resumable segments the scheduler can interleave ready higher-rank work between — is the eviction-shaped lever. Design space: explicitts::yield()polling a "critical work pending" signal vs coroutine nodes (co_awaitsuspension already exists for tasks) vs auto-slicing viaparallel_forguidance. Relates to 2.4's keep-out-zone hypothesis and 2.5's rank (what "higher-rank pending" means). Landed (2026-09):ts::yield()(task.h) runs one queued entry of a higher class inline (normal->high;low->high, then the globalnormalqueue) on the yielding worker's stack and returns - no suspension, so functor nodes andparallel_forbodies can yield, and the continuation keeps its core;parallel_foryields at every chunk claim. Rationale: design.md §3 "Yield points". A resume the nested task causes runs at the yield point, also inside a resumed segment (the nested dispatch sets the resume and inline trampolines aside; nesting is bounded to one level). Open: a pending signal finer than the priority classes (2.5's rank). -
[ ](P2, author 2026-07 — raised from the 2.3 adjacency) Frame-boundary overlap for designated nodes. Distinct from 2.3 (whole-graph pipelining): let specific off-path tails (audio mix, streaming finalization) spill past the run's settle into the next frame's early idle — per-node "may straddle the frame fence" annotation rather than two full runs in flight. Interacts with the run-quiescence invariant (one run at a time, full barrier between) — likely needs the spilling node to detach into a tracked dynamic task at the fence. Speculative; validate demand on the 4-worker fixture first. -
[x](P1, author 2026-08) Declaration order must not carry intent — explicit edges for logical ordering — DONE (2026-08). Author's ruling: we rely on the determinism of a COMPILED graph, not on node declaration order. A derived edge exists for safety (no data race); either direction is race-free, so an optimiser is free to reorder independent conflicting nodes. If node A must logically precede node B, that is intent and must be an explicitafter/beforeedge. (a)sample/game_frame.cppaudited. Two intent-bearing sets found, both now explicit in BOTH variants: the render producers intosubmit(the known set — it was declared explicitly only in the optimised variant, where staging removes the conflict, and left to declaration order in the baseline), and — the set the audit added — every last-frame transforms reader ahead of the flip. The sample's own comment said the render pipeline reads last frame's transforms because it is "declared BEFORE the flip", which is exactly declaration order carrying intent;flip.after(frustum_cull, occlusion_cull, shadow, cmd_record, particles, UI_node, submit, audio, vfx, debug_overlay[, cloth])says it instead. Every one of these coincides with a derived edge in the same direction, socompile()dedups and the schedule is unchanged (transform0 == 5and 2,700,000 draw commands held in both variants). (b) Rule stated indocs/guide.md§6.5 anddocs/design.md§2.3, with the two reasons it is load-bearing: an intent edge survives a refactor that deletes the conflict (which is exactly what the optimised variant'sDeferredstaging does), and the reordering optimiser is only available while no program has made declaration order load-bearing. (c) Thecompile()diagnostic is split out as 2.15 — deferred by the author, 2026-08. -
[ ](P3, author 2026-08 — split out of 2.14, deferred)compile()diagnostic for direction-only ordering. A diagnostic that flags a derived edge whose direction is the only thing ordering two nodes, so reliance on declaration order is visible rather than implicit. Enables the reordering optimiser the 2.14 rule is reserving room for: with the diagnostic, a graph can be checked for latent reliance before the optimiser is allowed to reorder anything. Deferred by the author when 2.14 landed — the rule and the sample audit were the valuable parts; the tooling can follow if reliance turns out to recur in practice. -
[ ](P3, doc + demonstrate — T20) Pre-compiled graph variants pattern. For discrete mode sets (game-loading vs in-game driving vs boss fight vs cutscene), the intended answer to occasional shape change is "keep N compiled graphs, pick one per mode" rather than rebuild machinery (research-deepdive.md §9.4). Document the pattern — but only worth promoting if a fixture demonstrates a significant per-variant scheduling difference (a cutscene graph genuinely out-scheduling the driving graph on the same work); build that demonstration before writing it up, else it is unsubstantiated advice. -
[x](P2, author 2026-08 — DONE for the four proven levers; one reverted) Borrowed-pointer node dispatch fast path + slim node completion. LANDED 2026-08 (commits: cache scheduler / borrowed dispatch / skip advance_pipe_links / slim completion). Measured on the same harness (re-measured baseline 662 ins/node on the current clang 21 toolchain; the 648 below was an earlier commit/build):- (b) cache scheduler pointer per run — a run resolves
global_scheduler()once intoRun_state::schedulerand dispatches its nodes through it. −23 ins/node. - (a) borrowed-pointer dispatch — a
flags.borrowedbit (set on every node block atcompile()) routes the queued node dispatch throughsubmit_borrowed/run_borrowed_dispatch, which carry a raw pointer with no dispatch-hop refcount (the block is owned byRun_statefor the whole run; runs are sequential/joined, so it cannot be freed mid-dispatch). Async / coroutine / multi-object blocks keep the owning path. −24 ins/node. - (c) skip
advance_pipe_linkswhenpipe_count == 0. ~−2 ins/node. - (1 in the cost map, the BIGGEST lever) slim node completion —
settle()fast-paths onflags.borrowed: a node block has no externalsync()/co_awaitwaiter, no continuations, and is never a nested child, so it fireson_completedirectly under the atomic flags, skipping thestd::mutex, thedone_cv.notify_all()that wakes nobody, and the empty continuations drain (aTS_SAFETY_CHECKSassertion guards the invariant). −153 ins/node. Total 662 → 459 ins/node (−30.7%), below the 522 pre-transformation block; wall-clock ~53 → ~45 ns/node (~15%) on the worker-less machinery microbench. TSan-gated (fulltsan/run.shclean + a high-exposure focused loop showing zero new races). Still open: (4) re-arm field resets — ATTEMPTED and REVERTED (flag-byte-store + skip-empty-token; gain sat below the measurement noise floor, not worth touching the hot re-arm path). (2) fullTask_ptrrefcount removal beyond the dispatch hop and (3) schedulersubmit/run_serial~80/node are untouched (structural). Historical root-cause analysis retained below. Callgrind (deterministic, 1000-empty-node worker-less driver, two Shipping-like builds; full report graph-regression-callgrind.md) root-caused the ~9%/node graph-execute regression vs the pre-transformation block: 522 → 648 instructions/node, and it is NOT the completion/add_nested/execution_flagpath (byte-identical, settle is cheaper now) but the ownership-carrying dispatch trampoline — coroutine-first unified node dispatch onto the generic refcounted block path (submit_ready(Task_ptrby value = atomic inc)→run_block_dispatchadopt+dtor = atomic dec, plusTask_ptrchurn), where the baseline queued a borrowed rawNode*with no refcount. Recoverable levers, in impact order: (a) borrowed-pointer dispatch for plain functor / zero-nested nodes — the block is owned byRun_statefor the whole run (exactly what the baseline relied on), so per-dispatch refcounting is redundant on the common case; keep the refcounted path only where a node body genuinely outlives its return (aco_await, an innerexecute()lend). The big lever. (b) cache the scheduler pointer per run (baseline cachedrun.scheduler; current re-resolvesglobal_scheduler()per dispatch, +11/node). (c) skipadvance_pipe_linkswhenpipe_count == 0(+3/node on object-free nodes). Concurrency-sensitive (touches the dispatch path) — TSan-gate any change. Optional: the frame-level cost is <1% (design §2.5), so this is perf recovery, not a fix — but the levers are low-risk and the mechanism is fully understood. Full per-node cost map (callgrind, current master, ~593 ins/node of framework on a trivial node — 97% machinery, 3% body). Ranked self-cost, and note the fattest target is a STANDING inefficiency that largely PREDATES the regression, so a bigger absolute win than 2.16's regression recovery: (1) node completion/settle ~183/node (31%) — every node'ssettle()takes astd::mutexand callsdone_cv.notify_all()though nothing waits on an individual node's condvar (only the run's terminaldonehandle issync()ed), plus an always-emptycontsvector ctor/dtor; slim graph-node completion to fireon_completeunder the atomic flags, skipping the mutex+condvar+vector. Inherent to the genericTask_control_blocksettle path, not graph-specific — a broad lever. (2)Task_ptrrefcounting ~134/node (23%, the 2.16 regression). (3) schedulersubmit+run_serial~80/node. (4) re-arm field resets ~68/node (batchable). (5)global_scheduler()per dispatch ~11/node. Targets 1–3 could reclaim ~half the per-node machinery for the functor-node case without touching the async/coroutine paths. Full report: graph-regression-callgrind.md.
- (b) cache scheduler pointer per run — a run resolves
-
[x](P2, author 2026-08 — DONE at the C4 review, 2026-08-17)execute()in a loop withoutsync()— API reconsideration. Resolved as[[nodiscard]]onexecute()(the handle is the run's only completion signal and the one-run rule's sequencing device; deliberate fire-and-forget spells(void)) plus a hardened in-flight fatal (the guard now also requires the previousdonehandle settled -remaining_nodesalone had a race window at the settle boundary). Implicit join was rejected (would block inside a task); run overlap remains 2.3. Original question:graph.execute()returns aTask<void>the caller is expected tosync()/co_await; one run per instance is the rule (a secondexecute()while a run is in flight is fatal,TS_SAFETY_CHECKS, theRun_state-corruption guard). Sofor (…) graph.execute();with no join between iterations trips that fatal in checked builds (and races theRun_statewith checks off), while the returned handle is silently discarded. Discuss whether the fatal is the right UX or the API should guide harder. Candidate directions:[[nodiscard]]on the return so a dropped run is at least flagged; a sharper fatal naming "previous run still in flight — sync() it"; or serializing/queueing back-to-back runs (which is 2.3 pipelined execution, the general fix). Decide whether this is a docs +[[nodiscard]]fix now or waits on 2.3. -
[ ](P3, author 2026-08 — parked until real-life usage data)compile({.derive = off})experiment: dynamic (arrival-ordered) conflict edges. From the static-vs-dynamic scrutiny session: keep the graph for declarations, per-node acquisition, and explicitafter/beforeedges, but skip conflict-edge derivation — conflicting nodes race to their pipes and the pipes arrival-order them (the cascade's canonical acquisition order keeps it deadlock-free). What it trades: per-pair-per-frame orientation nondeterminism (value jitter, no replay) for serializing only the actual accesses instead of the declared conflict set — the one real structural perf cost of derived edges (a declared-but-untouched write still serializes every reader). The architecture separates acquisition from ordering cleanly enough that this is a small, falsifiable experiment;game_framegives it a number, but per the author, don't derive significant decisions from toy-sample perf (the graph-free measurement — slower by +1.6%/+28.7% via resume locality — is one frame shape, not a verdict; logically dynamic edges absorb runtime fluctuation better, keep thinking). Two findings from the same session to carry with it: (a) derived edges can MASK missing intent edges (the sharpened reading of the graph-free draw-calls bug): the baseline'scmd_record→submitorder was held by a coincidental derived edge; whenDeferredstaging removed the conflict in the optimised variant the edge evaporated and only the explicitaftersaved the order — so the static graph fails late (after an unrelated refactor removes the conflict) where the dynamic world fails fast (day one). Companion diagnostic candidate: a compile-baseline edge-set diff —compile()compares against a stored baseline and reports removed derived edges ("no longer derived; if the order was intent, ratify withafter"), catching the loss at the moment the accidental protection disappears. Relates to 2.2/2.14 (unratified orientation). (b) How much the profiling/PGO story (2.4/2.5) actually depends on derived-edge determinism is unknown — average-frame stats over arrival-ordered conflicts are untested; at some point measure whether the trace/critical-path machinery survivesderive = off. Overall stance (author): most of these decisions need real-life data — defer until there is a real consumer codebase to measure. -
[ ](P2, author 2026-08-18 - from the C4 lend review, item 14) Harness check: caller touches a lent object while its lent nested run is unsettled. The lend containment argument assumes the caller awaits the run: an un-awaited lent run and the caller's own continued body can both touch the lent object "validly" (both carry covering contexts), and the harness cannot see it - the v1 non-quiet-scope fatal only fires at the SECOND lend. A direct fatal at execute() is impossible (whether the caller will await is not knowable there); the enforceable shape is a check at the caller's next guarded access: fatal if the object is lent to a nested run still in flight in the caller's scope (needs bookkeeping - the lent set is per-graph and re-bound per run; the caller-side check would consult the scope's unsettled children's lend sets). Until then the rule is documented discipline:co_await inner.execute()directly (the execute() comment and guide state it).
-
-
Scheduler 0.
[x]Single global scheduler — DONE (2026-07,d173d9a+2b48a5b; lifecycle API reworked 2026-08). Author chose single-global over the ambient-multi model (3.1). Exactly one scheduler process-wide, reachable viaglobal_scheduler(). The lifecycle is EXPLICIT, not lazy — a scheduler is heavy, so the subsystem-init shape applies (SDL_Init/SDL_Quit):ts::create_scheduler(Scheduler_config = {})brings the pool up (fatal if one already runs),ts::destroy_scheduler()signals quit, joins the workers and drains (fatal if none runs),ts::scheduler_running()answers whether one is up,ts::current_scheduler_config()returns the config it was created with, andScheduler_scope(config)is the RAII form (creates, or reconfigures by teardown+recreate if one is already running, and restores the previous config on exit) — the way to run a block on a specific worker count / idle policy.global_scheduler()FATALS when none is running rather than lazily constructing one, so a missingcreate_scheduleris a sharp startup error instead of a silently default-configured pool. All of these are coarse quiescent-point operations, not thread-safe against concurrent use.execute()drops itsScheduler&arg (uses the global);parallel_forfans out onglobal_scheduler(). Supersedes 3.1's ambient-override idea and itsLaunch_options{.scheduler}(nothing to select among). Residual (P3): compile-time ban on ad-hocts::Scheduler— DONE (2026-08). TheSchedulerctor is PRIVATE; the sole construction path is adetail::make_scheduler(config)factory (friend; returnsunique_ptr, class non-movable) used by the global holder, sots::Scheduler s{...}no longer compiles. Went further than the original plan: rather than migrate the unit tests to the factory (which would still stand up rival pools), every ad-hoc site (scheduler_tests.cpp, benchmarks, tsan,parallel_tests.cpp) reconfigures the GLOBAL for its scope (Scheduler_scope), so at most one worker pool ever runs. Thethread_local current_scheduler"which scheduler" selector was retired entirely —current_worker_index >= 0is the sole worker-vs-external discriminator (fast path,parallel_forfan-out, profiling bridges,Parallel_recorder::lane);current_or_global_scheduler()collapsed toglobal_scheduler(). Verified: Release, Debug and Shipping suites green, bench in-band, --stress deterministic, TSan clean.-
[~](P1, superseded by 3.0) Ambient (overridable) scheduler — was:launch/task/accessresolve to an ambient scheduler +Scheduler_scopeoverride +Launch_options{.scheduler}. The single-global model (3.0) took the simpler road: one reconfigurable global,Scheduler_scopereconfigures it rather than overriding among many. Kept for the record. [§D3] -
[ ](P3, rescoped 2026-07 — author: avoid named threads as much as possible) Task→worker affinity (escape hatch, NOT named threads). Named threads are the anti-pattern the access-declared graph exists to replace: a named render thread encodes "render after game, on this thread" as a thread boundary, where the graph encodes the same ordering as a dependency edge derived from data access — which scales (submission fans across all workers) where the thread does not (serial funnel, cores that can't lend to each other, cross-thread hand-off latency). The sample'sVersionedrender pipeline reading last frame's transforms IS "the render thread with one frame of latency," expressed as tasks with a version-choice dependency rather than a pinned thread — the named-thread-free pattern, generalized. So do NOT model named threads. The only real need is IRREDUCIBLE affinity — single-thread graphics APIs (GL, older D3D), OS main-thread requirements (macOS UI / window message pump), external libs that demand a fixed thread — met by a minimal "pin these tasks to worker N" mechanism (a reserved/pinned worker or an affinity tag) inside the one scheduler, off by default. NOT a separateSchedulerinstance: consistent with the single-global-scheduler model (variety lives inside the scheduler type, one live instance). Replaces the old "named thread as a single-workerScheduler" framing. -
[ ](P2) Run-on-all-workers — broadcast a functor to every worker (per-thread init / flush / warm-up). -
[ ](P2) Timed / delayed tasks — delay queue (timing wheel); one-shot / periodic / cancellable; re-arms a reusable task. -
[ ](P2) M2 stage 5 — promote high/low to per-worker deques (profiling-gated); a proper low-contention worker-submit benchmark. -
[ ](P2) Platform abstraction (~6 fns) —park/unpark(+timeout), thread spawn/name/affinity, cpu_count/topology,cpu_relax. Unblocks eventcount timeout + standby workers; console fiber backend later. [§D4] -
[ ](P2, raised 2026-07) Pluggable scheduler + a foreground/background two-pool variant. TheScheduleris one implementation of an interface; ship alternatives behind the sameTaskAPI (the ambient-scheduler work in 3.1 is the seam). The motivating variant: a separate background worker pool on OS-low-priority threads, deliberately OVERSUBSCRIBING the cores. The idea is OS-scheduler-assisted gap filling — background threads run only when the foreground pool leaves a core idle, and the OS preempts them out the instant foreground work is ready, so background work costs the foreground nothing (unlike userspacelow, which can't evict a runner — the exact failure we measured:audioatlowstill occupied a core the critical spine needed). This is distinct from 3.5 (promoting high/low to per-worker deques) and from D6 (userspace priority propagation): it moves the arbitration to the OS for the background band only, where preemption is legitimate. Prior art + adjacent notes: UE's two-pool foreground/background design with per-pool OS priority + affinity (task-systems-comparison.md §UE), the mobile QoS-over-fixed-pool note (§D4), and D6's "a fixed per-pool QoS-background attribute is the only legit OS-priority use." Costs to weigh: thread explosion / oversubscription tuning, the syscall/permission cost of OS priority, affinity interplay, and determinism (background timing becomes OS-dependent — keep it off the traced/asserted paths). Also relevant to this session's finding that a normal task buried in a local deque loses to globallowbecausefind_worksteals last (see 3.5 and the pending steal-before-global-low reorder) — a two-pool split would let the background band be genuinely non-interfering instead of merely deprioritized. Scope guidance (author, 2026-07, T26): ship at most 2–3 interchangeable scheduler implementations behind the sharedTaskAPI, not a plugin zoo. Each may expose a wake/idle policy (as the current one does) but nothing fancier at this stage. The engineering constraint that matters: factor the scheduler so its reusable parts (eventcount, deques, MPMC queues, valve, the submit/steal loop) are shared building blocks a new implementation composes, rather than each scheduler being a monolith — a fancy future scheduler should reuse most of what exists. Design-doc note added (design.md §3). Two distinct oversubscription mechanisms (UE, verified from source — keep them separate, both are SCHEDULER-impl concerns behind the unchangedTaskAPI, NOT task-layer): (i) background-band gap-filling — the variant above: a standing low-OS-priority pool the OS preempts out the instant foreground work is ready. (ii) dynamic oversubscription around a blocking region —FOversubscriptionScopebumps a count that wakes a parked standby worker (or spawns one under dynamic thread creation,IncrementOversubscription); surplus workers self-retire viaConditionalStandbywhen the count drops. (ii) keeps the pool alive when a task genuinely BLOCKS (mutex / I/O /Waiton unrelated work) — the case our current scheduler does NOT cover. (2026-08: awaits no longer belong on that list — aco_awaitsuspends the frame and frees the worker, and retraction is gone. What is left uncovered is opaque blocking inside a body, with no suspension point for us to use; see 5.3.) (ii) is the prior art for closing that gap in the alternative scheduler; see 5.3. -
[ ](P3, micro-opt, raised 2026-07) De-correlated per-worker park-spin timing. UE gives each worker a distinct prime-ish spin duration from a small table ({719,991,1361,…}[worker_id % 8], theYieldCyclesarg to the pre-park spin) so parked workers don't re-scan the queues in lockstep — a cheap thundering-herd mitigation. Ourspin_then_block/handoffidle loops spin a uniformspin_cycleson every worker. Cheap to add (a per-worker offset seeded at construction); measure whether it cuts contended eventcount re-scans under a fully-parked burst/drain. Prior art: UEFWaitingQueue::Park(task-systems-comparison.md §UE). -
[ ](P1, evidence 2026-08-18) Investigate the 2x-worker throughput collapse. Across two unrelated exploratory bench workloads (2026-08-18, since removed - a global-bottleneck flood and a multi-store contention mix), every row run withnum_workers = 2 x hwcollapses to a near-constant ~0.5-0.63 M tasks/s - regardless of lock mechanism, contention level, or task shape (even the uncontended uniform control), while the same workload athwworkers runs 2-4x faster. Workload-independence points at the scheduler, not the workload; prime suspect: with more workers than cores, some workers are always parked, so every producersubmitsees sleepers and pays a wake syscall (signal_submit), capping the flood producer at roughly 1/wake-cost - but that is a hypothesis, not a diagnosis. Matters beyond benches: any user configuring workers > cores (the oversubscription mitigation this library tells mutex users they no longer need) hits it. Diagnose with an empty-task flood at hw vs 2x across idle policies; until then, oversubscribed bench rows are not valid proxies for pool-oversubscription behavior. -
[ ](P2, external review 2026-08-28) Stop-token-aware workers for partial-construction unwind. If constructing worker N throws (thread-creation failure at startup, exceptions-on builds only), the scheduler dtor never runs and the N-1 built workers'~jthreadjoins hang:request_stopis inert because the worker loop checks onlyquit_. Fix shape (audited): give the worker lambda astd::stop_token, register astd::stop_callbackthat notifies the eventcount, and extend the exit check withst.stop_requested()- one extra bool on the idle re-scan path only. Exceptions-off configs terminate at the throw instead, so this is a niche startup-resource-exhaustion hang, not a shipping-config defect. Companion finding, decided as documented behavior: the MPMC overflow deque drains only when the ring empties (unbounded reordering under permanent saturation, self-healing on any dip) - the header states it; no change.
-
-
Allocation / control block Prior art (UE
FTask, verified from source — concrete shapes for the items below):FTaskis exactly one cache line (LOWLEVEL_TASK_SIZE = PLATFORM_CACHE_LINE_SIZE) with the body stored INLINE via an SBOTTaskDelegatesized to the remaining bytes — validates 4.2 (size the tunable-SBOFunctionbuffer so the common task functor never heaps). Per-size-class recycling isTLockFreeFixedSizeAllocator_TLSCache<256, cacheline>(TLS-cached, fixed-size, lock-free) — the concrete shape for 4.1 (eachExec<Body,R>/Result_block<R>instantiation is constant-size per type, so one free-list per size class). An oversized/overaligned closure falls back to a 64 KB-block linear (arena) allocator (TConcurrentLinearAllocator) — validates 4.6's SBO-overflow-to-arena path. Executable-task init refcount is 2 (one external handle + one in-system).[ ](P2) Per-type recycling free-list (Exec/Result_block/bare block).[ ](P2) Tunable-SBOFunction<Sig, N>replacingmove_only_function/function(also fixes the reservation-path closure alloc — inconsistency #5).[~](P2, mostly overtaken 2026-08) Small-vector / intrusive links for the block's edge vectors.prerequisitesis gone (deleted with retraction — it existed only for the retraction walk), pipe turns never touched the vectors, andsuccessorscollapsed to a singlenested_parentpointer: 320 → 280 → 264 B. Onlycontinuationsis left, and its fix is 4.7 step 2 (an intrusive waiter list with dependent-side nodes), not a small-vector.[ ](P2) ShrinkTask_control_block—completed/cancelled→bits; a futex wait primitive roughly halves the block (now 264 B, of whichstd::mutex+condition_variableare the dominant term; ties to the platform layer).[ ](P3) Multi-objectasyncstd::map→sortedvector.[ ](P2) Opt-in scoped bump arena (auto forparallel_for/graph-run, per-frame opt-in); rebasejournal.hstaging onto it.[ ](P1, author 2026-08) Reviewcontinuations; fold intosuccessors(then-as-normal-task) + intrusive edge storage.Task_control_block::continuations(type-erasedmove_only_functioncallbacks fired at settle) is a THIRD dependency mechanism next tosuccessors/prerequisites— it predates thethenrebase and doesn't feel right. Target:thenlaunches a normal task everywhere (it already is a real block with the producer as anum_locksprerequisite — audit the remainingattach()users:when_all's join, multi-async release,Deferred/Versionedinternals) socontinuationsfolds intosuccessorsand the block loses one of its three vectors. Pair with the storage fix (supersedes half of 4.3): a dependent knows its edge count at wiring time, so embed the edge nodes in the DEPENDENT's allocation and thread the producer's successor list intrusively through them (producer holds one head pointer, allocates nothing) —successorscan never be a fixed array on the producer (fan-in is unbounded), which is exactly why the storage must live dependent-side. Nested-task edges stay dynamic by nature (data-dependent fan-out, count unknowable at allocation). The pipe rebase (1.14, branchpipe-rebase) already makes pipe-turn prerequisites vector-free (embeddedPipe_links + thepipe_counttrigger; nosuccessors/prerequisitestraffic); this item does the same for ordinary edges. Touchesthen/when_all/settle/retraction — its own project, after the pipe rebase lands. Re-scoped (2026-08, post-6.4). Both landed prerequisites are in: the pipe is rebased (1.14) andthen/when_all/retraction are deleted, which tookprerequisiteswith them (block 320 → 280 B). Step 1 DONE (2026-08):successors→ a singlenested_parentslot, 280 → 264 B (clang-cl x64, measured). The audit that motivated the note foundsuccessorshad exactly one producer left —add_nested— and it runs once per child (a coroutine node's frame and a nested graph run are each attached once), so the fan-out was structurally 0 or 1. Astd::vector<Task_ptr>was costing 24 bytes plus a heap allocation on first push to hold one pointer. Now a bareTask_ptr, with aTS_SAFETY_CHECKSfatal on a second attachment (which would previously have silently queued a second parent, and would now silently drop one). Step 2, the remainder — designed, NOT implemented.continuations(std::vector<move_only_function<void(void*, bool)>>, 24 B + a closure allocation past SBO) is the last edge vector. Three users: the coroutineTask_awaiter(one per awaiting frame),Join_awaiter(one per joined child), andVersioned::publish's internal chain. Fan-in is unbounded in principle (N coroutines may await one handle) but is 0 or 1 in every measured path. The target shape is an intrusive waiter list with the nodes in the DEPENDENT: an awaiting frame knows its own edge atawait_suspendtime and already has a frame to embed the node in (this is exactly where 6.2's fusion pays off a second time), so the producer keeps one head pointer and allocates nothing, andsettlewalks a list instead of draining a vector. Estimated win: 16 B of block plus one allocation per suspended await. Why it is not done here: unlike step 1 it is not a local change — it rewritessettle's drain (the most concurrency-sensitive function in the core, whose current shape is load-bearing: everything is moved out UNDER the lock and fired after the notify, for the reset-race reason documented at the site), and it must re-prove the awaiter's two-state resume handshake against a list the awaiter now owns storage for. That is a project with its own TSan campaign, not a slimming pass. Sequence it after 4.1/4.2 (a size-class pool and an SBO callable would independently remove the allocation half, which may make the remaining 16 B not worth the risk — measure first). Note (author, 2026-08) — "the shipping block size" is no longer one number. This item has been assuming 264 B is what ships. That stops being true once users take up the standing invitation to leave part of the harness enabled in a shipping build, because two independent things then follow the rule policy rather thanTS_SAFETY_CHECKS: (a) the task'sNamed(24 B,task.h) is currently gated onTS_SAFETY_CHECKS, but the rules that print task identity are not all dev-only —deadlock_net(6.13) is available in shipping and its entire value is dumping the suspended frames, which without names is a list of pointers;in_task_sync(6.10) is likewise shipping-available. So the field's gate should follow "is any name-printing rule compiled in", notTS_SAFETY_CHECKS, and the block's shipping size becomes a function ofTS_ENABLED_RULES. (Object and node names are already kept in shipping, soaccess_rank's and the circular-wait detector's object naming is unaffected — this is only about task identity.) (b) 6.14's rank field onGuardedhas the same shape. Consequences for this item: size targets must be stated per rule-policy, not absolutely; and the interning idea rejected in the 2026-08 naming discussion (a global registry + an 8-byte pointer, dismissed because shipping paid nothing for names) acquires a real target in exactly this configuration — that was the recorded trigger condition, and this is it. Do not act now; settle it when 6.13 lands and the shipping-harness story is concrete.[ ](P2, author 2026-08) Cache-line alignment audit across components. A systematic pass over every hot shared structure for cache-line placement: separate fields written by different threads onto distinct lines (alignas(std::hardware_destructive_interference_size)where warranted), keep fields read/written together on one line, and check array elements for false sharing between adjacent entries. Inventory to cover:Task_control_block(the size-ordered cluster is packing-motivated, not sharing-motivated — line 0 packs the write-hot atomicsrefcount/num_lockstogether AND with the read-only dispatch-read setexecute/result_ptr/pipe_links/flags/token, trading one-line dispatch locality against handle-churn/indegree writes false-sharing that line;refcountis ~23% of per-node machinery per the graph-regression callgrind, so measure whether isolating the write-hot atomics onto their own line beats the locality win), the evolvedPipe(all queue state now lives under one mutex, so the interesting question shifted: iswriter_owner— read lock-free by everycommit()ownership check — on the right line relative to the mutex and the queue head, and does a coroutine frame's embedded link share a line with hot promise state),Pipe_linkarrays (adjacent links of one task live on one line; different lines' traffic collides — measure before padding, links are per-task not global), scheduler queues/deques (Chase-Lev top/bottom, MPMC slots;Busy_slot/Bucket_roware already padded — verify the rest), journal slots,Event_count. Measure with the existing benchmarks (contention series + R10 pipe fixture) — padding trades memory for isolation, so each change needs a number, not a vibe. Reaffirmed + rescoped (author, 2026-08): the WHOLE implementation, not just the inventory above — treat every hot or shared structure ininclude/+src/as in scope, and rederive the field-level analysis from the CURRENT layout rather than the specifics written above, which are now stale:Task_control_blockchanged twice since (the coroutine-first slimming to 264 B, then B2 droppingdispatch_arg/collapsingrun_state→body_claimedto 248 B, plus theTS_DEBUG_NAMES-gatedNamedfield), so thenum_locks/refcountline-sharing claim must be re-checked against the real struct. Dump the actual offsets (astatic_assert(offsetof(...))probe or the debugger's layout view) as the first step, identify which fields are touched by which threads on the hot paths (settle vs dispatch vs refcount vs the harness), then place the destructive-interference boundaries — measured, per the rule above. Do this after the perf-baseline/graph-regression work settles (it may itself surface a false-sharing culprit worth folding in).[ ](P3, 2026-09) Allocation-freePeriodic::next(). Eachnext()allocates one timed block: the returned task, with the timer bookkeeping embedded. APeriodiccould own one block and re-arm it every tick, the waySignal::reset()re-arms a signal, which makes a steady-state tick allocation-free. It needs a reusable settle for an executable block (result and state reset while no awaiter is attached), which blocks lack since executable-task reuse was deleted with the coroutine-first transformation. A 60 Hz clock costs 60 allocations a second today, so this waits for a workload with many clocks, or for the per-type free-list (4.1), which would absorb the cost without an API change.
-
Fork-join / parallel_for
[ ](P2) Intra-system entity interactions — ship the primitive menu:parallel_gather_apply(mailbox),parallel_for_colored+Interaction_coloring,Accumulator(commutative),Union_findhelper, + triage docs. Open author questions. [§D5][S]Priority propagation / inheritance — designed (pipe / graph / dynamic; opt-in surfaces; not OS thread priority); revisit on a demonstrated inversion. [§D6][ ](P3, re-scoped 2026-08) Reserve / standby workers — sequenced after the platform layer. The framing changed with 6.4: retraction is gone, and a coroutine wait SUSPENDS rather than blocking, so the fork-join case this used to cover is structurally handled (a suspended frame holds no worker). What remains is genuine blocking a task system cannot see through — an OS mutex, a synchronous file read, a driver call inside a node body — which still occupies a worker with no suspension point. UE's answer is dynamic oversubscription:FOversubscriptionScopearound the blocking region wakes/spawns a standby worker (IncrementOversubscription),ConditionalStandbyretires it after (verified from source). This is scheduler-impl territory — belongs to the alternative two-pool scheduler (3.7), not the task layer; both mechanisms sit behind the unchangedTaskAPI.[ ](P2, author 2026-08 — raise at the parallel_for review) Reconsiderparallel_forinheriting the caller's grant (esp. a WRITE grant). The helpers all carry the node's grant, so each passesTS_CHECK_ACCESSon the shared object - but the harness does NOT serialize helpers against each other. Data-race-freedom rests entirely on the disjoint-partition user contract (helper k writes disjoint slices), exactly likestd::for_each(std::execution::par). A body that writes a shared location (accumulating into one member, an overlapping range) races silently, undetected by the harness - the completeness/granularity footgun, but now manufactured by the fan-out. Revisit: is grant-inheritance the right model; is a guard/diagnostic for overlapping writes feasible (hard in general - it is the parallel-for aliasing problem); or is documenting the disjointness contract sufficient? Raised during the C2task.hreview (parallel_for inherits viasnapshot_access+Inherited_access_scope, joins synchronously so the grant window stays open).
-
Coroutines
-
[x]DONE (2026-08, subsumed by 6.4). Inline-when-free for awaited accesses is what the awaitable access verb does:co_await obj.access(fn)isawait_readywhen the pipe is free (runsfnon the caller) or whenwriter_owner == current task(reentrant, under the held grant), and suspends only when it would actually have to wait. -
[x]DONE (2026-08, 6.4 stage 1). Coroutine-frame / control-block fusion — one alloc for frame + block, not two.Task_promise<R>carriesTask_control_block coreas its FIRST member (so the block pointer doubles as the promise pointer) plus the result storage, and the block'sdestroythunk destroys the coroutine frame. A coroutine task therefore allocates exactly the frame; coroutines reduce allocations rather than adding them, as the item required. Remaining allocation work is 4.1 (routingoperator newon the promise to a size-class pool). -
[ ](P3) Priority setter on the promise (it stores one; no config channel yet). -
[x]DONE (2026-08, branchpipe-rebase) — coroutine-first transformation. The shakeup: static graph + coroutines for everything dynamic;then/when_all/builder-after/retraction/reuse/inline-trampoline removed;Task_scopenursery + implicit per-frame scopes + coroutine graph nodes + awaitable access verb added; every illegal case a fatal with a companion how-to test. (ts::nested/Task_scope/join_nestedwere subsequently REMOVED, 2026-08 — an unsafe concurrent grant-inheriting child; see coroutine-first.md §4.3.) Design of record + staged plan: coroutine-first.md. Landed §7 stages 1–6; the remaining §11 action list is tracked as 6.9 below. Subsumed 6.1 (the awaitable access verb IS inline-when-free) and delivered 6.2 (frame/block fusion — the promise embeds the block, so a coroutine task is one allocation). Note the inline-dispatch trampoline was removed only from the DYNAMIC surface:Graph_node::set_inlineand thedispatch_ready/inline_pendingmachinery survive as graph-internal. -
[x]DONE (2026-08) — circular-wait detector (TS_SAFETY_CHECKS). The suspended-ABBA deadlock (a task holding G1 suspends awaiting G2's turn while a G2-holder awaits G1) parks no thread — both frames suspended, all workers free, the run silently never completes; graph-invisible by definition (the accesses are undeclared). At suspension-on-a-pipe record edge {holder's grants -> awaited pipe} (the harness knows both), clear at resume, cycle-check on insert, fatal naming both tasks + both objects. Gates blessing waiting-rule case (c) (coroutine-first.md §2) in the guide. Scope ruling (2026-08, field survey): keep it on GRANT edges; do not generalize it to arbitraryTask/Signalawait edges. That generalization is what Linux has failed to merge twice in eight years — lockdep's cross-release (coveringwait_for_completion) landed in 4.14 and was reverted in 4.15 for false positives under a zero-false-positive policy, and its successor DEPT, which models exactly this class of event/completion edges, is unmerged after 4+ years because "the tool generates so many reports that it is difficult to get a real signal out of the noise" (LWN 1036222). Our grant edges are structurally lock-like with known holders, so the narrow form behaves; the general case is covered by 6.13's quiescence net instead, which is O(1) and cannot false-positive on an ordering it merely finds surprising. -
[ ](P2, scoped 2026-08 — needs the platform layer, 3.6) Signal-from-OS-completion helper — register an OVERLAPPED / fd / fence, get aSignal(the 9.2 packaging question, pulled by the first-class cross-frame pattern, coroutine-first.md §4.7). Design note (2026-08). The bridge itself is already trivial and needs nothing new: aSignalis a refcounted handle, so an OS callback captures one by value and callstrigger()— that is the whole "get aSignal" story, and 6.7'sFrame_gatenow covers the realignment half. What is NOT trivial, and what any packaged helper must answer, is who owns the waiting. Three shapes, in increasing commitment: (a) Nothing (today): the user registers the completion with the OS themselves and callstrigger()from whatever callback context the OS gives them. Zero API, zero portability surface; the user carries the caveat below. (b) A thin per-platform adapter —Signal signal_from(HANDLE overlapped_event)/signal_from(int fd, events)/ a GPU-fence variant. Each needs a THREAD to do the waiting (aWaitForMultipleObjectspool, an epoll loop), i.e. exactly the reactor the library has deliberately not built (9.2), and it multiplies with every platform. Not worth it until a real workload asks. (c) A reactor. Explicitly rejected in 9.2 (Rayon/Tokio are separate pools by design). The caveat that matters regardless of shape, and the reason this is not purely a packaging question:trigger()releases awaiting frames on the TRIGGERING thread via the resume trampoline. An OS completion callback (an APC, an IOCP worker, a driver callback) is the worst possible place to run arbitrary user coroutines — it may be a restricted context, and it is certainly not a place to run unbounded work. So an OS-triggeredSignalshould hop:ts::launch([s]() mutable { s.trigger(); })from the callback, which is one line and is exactly whatFrame_gate::open()does for the same reason. Action: document (a) + the hop as the sanctioned idiom in the guide; revisit (b) only alongside the platform layer (3.6), which is where the waiting threads would live. -
[x]DONE (2026-08) — per-frame gate.ts::Frame_gate(frame_gate.h; in the umbrella since 2026-08-22):co_await gate.next()parks a task until the frame loop's nextopen(). Shipped as a type rather than a documentedSignal::resetidiom because the hand-rolled version has two real hazards: a missed-wakeup window (a task reading the signal just before a boundary can attach to a gate about to be re-armed) andSignal::reset's precondition that every waiter is already released (fatal otherwise). The gate hands out the CURRENT frame's signal under a mutex and installs a fresh one atopen(), so a waiter always names a specific frame's gate and no re-arm race exists — trading one bare block per frame, which is noise at frame scale (Signal::resetstays available for the zero-alloc case).open()releases through the scheduler rather than inline, atPriority::lowby default (set_release_priorityoverrides): an inline trigger would run every parked frame on the frame loop's own thread beforeopen()returned, stalling frame start by an unbounded amount — the low-priority-resumption default this item asked for, made structural. 3 tests. -
[ ](P2, author 2026-08 — promoted to P1 on an aggregate, then DEMOTED by the decomposition that aggregate asked for) Symmetric task switching for coroutine resumption. Correction (2026-08): the P1 promotion below reasoned from "~95% of a chained stage is the wake/dispatch round trip, and symmetric transfer is the lever for it". The first half is true; the second does not follow. The measured split of the ~1930 ns default stage is frame 91 ns (5%) · same-thread queue hop 248 ns (13%) · first cross-thread hop 331 ns (17%) · idle-pool scale 770 ns (40%) · wake+park 490 ns (25%). Symmetric transfer can claim at most the same-thread queue hop — ~13%, and less in practice since the resume trampoline already skips the queue. Everything else is paid dispatching the stage outward, before any resume exists to transfer to. The real lever for the remaining ~1600 ns is locality, and the obvious shape (awaiting a not-yet-started task runs it inline on the awaiting thread) is exactly the retraction coroutine-first deleted on purpose — so it needs a design answer, not an optimization. Caveat on the measurement: the benchmark is a strictly serial chain, which maximally rewards locality and is therefore the most favourable possible case for that lever, not a representative one. Why P1: the resume round trip is now the largest measured cost in the coroutine core, established twice without looking for it. (a) Thecoro chndecomposition (N3): 88 ns/stage of actual coroutine machinery against 1809–2139 ns/stage for a chained wait — ~95% is the wake + dispatch + resume-on-another-thread round trip, not the frame. (b) The graph-free frame measurement (sample/game_frame.cpp, 2026-08): graph-free costs +56–131 µs/frame, matching ~50 suspend/resume round trips at ~1.8 µs; the +95 allocations/frame are under 2 µs of it. So the graph's performance advantage over hand-composed coroutines is resume locality, not allocation amortization — and this item is the lever, not the allocator. Not yet decomposed: how the ~1.8 µs splits between the wake syscall, the queue/eventcount hop, and the cache-cold resume. Measure that first (anIdle_policy::spinvsspin_then_blockcomparison isolates the wake syscall) — it decides how much of the gap symmetric transfer can actually claim. UE's runnable signature isFTask*(bool)— a body/segment can RETURN the next task, which the worker loop runs immediately on the same thread with zero queue/eventcount interaction ("Continuations were not themselves dequeued from any queue");CallAndMoverelocates the delegate storage in one step, so no second alloc for the returned continuation. This is the natural shape for a coroutineco_awaithanding back "resume me here next" — cleaner than routing the resume through our thread-local inline trampoline (dispatch_ready/inline_pending, task.h), which already gives same-thread + stack-safe execution but via a push+drain rather than a direct return-and-run. Scope the borrow to the coroutine path (pairs with 6.2's frame/control-block fusion — a fused frame that returns its own next segment); NOT a rework of general dispatch, which stays framework-driven so a queued resumption still gets a priority and the scheduler can interleave. Verified against UE source (FTask::ExecuteTask,FScheduler::ExecuteTaskloop). Prior art: task-systems-comparison.md §UE. Rescoped (2026-08, post-6.4): the coroutine path now has its own equivalent — a bounded thread-local resume trampoline (resume_pending/schedule_resume, coroutine_support.h) that resumes a released frame on the settling/granting thread with no queue hop, iteratively so the stack stays O(1).dispatch_ready/inline_pendingsurvive but are graph-internal (Graph_node::set_inline) and no longer carry resumption. So the borrow narrows to the worker-loop half — a runnable that RETURNS its next runnable, which would let a resume skip even the trampoline's push+drain. Measure the trampoline first (10.14) -- done, and it promotes this item:coro chnis ~95% wake/dispatch latency (2138/1959/1809 ns per stage, of which only ~88 ns is coroutine machinery), so a resume that skips the queue entirely is where the remaining cost is.Decomposed (2026-08, and it RESCOPES this item again). The measurement this item asked for first is done: the same
chain_corobenchmark run on differently-configured global schedulers (bench_coro_chain_on+Scheduler_scope, benchmarks.cpp — five of the configurations are kept as a permanent--benchsection). Idle policy isolates the wake syscall, worker count isolates the cross-thread hop. Release, 22 hw threads, quiet machine, 3 full runs; ns per awaited stage, median of 4 measured rounds each:configuration run 1 run 2 run 3 22 w, spin_then_block(thecoro chnline)1932 1996 1814 22 w, spin1414 1439 1458 4 w, spin(not kept)810 819 800 2 w, spin681 668 659 2 w, spin_then_block(not kept)906 892 902 1 w, spin_then_block360 366 366 1 w, spin335 337 344 coro nst(frame only, no scheduler)91 91 95 Split of the ~1930 ns default-configuration stage:
component isolation ns share coroutine frame (alloc, promise, settled await, destroy) coro nst91 5% queue round trip on ONE thread (submit, own-deque pop, dispatch, claim, settle, resume trampoline) 1 w spin − coro nst248 13% first cross-thread hop (one possible thief: steal + cache-cold block/frame/result) 2 w spin − 1 w spin 331 17% idle-pool scale (steal contention on the deques + a more distant core) 22 w spin − 2 w spin 770 40% wake + park 22 w s+blk − 22 w spin 490 25% The wake-syscall component the item predicted is real but is a quarter of the cost, and it shrinks with the pool: the same delta is 230 ns at 2 workers and 25 ns at 1 (nothing parks when the single worker always has work). The dominant term (57%) is that the awaited stage leaves the awaiting thread at all, and it gets worse the more idle workers there are to take it.
Consequence for this item: symmetric transfer can claim at most the 248 ns same-thread queue round trip, ~13% — and less than that in practice, because the resume already skips the queue via the resume trampoline; the worker-loop borrow only removes that trampoline's push+drain. It can claim NONE of the other 87%, because wake and migration are paid dispatching the awaited stage OUTWARD, before any resume exists to transfer to. The lever for the 1600 ns is locality, not transfer: keep a task its submitter is about to await on the submitter's thread. The obvious shape — an await of a not-yet-STARTED task runs it inline on the awaiting thread — is exactly the retraction that coroutine-first deleted (deliberately: it was unsound for pipe/
asyncwork and made the blue/red boundary mushy), so reviving it needs a design pass, not a patch; a weaker, purely scheduler-side variant is to make the just-submitted-and-about-to-be-awaited task unstealable for a short window. Note the benchmark is a strictly serial chain, which maximally rewards locality — a policy that keeps work local must not do so when the launcher does NOT immediately await, or it converts parallelism into latency. Downgrade candidate: on this evidence the symmetric-transfer borrow is a P2 (13% of a cost the trampoline mostly already collects), and the locality question is the P1 that inherits its "why". -
[~](P1, author 2026-08) Coroutine-first post-initial action list. The queue behind the landed transformation, from coroutine-first.md §11. In order: (a)[x]nested graph runs v1 — DONE (2026-08). The lend protocol landed as designed:bind_links_for_runintersects the compiled access set with the caller'sAccess_contextat everyexecute()and re-binds thecompile()-time link slab to the objects the run must actually take turns on — an unbound link IS a skipped turn, so no bypass flag threads through the pipe and admission keeps one code path. Inner conflict edges are untouched (they order inner nodes among themselves on a lent object) and the pipe never sees the lend (externalasyncs queue behind the caller's hold as before). Auto-scope-join default +Execution_options{.detach}(a detached run gets no lend -- it is not contained in the caller's grant window); all three fatals with companions. Two refinements vs the design text: the non-quiet-scope check filters SETTLED scope children (the scope list only drops entries at a join, so plain emptiness would fatal on a fire-and-settle child), and the in-flight fatal isTS_SAFETY_CHECKS-gated like its siblings. 10 tests + 3 death scenarios +stress_graph_nested_runsunder TSan; (b)[x]worker-less nested runs — DONE, no work needed (the serial trampoline already handles it; test added); (e)[x]concurrent shared-object graphs — DONE (2026-08), and it holds. The old task-internals §10 scenario-2 line conflated two cases: the "not supported" cause was the per-graph single-runRun_state, which says nothing about two DIFFERENT graphs. Canonical pipe-address ordering over the same address-sorted objects makes a cross-graph wait cycle impossible, and the pipe serializes two graphs' conflicting nodes exactly as it does a node against an async. Validated with graphs declaring the same objects in OPPOSITE order, run concurrently under an async hammer,Rw_probeoracle: suite test +stress_concurrent_graphsunder TSan. §10 scenario 2 relaxed to "safe, nondeterministic cross-graph ordering"; the same-GRAPH case is now a fatal instead of silentRun_statecorruption; (c) cancellation composition (pass the outer token intoinner.execute({.token}), test that outer cancellation drains the inner run); (d) trace attribution across nesting (inner work must not double-count in the outer trace's fold); (e) concurrent shared-object graphs validation (task-internals §10 scenario 2 predates the evolved pipe — with per-node admission + globally canonical acquisition it should now be deadlock-free; dedicated stress with theRw_probeoracle + TSan, and if it holds, relax the task-internals contract line to "safe, nondeterministic cross-graph ordering"); (f) parameter-grants sugar for shipped library sub-graphs (compile-time intent check only). Open discussion queue: coroutine-first.md §10 (waiting-rule relaxations, HALO reality on MSVC/clang-cl, the graph-free usage model). -
[x](P1, author 2026-08) Check the rule, not the incident — structural in-tasksync()fatal — DONE (2026-08).sync_wait's check dropped its!blk->readygate:sync()/take()inside a task is fatal whether or not the target has settled, so the first execution of a bad path fails deterministically.blocking_sync_diagnoseis now[[noreturn]], gated onTS_RULE_ON(TS_RULE_IN_TASK_SYNC)rather thanTS_SAFETY_CHECKS(only its sharp same-object arm needs the harness), and both messages name the participating task and object byts::Namedplus the escapes:co_await,try_take()(6.12), andts::Relaxed_scope{ts::Rule::in_task_sync}. Rule class advisory; shipping default compiled out. Two in-tree call sites were relying on the old timing gate and are the interesting evidence that the item was right:test_access_reentrant_under_own_grantdidr.sync()on a provably-settled task (nowtry_take()), and the worker-lesstest_single_threaded_sync_inside_bodygenuinely relies on the serial trampoline's drain hook — a claim the library cannot verify, so it now carries aRelaxed_scopeand doubles as the opt-out's companion test. New death scenariosync_settled_in_taskcovers exactly the shape that used to pass. Original text follows. The diagnostic insync_waitis gated on!blk->ready, so it fires only when the wait would genuinely park. That inverts its coverage: an in-tasksync()whose target is usually already settled never trips in dev (checked builds skew further toward "already done" via their own overhead), then parks a worker on the one frame a prerequisite runs long — and in shipping the check is compiled out entirely. A safety check whose trigger condition is the hazard's timing inherits the hazard's nondeterminism. Fix: make it unconditional —sync()inside a task is illegal whether or not the target has settled — which is what the waiting rules already say (coroutine-first §8 flag 4,sync()demoted to boundary-only). Deterministic: the first execution of the path fails, regardless of timing. Depends on 6.12 — the legitimateif (t.is_done()) v = t.sync();idiom needs a non-blocking spelling before the unconditional fatal removes it.parallel_forjoins stay structurally exempt. -
[x](P1, author 2026-08) Same fix for the guard-across-suspension fatal — DONE (2026-08). Theaccess_guard_depth > 0check moved fromawait_suspendtoawait_readyin all three awaiters (Task_awaiter,Access_awaiter,Join_awaiter), so an await under a live guard is illegal whether or not it suspends. ForJoin_awaiterthe check runs BEFORE the settled children are erased -- whether the scope happens to be drained at that instant is precisely the timing the rule must not depend on. Each of the three carries its own message naming the sanctioned form (functorco_await obj.access(fn); split the scope; multi-objectco_await ts::access(fn, a, b)instead of nested guards) and stating that the rule is structural -- no runtime opt-out, onlyTS_ENABLED_RULES(rule class structural; shipping default kept, as before the policy existed). The reentrancy exemption is now stated rather than emergent, which was the item's real content:detail::reentrant_under_held_grantreturns true when every pipe of the awaited block is write-owned by the current task -- i.e.Guarded::accesstook its reentrant arm and ran the body inline under the held grant, so the task is settled before theco_awaitis evaluated and cannot suspend by construction. It keys offPipe::writer_owner, which is always-on state, so the exemption works in every build. Deliberately narrow: a READ access under a READ guard is NOT exempt (it reaches the pipe, and a queued writer -- blocked by our own read hold -- makes it suspend). New death scenarioawait_settled_under_guardcovers the shape §2.3(f) of static-order-checking-and-ww-mutex.md identified as passing undiagnosed; companionsco await under guard, splitandco reentrant access under guard. Original text follows. Theaccess_guard_depth > 0check lives inawait_suspend, so aco_awaitthat happens to complete synchronously is never examined — the same timing-luck coverage as 6.10, one degree less bad (the check is notTS_SAFETY_CHECKS-gated, so the latent case aborts in shipping rather than silently serializing). Fix: hoist the test toco_awaitentry (await_ready) so any await under a held guard is illegal whether or not it suspends; cost is one branch on a TLS counter on a path that already branches. Not purely mechanical: today the reentrancy exemption is emergent (a reentrant same-object access never suspends, so it never reaches the check). Hoisting forces it to be stated — an awaiter for an object already in the grant set skips the check, everything else does not — which makes the rule legible rather than implicit. Land with 6.10; same principle, same test-matrix rows. Confirmed load-bearing (2026-08), not cosmetic: the ww_mutex evaluation found that because the fatal sits inawait_suspend, a hold-then-await runs fine whenever the target pipe happens to be free at that moment — so the illegal pattern executes undiagnosed on exactly the runs where timing is friendly, which is the A.1 inversion in its purest form. It also found that both guard-form ABBA shapes are already closed by these fatals; what survives is node/pipe-job declared grants plus aco_await, which is 6.14's territory. That narrows the residual hole and raises this item's value. -
[x](P1, author 2026-08) Non-blocking result accessors — DONE (2026-08).Task<R>::try_take()returnsstd::optional<R>, never blocks (empty when unsettled OR cancelled) and is therefore legal inside a task -- the escape valve 6.10 needs;co_await t.as_optional()performs the same wait asco_await tbut yields an empty optional instead of fatalling on a cancelled task (adetail::Optional_awaitablemarker from task.h, made awaitable by anoperator co_awaitints::detail-- ADL looks where the argument type lives -- resolving to anOptional_awaiterthat reusesTask_awaiter's handshake and only differs inawait_resume).sync()/take()are unchanged.ts::Maybe<R>dropped by the author: both verbs simply do not exist forvoid, since a void task has no result to be missing (is_done()answers the first question, and awaiting a cancelled void task already resumes normally). Both MOVE the result, so they jointake()under the existing "at most one mover, and it must be last" rule rather than adding a third kind of read. Original text follows. Fatal-on-cancelled is the wrong default for a valuesync(): the caller cannot check-then-take without a race, and the fatal punishes them for a state the callee chose. Do NOT changesync()'s return type — every call site that cannot be cancelled (no token in play, the common case) would pay an unwrap forever. Three spellings instead:sync()unchanged (blocking, boundary-only, fatal on cancelled — the "this cannot be cancelled" assertion);try_take()— non-blocking, empty when unsettled or cancelled, and legal inside a task (it never parks), which is also the escape valve 6.10 needs;co_await t.as_optional()— the red-side equivalent, replacing the fatal-on-awaiting-cancelled with a branch. Void asymmetry: definets::Maybe<R>asoptional<R>for non-void andoptional<std::monostate>for void — presence-testing and dereference stay uniform, so generic code needs noif constexpr. (Thebool-for-void variant buys only the declaration:*mis invalid forbool, so every generic consumer that extracts still branches — barely worth a name.) -
[x](P1, author 2026-08) Global quiescence deadlock detector — DONE (2026-08). Implemented in the three-part decomposition the author asked for, so the task layer stays uncoupled from the scheduler: (a) quiescence is scheduler-local —Scheduler::quiescent()= every worker in the idle path AND every queue empty, exposed through the plain function seamdetail::scheduler_quiescent()(defined in guarded.cpp, where the global holder is; same shape asdrain_serial_pending/blocking_sync_diagnose). Idleness is an explicit busy<->idle transition count inWorker_thread::main, not "parked": underIdle_policy::spina worker never parks, so a park-based test would never report idle. Two atomics per crossing, none per task. The seam returns false when no scheduler exists yet -- a blue wait before the pool is created must not conjure one as a side effect of a safety check. (b) the task layer interprets it from the boundary waiter — no global per-task counter, nothing on the hot path:Task_control_block::wait()already blocks, so it polls while it is there. Quiescence must hold CONTINUOUSLY for the window (default 2 s over 8 samples,ts::set_deadlock_net_window; 0 disables): one sample is worthless because a worker sits briefly between finding work and marking itself busy, and a long window is free because a real deadlock is permanent while a short one would fire on a slow-but-legitimate blue-to-blue handoff. (c)ts::External_wait(RAII, task.h) is the outstanding-external-wakeup counter, andFrame_gateholds one from the firstnext()of a frame until the matchingopen(). The fatal names it, per the item's own warning that a forgotten registration reports a correct program as deadlocked; it also namests::set_deadlock_net_window(0ms)andTS_ENABLED_RULES. Rule class net (no scoped opt-out -- it observes the whole process, so there is no call site to attribute a relaxation to); shipping default compiled out. Death scenariodeadlock_netplus two companions (External_waitsuppresses it; the window switch disables it). The report has three tiers (author follow-up, landed 2026-08). Rule: say everything cheap to collect, then name the flag that gets the rest. Tier 1, always -- the blocked waiter's identity, the quiescent window, the escapes. Tier 2, free wherevercircular_waitexists -- the livewait_edgesregistry is printed verbatim; its entries ARE the tasks suspended while holding a grant. Sound because it is a POST-MORTEM: an independent mechanism already concluded the system is wedged, so these edges explain a verdict rather than predicting one. (The Linux cross-release / DEPT precedent argues against learned edges used to PREDICT; it does not apply to printing them afterwards.) Tier 3,TS_SUSPENSION_REGISTRY-- every live suspension: the task, what it awaits (task / object / its own scope children) and what it holds. Covers the shape tier 2 cannot: a task suspended on a plain task await while holding nothing, i.e. the two-hop cycle. Records are embedded in the awaiter (which lives in the coroutine frame), so nothing is allocated; the fatal names the rebuild flag when the tier is absent. Default justified by measurement, not assumption (the assumption was wrong). Release x64, 22 hw threads, three runs each, registry ON vs OFF:fixture OFF ON delta 1 w spin(1 worker, serial)~357 ns ~385 ns +28 ns (+8%) coro chn(serial chain, 100% suspension)~1866 ns ~2025 ns +160 ns (+8.5%) 2 w spin~816 ns ~975 ns +160 ns (+19%) N w spin(22 workers)~1553 ns ~1647 ns +94 ns (+6%) graph 1.0/graph .05(real frame)4112 / 455 µs 4107 / 444 µs none (noise) free .05(graph-free, ~50 suspensions / 0.5 ms frame)~545 µs ~542 µs none (noise) So: ~30 ns/suspension uncontended (matching the estimate), no scaling cliff -- the per-suspension cost FALLS from 2 workers to 22, because records shard by address and concurrent suspensions land on different shards -- and **zero measurable cost on real workloads**. The ~8% appears only where one frame ping-pongs a single suspension between two threads and does nothing else, which is the structural worst case for any registry. The implementation defaulted it on wherever TS_SAFETY_CHECKSis on, arguing thatthis project's Release is itself a checked build. **Author ruling (2026-08): too expensive for release — default is DEBUG ONLY** ( TS_SAFETY_CHECKS && !NDEBUG). The ~8%is on a microbenchmark, but a per-suspension linked-list insert/remove is not something a release build should carry for a diagnostic that only pays off after a deadlock has already happened, and the escalation path is the whole design: tiers 1 and 2 are free and always present, and the report tells the user to rebuild with -DTS_SUSPENSION_REGISTRY=1and reproduce when it needs more.Sharding key is load-bearing: a coroutine resumes wherever its awaited work settled, so a record is linked by one thread and unlinked by another. Keying shards on the SUSPENDING THREAD put both halves of every suspension on one mutex and one cache line -- measured at +183 ns on 2 w spinand +157 ns onN w spin; keying on the record'sADDRESS removed the multi-worker component entirely. TS_DEBUG_NAMESlanded with it (the taskNamedmoved offTS_SAFETY_CHECKSonto itsown switch), and TS_SUSPENSION_REGISTRY=1forces it on -- an explicitTS_DEBUG_NAMES=0alongside is a hard#error. Both defines joinTS_ENABLED_RULESinthe layout-mismatch tripwire; verified firing (`lld-link: /failifmismatch: mismatch detected for 'TS_SUSPENSION_REGISTRY'`) with a deliberately mis-configured TU. Original text follows. The circular-wait detector (6.5) fires before a deadlock and names the participants, but only for the shapes it models: an edge is recorded solely when a task suspends on a pipe job while holding grants. Cycles mediated by a task-await edge are invisible — N holds G1 and awaits foreign task T (not a pipe job, no edge), while T awaits G1 (holds nothing, no edge); both frames suspend forever and nothing fires. The sound complement is Go's all goroutines are asleepcheck: **every worker idle + un-settled work outstanding +nothing external able to wake us ⇒ progress is impossible ⇒ fatal**, dumping every suspended frame. O(1), no waits-for graph, and it misses no deadlock shape — including lost-wakeup bugs that are not cycles at all. Two requirements: (a) an outstanding-external-wakeup counter — anything completable by a non-worker thread (I/O, a Signaltriggered off-pool,Frame_gate) registers while pending, soquiescence with a zero count is a genuine deadlock and not a legitimate wait; (b) idle must be an explicit idle-worker count + empty queues + zero in-flight, since spinworkers never park. Relationship: quiescence is the safety net (always-on candidate), 6.5's graph stays the precise dev-mode diagnostic. Ranks first of 6.10–6.14 — it also retires the blind spot above. Field evidence (survey, 2026-08): Go's equivalent check has a documented blind spot — it fires only when every goroutine is asleep, so any live background thread masks a partial deadlock (golang/go#13759). Our version is better positioned only because of the outstanding-external-wakeup counter, which supplies the "nothing can wake us" predicate Go lacks. The counter is load-bearing, not an optimization — and its failure mode (a forgotten registration ⇒ a FALSE deadlock report) means the fatal message must name its own escape hatch. -
[x](P1, author 2026-08 — the one gap the field survey identified) Declared rank for dynamically-awaited objects — DONE (2026-08).ts::Rank{n}(access.h) as an optional second leading argument onGuarded/Versionedafterts::Named; stored on the pipe and carried into everyAccess_contextentry alongside the grant epoch, so the held side is read from the context that already exists. Checked atco_awaitENTRY (await_ready, next to 6.11's guard check) in both awaiters, so it fires deterministically on the first offending await rather than needing two halves of a cycle to interleave. As the item required, the rank is NOT defaulted (to address order or anything else) and unranked is the strict state: holding an unranked object forbids dynamic awaits entirely. Two rejected shapes, two death scenarios (access_rank_descends,access_rank_unranked); two companions (climb; per-scope opt-out). Rule class advisory --ts::Relaxed_scope{ts::Rule::access_rank}is the escape; shipping default compiled out, and likecircular_waitit is masked off entirely withoutTS_SAFETY_CHECKS, because its held-side input IS the access context. Two things fell out of implementing it, both kept: (1) Stale grants must be skipped. A detached coroutine carries its launcher's grant snapshot for its whole life; without the staleness filter (the same epoch testcheckuses) it would be treated as a holder forever and every later await rejected. This showed up immediately ingame_frame's fire-and-forget streaming coroutine. (2) The rule makes a genuine ABBA unrepresentable, which means thecircular_waitdeath scenario can no longer be constructed without opting out of the rank rule --abba_bodynow carries aRelaxed_scope. That is the honest relationship between the two: rank prevents, the detector is what remains for programs that opted out. Adoption in-tree: threegame_frameobjects (input/assetslow,asset_sourcehigh) and the coroutine-graph-node test's trio. Everything else never awaits outside its declared set and needed no rank -- which is the intended shape. Original text follows. The canonical pipe-address order covers BATCH acquisition (multi_acquire, node declared sets) and makes it deadlock-free. Nothing relates a grant a task already HOLDS to an object it awaits LATER — that single missing ordering constraint is the entire suspended-ABBA hole. The field's cheap answer is a lock rank:Guardedtakes an optionalts::Rank{n}(alongsidets::Named), and every dynamic await assertsrank(target) > max(rank of everything currently held). Havender's argument then makes the cycle unrepresentable. Why this shape over the alternatives: O(1) (one TLS max against one field;Access_contextalready tracks held entries — Go'slockrankis a load and a branch, and is off by default only for a struct-size reason, not a speed one), and it fires deterministically on the first offending await, where a circular-wait detector needs both halves to actually interleave — a scheduling coin-flip. Driver Verifier makes the same choice explicitly: it bugchecks on "the hierarchy violation… not when an actual deadlock situation is occurring". Composes with what exists: batch acquisition keeps address order; only the sequential held-then-await edge needs ranks, so only objects that are actually awaited need one. Design detail that is load-bearing: do NOT default the rank to address order — rejection would become ABI-dependent and non-reproducible across builds. Require an explicit rank for any object that may be dynamically awaited; default = "no dynamic await permitted while held", the strictest and most common case. Honest cost, the standard one for this family: a strict order rejects some correct programs; the escape is to restructure (split the node — the §10.4 preferred form) or opt out per scope. Prior art: Goruntime/lockrank.go, Williams'hierarchical_mutex, LinuxCONFIG_PROVE_RAW_LOCK_NESTING, Boyapati et al. OOPSLA'02 (the type-level version). Free static complement — EVALUATED (2026-08), adopt narrowly. Full report with compiler output: static-order-checking-and-ww-mutex.md. clang-cl 22.1.3 ships in VS 18 Community, soACQUIRED_BEFOREis default-on under plain-Wthread-safety, and the flag is silent over the current headers — enabling it costs nothing today. Two claims in the earlier draft of this item were wrong and are corrected here: (1) it does NOT cycle-check the declarations themselves — there is no declaration-level check at all (verified under-Weverything); the analysis is entirely use-driven, so an unused inverted rank pair is never diagnosed. (2) "named globals only" is false — the attribute fires on struct members through aWorld¶meter and through lambda captures, so all 38 ofgame_frame's objects are coverable. Real limits, measured: function locals cannot carry the attribute at all (hard-Wignored-attributesrejection) — 268 of the repo's ~410Guardedobjects, giving ~9% repo coverage against ~75% of the samples. And the held side is unautomatable: a node body must hand-writeREQUIRES(world.phys, world.nav)duplicating whatcompile()already derives (42 sites ingame_frame), with silent drift when it diverges. One defect worth knowing:ACQUIRED_AFTERis silently order-dependent — identical code warns or not depending on source position relative to an unrelated acquisition. Verdict: shipTS_CAPABILITY/TS_ACQUIRED_BEFOREmacros (no-ops on MSVC, verified zero-noise) plus a guide recipe, as a free static mirror of the ranks this item makes users write anyway. Strictly a subset of 6.14 at higher annotation cost — 6.14 first. -
[x](P1, author 2026-08 — big, design first) Escape hatches for the waiting rules, with a declared shipping policy — DONE (2026-08). Design of record: waiting-rule-policy.md; mechanism ininclude/ts/rules.h. Landed: a flatts::Ruleenum mirroringTS_RULE_*preprocessor bits (so a disabled rule's code AND state vanish), three classes over it,ts::Relaxed_scope+ts::set_default_relaxed_rules, andTS_ENABLED_RULESwith the samedetect_mismatchODR tripwire asTS_SAFETY_CHECKS. Defaults reproduce today's behavior exactly (everything checked;await_under_guardonly in shipping). The two existing checks were rewired onto it:await_under_guardandcircular_wait. The design question the item left open answered itself, against the assumption in (a): the runtime opt-out cannot be uniform. Relaxingawait_under_guardis not merely permissive but unsound — aAccess_guardinstalls its ownAccess_contextas the thread's current grant, so a frame suspended under one resumes with the promise's snapshot installed over it and then restores acurrent_accesspointer captured on a different thread. That splits the rules into advisory (scoped opt-out + compile-out), structural (compile-out only; escape is the sanctioned form) and net (6.13; no call site to scope, escape ists::External_wait). Inheritance resolved as "yes, but not viaAccess_context": the relaxation is its own thread-local, carried across coroutine segments by the promise (Relaxed_carrier) —Access_contextis snapshotted at frame CREATION, so a mask living in it would be silently dropped by anyRelaxed_scopeopened mid-body. (Thewith_inherited_accesspath that once carried it into gated bare-task children went away withts::nested;parallel_forhelpers inherit the grant but not the relaxation.) Obligations on a new check are the doc's §8 checklist. Original text follows. Raised by the observation that a user may uphold a rule by means we cannot see — an external lock discipline, a phase invariant, a platform guarantee — in which case our check is a false positive and there must be a way out. Two axes, both needed: (a) Opt-out granularity — a scoped RAII form (ts::Relaxed_scope{ts::Rule::foreign_await}or similar: documents the claim at the site, keeps the rest of the program checked) and a global default for teams that want the rules as advice. Scoped is the primary form; global is the blunt instrument. (b) Compile-out policy — which checks a shipping build keeps. This is the reason every rule check must be capable of being compiled out even when it is currently always-on (the guard-across-suspension fatal is always-on today with no way to disable it, which is exactly the case that motivated this item). The user should be able to declare the set: e.g. keep the cheap structural checks, drop the expensive diagnostics. Design questions to settle: the rule taxonomy (what is individually switchable — a flat enum of rules, or classes like structural/diagnostic/net?); interaction with 6.10–6.14 (each new check must state its escape and its shipping default when it lands, not retrofitted); whether an opt-out is inherited by nested/child tasks (probably yes, viaAccess_context, but that makes it wider than the lexical scope suggests); and how an opt-out interacts with the quiescence net (6.13), which cannot be locally suppressed because it fires globally. Prior art: TSan suppressions,[[clang::no_sanitize]], lockdep's per-class annotations, Orleans' scopedAllowCallChainReentrancy(). -
[ ](P2, author 2026-08)when_allas an AWAITER (not a Task combinator). The deletedwhen_allwas aTask-level join (its own block, a result tuple, ~6 allocs — the worst per-op offender in the old allocation audit). The awaiter form is a different and much cheaper animal: a countdown plus one resume, no join block, no tuple. It is also a popular and expected feature in coroutine libraries independent of any static graph, so its absence is a gap in the graph-free story (guide §6.4). Concretely wanted for the cross-frame pattern, whereco_await io_done; co_await gate.next();is two suspensions and two potential worker wakes for what should be one wait — see the narrowergate.next_after(io_done)idea in 6.9(g). Design: variadic awaitables, resume once when all have fired, results by reference from the individual handles rather than moved into a tuple (avoids the old allocation profile entirely). -
[ ](P2, author 2026-08 — reopened by the 6.8 decomposition) Retraction, or another locality lever. The resume decomposition (6.8) shows ~57% of a chained coroutine stage is the work leaving the awaiting thread — the awaited task is dispatched outward to whatever worker steals it, and the resume is dispatched again afterwards. That cost grows with the number of idle workers (1930 ns at 22 workers, 681 ns at 2, 335 ns at 1). The classic lever is exactly what coroutine-first deleted on purpose: awaiting a not-yet-started task runs it inline on the awaiting thread (retraction). It was deleted because as a blocking mechanism it could not be made safe under access control (see retraction-vs-pool-exhaustion.md and coroutine-first §3) — but the suspension-based form is a different proposition: a suspended awaiter holds no thread, so the pool-exhaustion argument that killed it does not apply. What must be re-derived from scratch: whether running the awaited task inline is safe with respect to the awaiter's held grants (it inherits the context, so this is waiting-rule case (a) — plausibly yes), whether the claim protocol can be reintroduced without the claim/generation machinery that was removed, and what it does to determinism. Measure before building: the 6.8 benchmark is a strictly serial chain, the case that maximally rewards locality; a realistic fan-out workload may show much less. Do not reintroduce blocking retraction. -
[ ](P3, evaluated 2026-08)TS_CAPABILITY/TS_ACQUIRED_BEFOREmacros — a free static mirror of 6.14's ranks. Report + compiler output: static-order-checking-and-ww-mutex.md. Ship thin macros wrapping Clang's thread-safety attributes (no-ops on MSVC, verified zero-noise; clang-cl 22.1.3 ships in VS 18, soACQUIRED_BEFOREis default-on under plain-Wthread-safetyand is silent over the current headers) plus a guide recipe for the one shape that works: node body annotatedREQUIRES(<declared set>)+ aSCOPED_CAPABILITYorder token at the dynamic await, which produces the exact ABBA diagnostic with zero false positives in free-function, lambda and coroutine forms. P3 because it is strictly a subset of 6.14 at higher cost: function locals cannot carry the attribute at all (~9% repo coverage, ~75% of samples), the held side must be hand-written per body (42 sites ingame_frame) and can drift silently from whatcompile()derives, there is no declaration-level cycle check (use-driven only), andACQUIRED_AFTERis silently source-order-dependent. Do 6.14 first; revisit this as an opt-in extra for users who want a compile-time signal on their own named state. -
[x](P2, author 2026-08) Debug name on every task — DONE (2026-08), as author-specified. Onets::Named(include/ts/named.h) for nodes, objects and tasks;Node_namedeleted. It stores{literal, file, line}— not a wholesource_location, sincefunction_name()is never used — displayed as the literal elsebasename(file):line. Required for nodes and objects (their unnamed constructors and thenode<N>/objNfallbacks are gone;ts::Named{}is the "identify me by my call site" spelling); optional for tasks, where the verb captures the site by default. The three design points settled: (a) cost — node and object names stay in shipping (they feed the DOT dump and the trace, both shipping-capable, over a handful of entities); the task'sNamedisTS_SAFETY_CHECKS-gated, field included, so the 264 B block is unchanged in shipping. (b) the coroutine case — a promise sees the coroutine's arguments, not its call site, so a frame cannot capture one; it INHERITS the identity of the task it was created inside, which for a coroutine node body is the node — exactly the participant a diagnostic should name. (c) plumbing — a defaultedsource_locationcaptures the CALLER of the function declaring it, so it is declared only on the outermost verb the user calls (launch/nested/Guarded::access/async/Deferred::commit) and the resultingNamedis passed down every internal layer explicitly;tests/named_tests.cppasserts captured sites land in the test file rather than a library header. The pack-ending multi-objectts::access/ts::asynccannot take a trailing default, so they carry only an explicit{.name = "..."}— documented, not worked around. Payoff landed with it: the circular-wait fatal now readstask 'nodeA' holding 'objA' awaits 'objB', while task 'nodeB' holding 'objB' awaits 'objA'. Also resolves Inconsistency 7. Original text follows. Graph nodes are named (Node_name, implicit from a string literal,{}captures theadd_nodecall site viastd::source_location) and objects are named (ts::Named), but dynamic tasks are anonymous. Since coroutine-first made coroutines the whole dynamic vocabulary, and since their ergonomics are a stated goal, every task should carry a name the same way: default = thestd::source_locationof the creation site, with an optional user-supplied literal overriding it — the exactNode_namepattern, reused rather than reinvented. Why it pays off now: the diagnostics landing in 6.10–6.14 all print tasks. The waits-for fatal currently readstask (block 0000018EB62717B0) holding 'objB' awaits 'objA'— the objects are named becausets::Namedexists, the tasks are raw pointers because nothing equivalent does. 6.13's quiescence net is meant to dump every suspended frame when it fires, which is only useful if the frames identify themselves. Trace attribution (trace_owner) has the same gap for non-node work. Design points to settle: (a) cost — aconst char*plus asource_locationis ~3 pointers + 2 ints per block, against a control block deliberately shrunk to 264 B (4.7 is still trying to shrink it further), so decide between always-on (likeNode_name, kept in shipping) andTS_SAFETY_CHECKS-gated, and consider storing thesource_locationalone and formatting lazily; (b) the coroutine case is the interesting one — a coroutine's useful name is its function, whichsource_locationat the creation site gives only indirectly; check whatsource_location::current()as a defaulted parameter actually captures forco_awaited frames versuslaunch/asynccall sites, and whether the promise can capture it atget_return_objectinstead; (c) plumbing throughlaunch/async/nested/access verbs without adding a parameter to every signature (a leading name arg is not expressible after an object pack — the same constraint that forcedNode_nameto lead). Ratify alongside Inconsistency 7 (the naming surface review, which explicitly wants this settled before more tooling depends on it) — this item is exactly that dependency arriving. -
[ ](P2, BUG — sibling of 7.3, found 2026-08) Coroutine-frame captures outlive the task's settle. The 7.3 fix destroys a FUNCTOR task's body before it settles, but a COROUTINE frame is a separate lifetime: the frame (and anything stored in it — a by-value coroutine PARAMETER holding a resource, or a variable that lives across a suspension) is destroyed by the frame's owndestroythunk at refcount-zero, on a worker, after the task settled — the same "captured resource outlivessync()" class 7.3 fixed forExecutable. A by-valueRecorderparameter to a coroutine would race exactly asgf_propagationdid. Not currently hit (the samples don't pass resources by value into coroutine params), so lower urgency, but a real gap in the same guarantee. Harder than 7.3: a coroutine's locals die atco_return(beforefinal_suspend), so those are already fine — the exposure is parameters and cross-suspension state, which the frame owns until teardown. Options: destroy parameter/frame-state atfinal_suspend(before the completion the awaiter observes) rather than at frame destruction; or document that resources must not be passed by value into a coroutine that outlives the caller (weaker, and the un-checkable footgun 7.3 argued against).
-
-
Deferred / Versioned
[ ](P2) Main chain (deferred-versioned-state.md §6) — journalmem_profilebaseline → per-journal bump arena → record-stream slots → typed command tier (Deferred<T,Cmd>) → sort keys / hooks / dirty-set → render-queue fixture.[ ](P2) Lock-freestage()— kill the per-slot mutex (it exists ONLY for the dynamic stage-vs-cut race; single producer per slot otherwise — handoff doc §5). Falls out of 7.1's arena step: single-producer chunked bump allocation makesstagea lock-free bump, and the cut becomes a chain-head exchange.Parallel_recorderalready gives thread-keyed slots (per-worker + overflow lane); this removes the last lock on the staging path. Split out of 7.1 for referenceability — implement together with the arena.[x]DONE (2026-08) — a task's captured resources are now destroyed before it settles, not at refcount-zero. Was: a functor task'sExecutableheldBody body;as a plain member, so the closure (and anything it captured — e.g. aRecorder<T>into aDeferred/Versionedjournal) was destroyed only when the block's refcount hit zero, on a WORKER, after the task settled and async()returned. So a captured resource outlived thesync()meant to bound it — ingame_frame,gf_propagation'sRecorder<Transforms>released a journal slot on a worker while MAIN was constructing the next frame'sWorldat the same reused address (a real data race, ~3% ontsan/run.sh, root-caused during the graph-dispatch opt work). Fix:Executable::rundestroys the body in the TYPED context, right after the invoke (and in the cancel branch), beforecomplete()/settle(). The body moved to a flaglessunion { Body }so~Executableno longer auto-destroys it; aTS_SAFETY_CHECKS-only assertion in~Executablecatches a never-run block (the union leak). No fn-ptr —run()is typed, so it destroys directly.Piped_executable(multi-objectasync) is covered automatically (it embedsExecutable). The guarantee: captured resources are gone before the task settles, i.e. before anysync()/co_await(or downstream gated on settle) is woken — so a journal/Recorderdrain is unnecessary (contrast~Guarded's pipewait_until_idle, kept because pipe release is unbounded). This is the right seam: destroying a closure is bounded (no cascade), unlike pipe release, so it can run synchronously beforenotify_all. Guarded deterministically bytests/task_tests.cppcaptures destroyed before sync(holds the handle,sync()s, asserts the capture is gone — fails pre-fixdestroyed==0, passes post-fix) +game_frameunder TSan. Not covered (separate lifetime, see 6.20): a coroutine frame's captures.
-
Task chaining
[—]MOOT (2026-08, 6.4). Results-on-after—afterand the whole builder are deleted; a dynamic task that needs a prerequisite's result writesauto r = co_await t;. Nothing left to design. (Typed results INTO a graph node remain open as 2.1.)
-
Research / shelved
-
[S]std::execution senders — shelved; model the concepts for interop, prototype access-context-as-env as the one novel spike; do not re-found the engine on senders (the monomorphic runtime block earns its keep). [§D7] -
[ ](research note — T13.4) Async I/O story. Untouched so far. The library is CPU-compute-first; async I/O (file/socket/GPU-transfer completion) is a different axis — a blocked I/O wait must not tie up a worker. Today the sanctioned bridge isSignal: an external completion (OS overlapped-IO callback, GPU fence,io_uringCQE) callssignal.trigger(), and aco_awaiton thatSignalresumes CPU work — fire-the-IO, suspend, never block a worker. (2026-08: the coroutine-first cross-frame pattern, coroutine-first.md §4.7, makes this the first-class shape rather than an idiom, which is what pulled 6.6 out of this note.) That covers "react to completion" without an I/O runtime. What we deliberately do NOT provide (and probably shouldn't, cf. Rayon/Tokio being separate pools by design): an I/O reactor, readiness polling, or a socket/timer API. Open question to revisit only on demonstrated demand: whether a thinSignal-from-OS-completion helper (register an OVERLAPPED / fd / fence, get aSignal) is worth packaging, or stays a documented idiom. Ties to the Signal-examples doc item (10.4). -
[ ](research note — T25) CPU transient aliasing. Render graphs reuse one block of memory for two scratch resources whose lifetimes don't overlap (research-deepdive.md §4.4), derived from declared first-write/last-read. The CPU analogue for us would be per-node declared transient buffers withcompile()computing [first-writer, last-reader] windows and aliasing storage across non-overlapping windows — a potential differentiator, but it needs a task-system feature we don't have: a declared transient-resource concept distinct from aGuardedobject (a scratch buffer owned by the graph, not a persistent guarded instance). The coarse version already exists (the per-run bump arena, 4.6 — everything dies at run end); the fine version is speculative and gated on that new concept. Record only; act only if a concrete workload needs graph-derived scratch-memory reuse. -
[ ](P2, research, author 2026-07) Survey high-level parallelisation patterns to generalise. A research pass over the broad parallel-programming pattern catalog (structured/nested parallelism, dataflow + pipeline, map-reduce / scan, fork-join variants, actor / mailbox, work-stealing idioms, GPU-style dispatch, ECS scheduling, render-graph patterns) to FARM for patterns this library could generalise into first-class primitives or documented idioms, the wayVersioned/Deferredgeneralised double-buffering and command buffers. Output: a ranked candidate list with "already covered / cheap idiom / worth a primitive / out of scope" verdicts, feeding 5.1's primitive menu and the samples. Complements the existing engine research (task-systems-comparison.md, research-deepdive.md) which studied task SYSTEMS; this studies parallel PATTERNS. -
[ ](P1, research, author 2026-07) Find a real integration / validation target. A deep research pass for potential applications of the library, with the concrete goal of finding an OPEN-SOURCE project to integrate with and validate the access-declared-parallelism approach on a real workload (not a synthetic sample): ideally a game engine, renderer, simulation, or other frame-structured C++ codebase with exploitable coarse-grained parallelism and thread-unsafe shared state. Deliverable: a shortlist of candidates with fit assessment (workload shape, existing threading model, integration surface, license, community activity) and a recommended target to prototype against. The strongest available validation of the design and the best source of real API pressure, so prioritize it. Ties to the going-public story (tmp/docs/going-public.md - internal, not in the repo).
-
-
Tooling / infra
-
[ ](P2) Benchmark regression baseline + compare step (store medians, flag regressions). (postponed — not blocking) Also assert allocation-free re-runs here (T19):execute()on a compiled graph should show 0 allocs via--memprofileon the 2nd+ run — a cheap regression guard for the re-arm design, folded into this step rather than a standalone test. -
[ ](P2) Proper ASan build config (reachable via/p:EnableASAN=truetoday); portable TSan build story. -
[ ](P3, docs — T13.1/T13.2) Renderer-pattern samples. Single-file samples, not out-of-box machinery (the author's call — these are idioms over existing primitives, not new API): (a) parallel command-list recording — fork records into an indexed slot array (chunk i → slot i), a sequential tail submits slots in order (GPU exec order = submission order); showsparallel_for+ ordered-slot join. (b) dynamic reprioritization — the PSO-compile pattern: a queued background task that must jump the queue when its result becomes needed this frame; cancel-and-relaunch-at-higher-priority is the viable mechanism and the user can already do it — the sample documents the idiom; only add a helper if the raw form proves error-prone. (c) RHI-stream-as-pipe — the render/RHI split as aPipe/Deferredordered stream fed from many recording threads (incl. insideparallel_for), rather than a dedicated named thread — reinforcing 3.2's named-thread-free stance. -
[ ](P2, docs — T13.5)Signalusage examples.Signalis under-documented for how much it enables: external-completion bridge (GPU fence / OS async-IO callback →trigger()→then), manual join/barrier, phase gate, one-shot broadcast, reusable phase gate viareset(). A worked set of examples in guide.md (and the async-IO idiom from 9.2). Author flagged Signals as an interesting, under-covered primitive. -
[ ](P3, docs — T24) Same-program comparison appendix. A short "same task graph in ours / Taskflow / TBB flow graph" appendix — we win the wiring count by construction (derived edges = zeroprecede/make_edgecalls), the measurable form of the ergonomics claim (research-deepdive.md §14). Low priority; informative value uncertain until written. -
[ ](P1-cheap, docs) "Declaration order is program order" contract. One paragraph in guide.md making explicit the contract the graph already implements (conflicting nodes run inadd_nodeorder — the STF/pipe-FIFO philosophy applied to the graph). Removes the "invisible edge" surprise the derived-edge model risks; prerequisite framing for the ordering-ambiguity discussion (ordering-ambiguity.md §4.1). Do with the next docs pass. -
[ ](P2, docs) Layered-disclosure front page + scope statement. guide.md should let a reader be productive with the six core functions before meetingDeferred/Versioned/coroutines/parallel_for(API-surface width is the adoption risk, research-deepdive.md §14); and state once, plainly, the scope boundary (O(100)-node coarse frame skeletons; problem-given million-node graphs are Taskflow/HPC territory — §16). -
[ ](P2, docs) Known-limits section. Draft written in limits.md (2026-07): the honest catalog of what the harness and access declarations do NOT catch — coverage gaps (uninstrumented methods, PODs, escaped refs, sub-object escapes, stale inherited grants, shipping builds) and the failures no declaration system can catch (semantic/order races, deadlock-by-misuse, the completeness hazard), plus the TSan-as-complement posture. Refine and fold into guide.md. -
[ ](P2, trace — handoff to the tracing session) Serial-baseline trace lane. Worker-less (single_threaded) graph runs are excluded from the parallel trace fold (their starts are cumulative serial offsets; dispatch-wait measures the trampoline; the critical path degenerates; utilization has no workers) — but their per-node body durations are the CLEANEST cost-model input there is (no contention, no co-runner cache interference): exactly the per-nodetotal_work2.4's simulator needs. Add a separate duration-only per-node aggregate for worker-less runs + a serial-vs-parallel delta view in the SVG (a node much slower in parallel = contention/false-sharing suspect — a diagnostic nothing else provides). Lives ingraph_trace.h/trace_stamps.h(the tracing session's files; the fold-skip hook instatic_task_graph.cppalready passes the skip flag). -
[ ](P2, trace) Internal profiling: decompose M to optimise the machinery. The ranked 6-point plan in profiler-guided-optimization.md ("Internal profiling" section). Start with 1+2: (1) split machinery into named phases — dispatch/submit already have bridges; add per-worker acquire (pipe_acquire/ handoff) and completion (successor release + dep-counter + object release) accumulators, same armed/relaxed gating; (2) queue-lock-wait per worker (request→hold on the priority-queue mutex) + MPMC CAS-retry counts — prices the sharded/lock-free queue redesign before doing it. (3) cheap ratio counters ride along (find_work hit/miss, local-pop vs steal, park/unpark) — overlaps the per-kind/scheduling-counters trace item; land together. Then 4–6: per-node/per-kind M attribution (reuse thetrace_ownerseam), per-feature microbench ladder over the ~194 ns floor, wake-latency histogram (idle split: no-work vs parked-while-work-available). -
[ ](P2, docs/samples, author 2026-07) Lead users to coroutines. Review every sample (game_frame, physics, blackboard, and the renderer-pattern samples from 10.3) and identify places where a coroutine (co_await ts::read_only/read_write, awaited accesses, suspension across a dependency) would read cleaner than the currentthen/when_all/ continuation shape, then convert or add a coroutine variant wherever it genuinely improves clarity, so the sample set actively demonstrates coroutines instead of treating them as an advanced afterthought. Goal: coroutines become a first-class, encouraged style in the samples and in the guide's disclosure order (10.7), not a footnote. Pairs with 1.15: cleaner high-level coroutine code is the intended alternative to low-level task launches. -
[ ](P2, docs/samples, author 2026-07) Multiple / nested static graphs sample. A single-file sample showing more than oneStatic_task_graphin play, and graphs nested within graphs: a node whose body drives a sub-graph (via nested tasks so it does not block a worker), and independent graphs composed at a higher level. Demonstrates the composition story (large frames built from reusable sub-graphs) and surfaces the sharp edges: object lifetime across graph boundaries (aGuardedtouched by two graphs), the run-quiescence invariant per graph, and whether driving a sub-graph from inside a node risks worker exhaustion (the never-block-in-a-node rule; establish whether retraction/nested cover it). Investigate feasibility first: if a clean composition pattern falls out, ship the sample, else document the constraints. Relates to 2.3 (pipelined execution) and 10.3. Update (2026-08): the mechanism this sample needs is 6.9a (nested graph runs v1 —co_await inner.execute()with the lend protocol); sequence the sample after it, and the "does driving a sub-graph from a node risk worker exhaustion" question is already answered (it does not: an awaited inner run suspends the outer frame, holding no worker). -
[x]DONE (2026-08) — Wave-2writer_ownertests F1–F4 (pipe-rebase-tests.md §F), inpipe_tests§F. Re-scoped for the evolved pipe: F1 (owner set inside a write body, null outside — plus the other half of the invariant, that a READER hold publishes no owner, orcommit()would take its inline arm under a read grant) and F4 (each pipe of a multi-object write names the same block; an untouched object stays unowned) as written; F3 re-aimed at the two inline arms — anaccesson a free pipe is a real admission and publishes its own block, while the REENTRANT arm must leave the outer block as owner (republishing or clearing on the inner settle would mis-dispatchcommit()for the rest of the outer body); F2 re-scoped — the explicit graph write handoff it tested is deleted, so it now pins release-then-admit (two chained writes each name their own block, never the predecessor's, never null). NoPipe_probetype was needed:Guarded_access::pipe()plus a localowner_of()reads the field directly. 517 → 536 checks. -
[x]DONE (2026-08) — the coroutine chain re-measured and decomposed.coro chnreads ~1800–2100 ns/op against ~104 ns/op for a singleco_await, which looked like a coroutine cost worth chasing. It is not. Its figure is per STAGE (50 awaited launched stages, 50 ops reported), and each stage is a full scheduler round trip. A newcoro nstbenchmark runs the identical 50-stage shape with each stage a plain coroutine call — eager tasks complete on the calling thread, so the await takes theawait_readyfast path and only coroutine machinery is measured: 87.9/88.5/87.9 ns across three quiet-host runs (±0.6%) vscoro chn2138.9/1959.2/1809.3 (±8%). Coroutine cost is therefore under 5% of the chain figure; the rest is wake/dispatch latency (cross-checked againstlaunchat 900–1070 ns — a chain stage costs ~2x, which matches paying a wake in each direction). So there is no allocation-shaped gap here (6.2 already landed), and the lever if it ever matters is the worker-loop half of 6.8 or the idle policy, NOT 4.1/6.2. Full verdict in pipe-rebase.md §0.4; keep both benchmark lines. -
[x](P1-cheap, CI) Gate Shipping (TS_SAFETY_CHECKS=0) on every push — DONE. The safety-gated surface grew substantially (circular-wait detector state, grant epochs, scope bookkeeping,writer_owner's always-on exception), and the convention is that safety-only state is FULLY compiled out — a mis-gated field or an#if-invisible behavioral dependency only shows as a Shipping build/link/behaviour break..github/ workflows/ci.ymlruns awindows-shippingjob (clang-cl, thewindows-shippingCMake preset) on every push tomasterand every pull request: configure, build,--tests, with the exe/pdb/crash dump uploaded as artifacts on failure.--stressstayed on the TSan leg rather than duplicating into the Shipping one. -
[ ](P3, tooling/docs — trace tooltips) Deep-link the glossary tooltips to per-term doc pages. Each header glossary term (core utilization, critical path dead time, core-bound/dependency/mixed, core-time, body, framework overhead, idle, frame time, critical path, orchestration) carries a short hover tooltip (graph_trace.hwrite_SVG, the.termoverlay). When the public docs exist, give each term its OWN page — the full explanation (what it measures, how it's computed, the threshold bands) plus concrete guidance for improving it (the cause-specific fixes the tooltip only summarises) — and turn each tooltip into a link to that page (a footer line the overlay renders as a clickable anchor / opens in a new tab). The tooltip stays the short form, the page is the long form. Gated on the docs site existing; until then the tooltips are self-contained.
-
17. `[ ]` **(P3, parked by design — per-run trace metric sinks.)** Tracing's counter layer
(busy/body/orchestration slots, bucket rows, the per-owner sink) is scheduler-global and
single-consumer; overlapping traced runs are fatal since 2026-08-28 (`arm_busy_tracking`).
Moving the counters into per-run sinks (a sink pointer riding the task, inherited like
`current_trace_owner`) is tractable (~2-3 days: block/TLS plumbing, fold rewiring, the
worker-less oracle and `Scheduler_profiling` seam, tests) but was deliberately parked:
utilization and idle are properties of the shared pool, so concurrent traced runs would
produce mutually contaminated headline numbers wherever the counters live - only
attribution (B, per-node busy, task counts) can be made per-run. Revisit if a real user
asks to trace two graphs concurrently, and ship with that caveat stated.
Small API/doc frictions introduced or exposed this session — cheap to fix, costly to change after public. Feed the going-public "API-stability pass".
run_inlinedead foraccess/async— RESOLVED (Access_options split). The access surface now takesAccess_options{token, priority}(task.h), separate fromTask_options{token, priority, run_inline}(then/task builders) — passingrun_inlinetoaccess/asyncis now a compile error, not a silent no-op. Revisit during code review (flagged in-code): whether the two option types should share a base or stay split.- Multi-object
ts::access≠ opportunistic — FIXED (2026-08). Was: single-objectaccessruns inline when free while multi-objectts::accessscheduled likets::async. Both now share one caller-ownedAccess_opand one dispatch policy (lend, then all-or-nothing inline probe, else the queued cascade) — see §1.2 and multi-access-op-design.md. - Stale
asynccomments after the split — FIXED. Sweptguarded.h(default_scheduler,pipe_try_inline, the two access/async doc blocks). pipevsqueueterminology — DONE (2026-08-23, API-stability S9). The original decision stands and is now implemented:docs/guide.mdno longer contains the word - the concept is spelled "the object's queue", "its turn" and "admission", and the glossary defines those instead.README.mdnever used it. Pipe stays in the internal docs (design/internals/pipe-rebase) and in the headers, wherePipeis the real type name. The field keeps the namequeued(author decision): it is the only place a user types the concept, and it is already in the user vocabulary.- Reservation path still heap-boxes a closure — folded into the allocation campaign.
Pipe::Job::on_acquired(move_only_function) allocates on the graph / multi-async reservation path (unlike the allocation-free block/async dispatch); the tunable-SBOFunction<Sig,N>item is the fix. Not a standalone task. - Safety-field gating convention — RESOLVED (fully gated, 2026-07). The convention is
ratified in CLAUDE.md's style section: safety-only state is fully gated by
TS_SAFETY_CHECKS, fields included; the only unconditional exception is genuinely free storage (a spare bit in an existing packed byte, e.g.Flags::pipe_job). T1's two style-1 spots aligned:Pipe::write_epochandAccess_context::Entry's epoch pair are now gated (capture sites go throughdetail::pipe_epoch, which returns null with the harness off — shipping contexts shrink by 16 bytes/entry). The "layout stability" argument was hollow — mixed-config TUs violate ODR through inline-function bodies regardless — and is now enforced instead of argued: a link-time tripwire inaccess.h(#pragma detect_mismatchon MSVC-family; a config-named anchor symbol, defined inaccess.cpp, elsewhere) makes a mixed-config link fail with a diagnostic. The same convention governs the shelved interval-grant entry layout (escaped-refs-hardening.md §6.1) if revived. - Entity-naming approach — RESOLVED (author-designed, 2026-08). Ratified as the leading
ts::Namedwrapper, unified across all three entity kinds:Node_nameis deleted andadd_nodetakests::Namedin the same leading position;Guarded/Versionedkeep theirs; tasks carry one too.Namedis a literal or a captured call site ({literal, file, line}, implicit from a string literal,ts::Named{}for the site alone), displayed as the literal elsebasename(file):line. Required for nodes and objects — their unnamed constructors and thenode<N>/objNordinal fallbacks are gone — and optional for tasks, where the verb captures the creation site by default. The load-bearing implementation rule (the defaultedsource_locationmust sit on the OUTERMOST function the user calls, with the resultingNamedpassed down explicitly) is stated ints/named.hand pinned bytests/named_tests.cpp. Specifies 6.19.
Today every access heap-allocates a Task_control_block (~288 B) even when it runs inline;
the coroutine guard is the proof it needn't (stack-only Access_guard under a scoped grant).
Three tiers: (1) void access on a free grant → run under a stack-scoped grant, return an
already-done Task<void> sentinel (no block; ships independently). (2) small non-void R →
Task<R> becomes a discriminated union inline-value | block-ptr | done-void, storing R inline
below an SBO threshold (lean fixed-small ~2–3 words; a discriminant branch in
sync/then/is_done, not hot-path); larger R → heap. (3) large R / deferred → heap (the
floor). Composes with the SBO-Function and per-type free-list items.
The harness only fires from instrumented methods — a forgotten TS_CHECK_ACCESS() is a silent
hole. Belts: a clang-tidy check (every public method of a guarded type opens with the macro;
needs a way to mark guarded types), and a POD wrapper — a POD in Guarded has no methods to
instrument, so undeclared field access is invisible; offer a value-mediating Guarded_value<T>
or lean on the range harness (D-adjacent) so a grant covers the POD's bytes.
Superseded by the single-global choice: exactly one process-wide pool, reconfigured via
configure_scheduler/Scheduler_scope, with construction factory-gated and the
thread_local current_scheduler selector retired (2026-08). The multi-pool / ambient-override
direction below was not taken. Kept for the rationale record only.
launch/task/access are hardwired to a process-wide default_scheduler(); only
graph.execute(scheduler) is explicit. The global default is the right terse default, but the
hard global blocks testability (deterministic/inline scheduler), embedding (a library imposes
its pool on the host), multiple pools, and named-thread targeting. Fix (the thread_local current_scheduler half-exists): resolve to the ambient scheduler, add a scoped
Scheduler_scope guard(s) (tokio enter()), optional per-op Launch_options{.scheduler};
Guarded binds ambient at construction. Overridable ambient — not full-explicit, not hard-global.
~6 functions behind a header: park(addr, expected, timeout?) / unpark_one / unpark_all
(Win WaitOnAddress, Linux futex, macOS __ulock/os_sync; std::atomic::wait lacks the
timeout needed for standby workers/watchdogs → plan the raw-futex swap), thread_spawn /
set_name / set_affinity, cpu_count + minimal topology, cpu_relax/yield. Everything
else (eventcount, deques, MPMC queues, loop) stays portable. Console = fiber backend behind the
same Task API; mobile = QoS + oversubscription over a fixed pool.
Inside one coarse Guarded grant, an internal parallel_for where entity i reads+writes
entity j. Per-entity locks are the anti-pattern (contention/convoying/SIMD-hostile/non-det —
loses to privatization even at 8 threads). Cross-domain survey (Box2D v3, Jolt, Unity DOTS,
Avian, LAMMPS/GROMACS, PRISM, Galois, Bevy/Flecs) → ship: (1) parallel_gather_apply<Msg>
(two-pass mailbox: record→bin-by-target→apply; deterministic via sorted inboxes; the intra-loop
sibling of the graph command buffer) — the general default; (2) parallel_for_colored +
persistent Interaction_coloring (greedy bitset, overflow→serial) — freshest-value Gauss-Seidel;
(3) Accumulator<T> per-worker slabs + reduce — commutative subset; (4) Union_find
helper + doc pattern for islands (not an executor); (5) triage docs + the anti-pattern note.
All run inside the coarse grant — same trust model as disjoint parallel_for chunks; no
pipe/graph/harness changes. Open author questions: commutativity share; Gauss-Seidel freshness
load-bearing?; pair provenance (persistent→coloring vs discovered→mailbox); hot-entity max degree;
determinism default; POD-first vs closure.
Priority flows backward along dependency edges (consumer→producer, taking max) — the dual of
cancellation's forward flow. Keep order, only raise dispatch priority (never reorder). Three
instantiations: pipe (dispatch head at max(own, queued-behind), O(1), on by default),
static graph (reverse-topo max once at compile(), no residual window — do first), dynamic
chains (donate at launch, bounded, opt-in on the consumer). Caveat: unbounded propagation floods
the high queue and starves background → opt-in/bounded, per-workload. Not OS thread priority
(syscall cost, permission-gated; a fixed per-pool QoS-background attribute is the only legit
OS-priority use → platform layer).
Pipe-rebase addendum (2026-08): the tail-chain pipe (1.14, branch pipe-rebase) makes
pipe order strictly FIFO by construction — a task's priority applies at dispatch (queue
placement once its turn arrives) but never jumps the line, so a high-priority task queued
behind a long low-priority body waits out the body. That is today's semantics too, but the
pipe instantiation above ("dispatch head at max(own, queued-behind)") must be re-expressed
on the chain when this item lands: the "head" is now the running link's owner, and propagation
= walking next links backward-in-priority to raise the owner's dispatch priority at its
turn. Review the interaction then; the chain makes the walk cheap (the links are right there)
but the raise must happen before the owner's dispatch_ready reads flags.priority.
Lazy typed sender + receiver + connect→op-state (stack-allocatable) + start; algorithms fold
into one nested op-state. Could give: zero-alloc structured composition (the structural answer to
allocation, but only for the awaited subset), and — the distinctive angle — env/queries as the
Access_context carrier (typed, composable replacement for thread_local current_access;
opens compile-time access checking). Tensions: clashes with the deliberately not-templated
monomorphic block (compile-time + error-message tax), doesn't fix fire-and-forget allocation,
different usage model, P2300 still settling. Verdict: coroutines first (done); prototype
access-context-as-env as the one spike; don't rebuild the engine.