perf: pipeline commitment with execution in parallel path (async ComputeCommitment + BAL-driven warmup) #19791

Background

Performance analysis of parallel and serial mainnet sync nodes (~24.4M–24.6M blocks) shows the per-10,000-block cost breakdown:

Two observations drive this issue:

1. Commitment is the bottleneck in the parallel path during sync — exec is already hidden behind the previous batch's commitment across batch boundaries. But at tip (one block at a time, 12s slots) this pipeline collapses: each block commits before the next starts.

2. At tip, commitment dominates per-block latency — with small key sets per block (~1–5k vs ~300k during sync), commitment takes seconds while exec takes hundreds of ms. Pipelining them is the primary lever for reducing tip latency.

Serial path is out of scope — it remains unchanged and serves as the correctness baseline.

Two execution modes

This work introduces two distinct paths depending on whether a Block Access List (EIP-7928, Amsterdam+) is present.

Exec-driven path (no BAL — sync, pre-Amsterdam blocks)

Normal execution → state writes → async pipelined commitment.

Block N+1 arrives (no BAL)
  ├── [B] Sync: consume block N commitment result (check root hash)
  │       FlushDeferredCommitmentUpdates(tx) → MDBX
  ├── [C] Execute block N+1 (parallel EVM workers, writes to VersionedWrites)
  ├── [D] applyVersionedWrites → SharedDomains.mem
  │       Flush mem → MDBX
  └── [E] Launch async commitment goroutine (paraTrieDB, deferred updates)
Block N+2 starts immediately at [B]

BAL-driven path (BAL present — Amsterdam+ blocks at tip or in full block sync)

The BAL contains the complete per-tx state diff with values (balance, nonce, code, storage changes). This means we can apply state changes and compute commitment from the BAL directly, without waiting for execution results.

Block N+1 arrives (BAL present — all state diffs known)
  │
  ├── [A] Validate BAL against header BAL hash (fast, before exec)
  │
  ├── [B] Sync: consume block N commitment result (check root hash)
  │       FlushDeferredCommitmentUpdates(tx) → MDBX
  │
  ├── [C] Apply BAL final-state writes → SharedDomains.mem    ← NO exec needed for state
  │       Flush mem → MDBX
  │       Launch async commitment goroutine (paraTrieDB, deferred updates)
  │
  ├── [D] Execute block N+1 in **comparison mode** (concurrent with [E])
  │       EVM runs normally, but writes go to a comparison buffer (not SharedDomains)
  │       At end: compare exec outputs against BAL values
  │       If mismatch → reject block (BAL is invalid)
  │
  └── [E] (commitment goroutine from [C] runs concurrently with [D])
            reads BAL key set via paraTrieDB (branch nodes pre-loaded)
            hashes Patricia trie
            defers branch updates in memory

Block N+2 can start (BAL apply + exec) once both [D] and [E] complete.

Key property: The exec path does not write to SharedDomains. The BAL is the source of truth for state changes. Execution validates correctness but does not drive the write path. This means commitment starts as soon as the BAL is applied — fully concurrent with EVM execution.

Must work without BAL: The exec-driven path is always the fallback. The code must select the path at runtime based on whether a BAL is present.

Why the scaffolding already exists

1. paraTrieDB / EnableParaTrieDB — read-only MDBX handle opened alongside the write tx. Commitment goroutine reads branch nodes here without blocking the write tx.

2. SetDeferCommitmentUpdates / FlushPendingUpdates — branch updates accumulate in memory, applied at a sync point. Decouples the commitment goroutine from the write tx.

3. VersionedIO / BAL comparison — AsBlockAccessList() already derives a BAL from exec outputs. The inverse (applying a BAL to state, then comparing exec outputs to it) uses the same data structures.

4. ProcessBAL — already validates exec-derived BAL against header BAL hash. The BAL-driven path inverts this: apply the header BAL, then validate exec produces the same diff.

Required changes

Phase 1 — async ComputeCommitment (exec-driven path, core change)

db/state/execctx/domain_shared.go

• Make ComputeCommitment operate entirely via paraTrieDB + deferred writes
• Add ComputeCommitmentAsync(ctx) <-chan commitResult
• commitResult carries {rootHash []byte, err error}

execution/stagedsync/exec3_parallel.go

• Replace blocking ComputeCommitment with ComputeCommitmentAsync
• Store pendingCommitment <-chan commitResult across loop iterations
• Sync at top of apply cycle, FlushDeferredCommitmentUpdates before next flush

Phase 2 — BAL-driven path

execution/stagedsync/exec3_parallel.go

• On newPayload for Amsterdam+ blocks with BAL:
  1. Validate BAL hash against block header
  2. Apply BAL final-state writes to SharedDomains.mem (new ApplyBALWrites function)
  3. Flush mem → MDBX, launch async commitment goroutine
  4. Run EVM workers in comparison mode: writes to comparison buffer, not SharedDomains
  5. After exec: compare comparison buffer against BAL values → accept/reject

execution/state/rw_v3.go

• Add comparisonCollector — a StateWriter that records writes into a buffer without touching SharedDomains
• Add comparison function: diff comparison buffer vs BAL's AccountChanges slice

db/state/execctx/domain_shared.go

• Add ApplyBALWrites(bal types.BlockAccessList) — applies BAL final-state values directly to shared domains mem buffer

execution/stagedsync/exec3_parallel.go (continued)

• Runtime selection: if bal != nil { balDrivenPath(...) } else { execDrivenPath(...) }
• Both paths converge at the sync point (step [B]) consuming the pending commitment result

Correctness constraints

• Single outstanding commitment: only one goroutine at a time — enforced by sync point [B]
• BAL validation first: BAL hash validated against block header before any state writes
• Comparison is mandatory: EVM must still execute fully in BAL-driven mode; mismatch → bad block
• Fallback path: any block without a BAL (sync, pre-Amsterdam) uses exec-driven path
• MDBX snapshot isolation: paraTrieDB sees block N's committed state; unaffected by block N+1 mem writes
• Error handling: goroutine errors surface at sync point [B]; bad-block handling unchanged

Performance impact

Exec-driven (sync): Commitment already pipelined across batch boundaries. Phase 1 extends this to single-block boundaries at tip — removes commitment from the critical path.

BAL-driven (tip): Commitment starts as soon as BAL is applied (before EVM starts). For a 12s slot with ~200ms exec + ~2s commitment, tip latency collapses from ~2.2s to max(0.2s exec, 2s commit started earlier) ≈ 0, bounded by block propagation time.

Dependencies

• PR state, stagedsync: IBS 2-cache refactor phase 2+3 — replace StateUpdates with VersionedWrites #19711 (ibs-2cache-phase2-3) must merge first — it cleans the VersionedWrites → applyVersionedWrites → SharedDomains write path that this work builds directly on.

Out of scope

• Serial execution path (unchanged, used as correctness baseline)
• Parallel Patricia hash (ParallelHashSort) — can be layered on top as a follow-on
• Tx-level conflict reduction (separate workstream)