feat(sync): default public SWM to RFC-64 scheduling - #2275
Conversation
a1c16ce to
5f1746f
Compare
| // Product-default scheduling is deliberately separate from the signed | ||
| // catalog authority surface above. Every explicitly requested CG is | ||
| // eligible for RFC-64 selected PUBLIC-SWM scheduling; private CGs retain | ||
| // curator recovery, and only `completeSwmProviders` may prove a whole |
There was a problem hiding this comment.
🔴 Bug: Status omits network-default context graphs from the RFC-64 requested scope
What's wrong
The new status field is documented as the daemon's requested subscription scope and is called out as a preflight surface, but it excludes context graphs that are requested via the active network overlay. Those CGs are still subscribed at startup and, if public, use the selected scheduling behavior added in this PR, so the API can report a false negative.
Example
With a network overlay whose defaultContextGraphs is ['network-public-cg'] and no explicit config.contextGraphs, daemon startup subscribes and catches up network-public-cg, but /api/status reports rfc64SelectedPublicSync.requestedContextGraphs: []. A preflight client using this new status field will miss a CG that is actually using the default selected-public scheduling path.
Suggested direction
Build this projection from the same effective subscription inputs as lifecycle startup, including network defaults, so the public status API does not under-report selected scheduling scope.
For Agents
In packages/cli/src/daemon/routes/status.ts, align rfc64SelectedPublicSync.requestedContextGraphs with the daemon startup subscription scope. Reintroduce/use resolveNetworkDefaultContextGraphs(network) alongside config and catalog CGs, preserving de-duplication. Add a status-route case with a non-empty network.defaultContextGraphs to prove the projection matches lifecycle startup scope.
There was a problem hiding this comment.
🟡 Issue: Do not keep a config-backed fallback for the live selected-sync scope
What's wrong
The new status field is explicitly described as a live runtime projection, but the route now carries two definitions of that scope. That makes future maintenance harder because readers must reason about both the agent-owned live state and a stale startup-config reconstruction path.
Example
After a runtime subscription mutates the agent's sync scope, any route test double or alternate agent adapter that omits getSyncContextGraphIds() silently reports resolveContextGraphs(config) + catalog instead of the live scope. That keeps the stale projection alive instead of making the boundary fail clearly.
Suggested direction
Use one canonical source for rfc64SelectedPublicSync.selectedContextGraphs: the live agent accessor. If compatibility with older agents is truly needed, put that adaptation in a narrow versioned adapter rather than in the route body.
For Agents
In packages/cli/src/daemon/routes/status.ts, make the live sync-scope accessor part of the required status agent boundary and remove the fallback branch. Update status-route-rpc.test.ts stubs to supply the accessor explicitly. Preserve the response shape and deduping behavior.
| // terminal verdict describes only that peer's local manifest and must not | ||
| // replace multi-peer union convergence. A complete-provider request that | ||
| // receives an older/raw host response remains fail-closed. | ||
| const progress = terminalBoundaryRequired |
There was a problem hiding this comment.
🟡 Issue: Selected scheduling and terminal proof are conflated in normalization
What's wrong
This PR introduces two different concepts: selected scheduling for public SWM, and terminal whole-scope authority for complete providers. The invocation path now distinguishes them, but the normalization path still uses only terminalBoundaryRequired to decide whether a selected-lane result gets selected-SWM freshness semantics. That leaves a misleading boundary where selected-lane payloads from ordinary public peers are treated like raw durable progress, making the implementation harder to reason about and easy to regress.
Example
An ordinary public peer can now be invoked through the selected lane and return { kind: 'selected-shared-memory', shared: { failedPhases: 1, snapshotPlaneIncomplete: 1, resolvedSnapshotPlaneIncomplete: 1, ... }, selectedScopeComplete: true }. Because terminalBoundaryRequired is false for that peer, this path uses classifyDurableProgress(payload) and treats the resolved selected-lane yield as a blocking phase failure. The terminal proof should be ignored for ordinary peers, but the selected-lane freshness normalization should not disappear with it.
Suggested direction
Model selected-SWM scheduling and terminal-boundary authority as two explicit axes. The code-judo move is to make a single per-peer shared-memory policy and let normalization use selected freshness independently from terminal proof.
For Agents
In packages/cli/src/catchup-runner-worker-impl.ts, replace the separate booleans with a small shared-memory peer policy object, e.g. { selectedSchedulingRequested, terminalBoundaryRequired }. Pass that object to both the RPC invocation and normalization. Preserve behavior that only terminalBoundaryRequired peers can increment selectedScopeCompletePeers, but use classifySharedMemoryFreshness whenever the selected lane was requested. Add or update a worker test where a non-complete-provider public peer returns resolved selected-SWM freshness counters and is classified cleanly without being promoted to selected-scope proof.
There was a problem hiding this comment.
🟡 Issue: Model selected-SWM scheduling and terminal authority as one explicit policy
What's wrong
The PR introduces an important distinction between selected scheduling and graph-complete terminal authority, but represents it as two correlated booleans threaded through an already dense orchestration path. The code works by comments and local naming rather than by a boundary that preserves the invariant.
Example
selectedSchedulingRequested and terminalBoundaryRequired have the same type and adjacent lifetimes. Accidentally passing the terminal flag to invoke('syncSharedMemory', ...) or the scheduling flag to normalizeCatchupSharedMemoryResult(...) would compile while changing the selected-SWM semantics.
Suggested direction
Replace the pair of loose booleans with a named policy object or discriminated union, then pass that policy to the RPC adapter and normalizer. This would centralize the RFC-64 distinction and let the comments shrink into type names.
For Agents
In packages/cli/src/catchup-runner-worker-impl.ts, extract a small resolveSharedMemoryCatchupPolicy(request, prepared, peerId) or closed union. Preserve: public non-private CGs request selected scheduling, only authoritativeSharedMemoryPeerIds require terminal selected proof, and shared-first remains authority-only. Keep the existing ordinary-public, complete-provider, and private-CG tests around that mapping.
| }, | ||
| // Product-default scheduling is deliberately separate from the signed | ||
| // catalog authority surface above. Every explicitly requested CG is | ||
| // eligible for RFC-64 selected PUBLIC-SWM scheduling; private CGs retain |
There was a problem hiding this comment.
🟡 Issue: Status builds a public scheduling projection from an unclassified config list
What's wrong
The new status block claims to expose RFC-64 selected PUBLIC-SWM scheduling, but it is assembled inline from config/catalog arrays in the status route. That duplicates part of the scheduling model outside the runner and does not carry the runner's public/private classification. This makes the status API a parallel approximation of the runtime policy rather than a canonical projection of it.
Example
A config containing a private CG in contextGraphs will still show that CG under rfc64SelectedPublicSync.requestedContextGraphs, even though the worker path deliberately sends selectedSchedulingRequested = false when prepared.isPrivateContextGraph is true.
Suggested direction
Move this projection behind a small named helper that owns the RFC-64 selected-public status model and either filters with the same public/private invariant as catch-up or names the field as an unclassified requested scope.
Confidence note
This is based on the route projection and runner policy visible in the diff; if catalog-selected graphs are guaranteed elsewhere to be explicit public subscriptions, the author should point to that invariant and preferably encode it in the projection helper.
For Agents
Extract a typed status projection helper near the RFC-64/catch-up boundary, or have the route call a canonical agent/config helper that can apply the same public/private policy as catch-up preparation. Preserve the current response shape, but avoid building a public-only scheduling status from an unclassified config list inside the monolithic route. Add a status-route test for a private configured CG or explicitly document and encode why this projection is intentionally only a requested-scope approximation.
| authoritativePeerId: 'peer-curator', | ||
| authoritativeSharedMemoryPeerIds: [], | ||
| isPrivateContextGraph: false, | ||
| peerIds: ['peer-curator'], |
There was a problem hiding this comment.
🟡 Issue: Single-peer test does not verify ordinary selected-complete replies keep multi-peer convergence alive
What's wrong
The changed behavior explicitly separates selected scheduling from graph-complete authority across candidate peers. The added test verifies the selected flag and the completion counter, but not the risky part: that an ordinary peer's selected terminal verdict cannot stop the remaining peer walk and strand SWM rows held by another peer.
Example
A regression where ordinary selected-complete SWM replies set authorityProven.sharedMemory would still pass this test, because there is no peer-member that should be contacted afterward. A stronger sketch would use two public peers, have the first ordinary peer return { kind: 'selected-shared-memory', selectedScopeComplete: true }, have the second return SWM data, and assert both peers were called and selectedScopeCompletePeers remains 0.
Suggested direction
Make this regression test include at least two candidate peers and assert the later peer is still contacted for shared memory after the first ordinary peer returns a selected-complete wrapper.
For Agents
In packages/cli/test/catchup-runner-worker-impl.test.ts, extend the public selected-default test or add a sibling multi-peer case. Preserve the selected flag assertion for ordinary public peers, but prove that an ordinary selected-complete response does not terminate or narrow the SWM walk before a later candidate peer is reached.
| // catalog authority surface above. Every explicitly requested CG is | ||
| // eligible for RFC-64 selected PUBLIC-SWM scheduling; private CGs retain | ||
| // curator recovery, and only `completeSwmProviders` may prove a whole | ||
| // public SWM scope terminal. The harness knows its generated CG is public |
There was a problem hiding this comment.
🟡 Issue: Catalog-backed selected-public status projection is untested
What's wrong
The new API contract exposes catalog-backed context graphs separately and also folds them into the requested selected-public scheduling scope. Existing activation tests still assert only the older rfc64PublicCatalog block, so regressions in these new fields would pass green.
Example
If /api/status accidentally returned requestedContextGraphs: resolveContextGraphs(config) only, or always returned catalogBackedContextGraphs: [], the new test at lines 413-423 would still pass. A catalog-enabled status test should expect requestedContextGraphs to include the catalog-selected CG and catalogBackedContextGraphs to equal that catalog scope.
Suggested direction
Add status-route coverage for the catalog-enabled projection, not just the explicit-subscription/no-catalog case.
For Agents
In packages/cli/test/status-route-rpc.test.ts, amend one of the RFC-64 activation tests or add a focused one with rfc64PublicCatalog.enabled: true and an accepted policy. Assert the new rfc64SelectedPublicSync block includes both explicit and catalog-backed scopes as intended, including de-duplication if both sources name the same CG.
2 participants
Impact
An explicit foreground subscription to a public Context Graph now uses the RFC-64 selected SWM scheduler and bounded continuation lane by default on every candidate peer. Operators no longer need a signed-catalog manifest merely to get selected scheduling for the CG they explicitly requested.
This does not sync all public CGs and does not make ordinary peers authoritative. Multi-peer union convergence remains in force unless the operator explicitly pins an RFC-64
completeSwmProviderspeer. Only that stronger provider assertion may prove the whole SWM scope terminal. Private CG curator recovery and chain-driven VM recovery are unchanged.GET /api/statusnow exposes the independently testable product default asrfc64SelectedPublicSync, sourced from the agent's live sync scope rather than stale startup configuration. This gives the Blackbox harness a fail-closed, exact-CG preflight after subscription and before it publishes or spends.Before
sequenceDiagram participant User participant Receiver participant Peers User->>Receiver: Subscribe to public CG with SWM Receiver->>Peers: Ordinary bounded peer walk Note over Receiver,Peers: RFC-64 continuation required a pinned complete providerAfter
sequenceDiagram participant User participant Receiver participant Peers User->>Receiver: Subscribe to explicit public CG with SWM Receiver->>Receiver: Add CG to live sync scope Receiver->>Receiver: Select RFC-64 scheduling by default Receiver->>Peers: Run selected scheduling and bounded continuation alt Complete provider explicitly pinned Peers-->>Receiver: Graph-complete terminal proof Receiver-->>User: Whole SWM scope proven else Ordinary candidate peers Peers-->>Receiver: Per-peer content and progress Receiver->>Peers: Preserve multi-peer union convergence endSafety boundaries
selectedScopeCompletePeersor terminate the graph-wide walk.Evidence
git diff --check: clean.Harness
Companion harness PR: OriginTrail/dkg-blackbox-harness#15. It refuses an SWM run before ticket issuance/publication unless the exact run CG appears under
rfc64SelectedPublicSync.selectedContextGraphswith the product default enabled.