Skip to content

fix(extensions): harden plugin rpc bridge against stdin flush hangs, watchdog deadlock delays, and manifest request timeouts #666

Description

@ZhuchkaTriplesix

Problem Statement

During the extension driver resilience audit, three edge-case failure modes were identified in JsonRpcStdioClient and PluginRpcBridge:

  1. Unbounded _stdin.flush(): In JsonRpcStdioClient.sendRequest, await _stdin.flush() has no timeout. If a child process deadlocks or stops draining stdin, when the OS pipe buffer (~64KB) fills up, _stdin.flush() hangs indefinitely without triggering request timeout.
  2. Watchdog Deadlock Delay: When SandboxWatchdog detects a ping timeout (deadlock), it kills the handle, but cancels _exitSub beforehand so PluginRpcBridge does not receive the exit event to abort in-flight requests immediately with PluginDeadlockException.
  3. Short Request Timeout vs Manifest: Extensions like clickhouse-query-ext specify analytical timeouts (e.g. timeout_seconds: 600), but PluginRpcBridge hardcodes requestTimeout = const Duration(seconds: 30).
  4. Handshake Timeout Cushion: A strict 3s handshake timeout can cause false positives during cold start or bwrap setup on slow systems; also does not include stderr lines when timing out.

Proposed Solution

  1. Wrap _stdin.flush() with a bounded timeout (Duration(seconds: 3)) in JsonRpcStdioClient.
  2. In PluginRpcBridge.onStopped, if reason is deadlock, immediately call _failPending(PluginDeadlockException(...)) and dispose.
  3. Configure requestTimeout in PluginRpcBridge based on manifest.sandbox?.permissions?.resources?.timeoutSeconds when available.
  4. Increase default handshake timeout to 6 seconds and capture last stderr lines on handshake timeout.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions