Skip to content

feat(runtime): add Velo TCP and UCX response transport with configurable batching #16026

Open

Description

@jthomson04

Summary

Add an optional Velo response transport to Dynamo, with TCP and UCX message transport support. Velo owns batching, flow control, and stream lifecycle. Keep the existing TCP response transport as the default and keep QUIC separate.

Implementation: #15231.

Requirements

  • Select Velo with DYN_RESPONSE_PLANE=velo and its transport with DYN_VELO_RESPONSE_TRANSPORT=tcp|ucx; gate UCX behind the velo-ucx feature.
  • Use a shared Velo service with Runtime ownership and shutdown, while preserving typed prologues, response payloads, errors, ordered completion, Stop/Kill, and consumer cancellation.
  • Validate configuration at startup and reject incompatible response protocol versions before generation.
  • Default to a maximum 5 ms batching delay. Support DYN_VELO_RESPONSE_BATCH_MS=0 for the existing greedy policy and positive integer values for other delays. Reject invalid values at startup.
  • Preserve the 60 KiB batch cap, credit window 32, four TCP lanes, and current reply timing.

Acceptance

  • Response lifecycle, startup, and configuration tests pass for TCP and UCX over TCP; formatting and warnings-denied Clippy pass.
  • Record hardware RDMA checks and the directional comparison with QUIC and default TCP in the PR. Distinguish measured sources from later configuration changes and retain measurement limits.
  • Keep benchmark scripts and data local, preserve the default transport, and obtain passing required CI and review approval before merge.

This issue tracks the integration in #15231, not a new transport framework or a redesign of router response forwarding.

Activity

  1. added
    needs-triageIssue awaiting triage review
    runtimeCODEOWNER area -> @ai-dynamo/dynamo-runtime-codeowners
    on Oct 11, 2026
  2. jthomson04 commented on Oct 19, 2026

    @jthomson04
    ContributorAuthor

    DEP for this change: #16809 (make Velo the default response plane).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    dynamo-runtimeRelates to the dynamo-runtime componentenhancementNew feature or requestlanguage::rustIssues/PRs that reference Rust codeneeds-triageIssue awaiting triage reviewruntimeCODEOWNER area -> @ai-dynamo/dynamo-runtime-codeowners

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions