Skip to content

[Rust] Extract shared mux core - #865

Open
danilonajkov-db wants to merge 2 commits into
mainfrom
danilo-najkov_data/effort/mux-streams-core
Open

danilonajkov-db wants to merge 2 commits into
mainfrom
danilo-najkov_data/effort/mux-streams-core

Conversation

@danilonajkov-db

@danilonajkov-db danilonajkov-db commented Sep 22, 2026 •

Copy link
Copy Markdown
Member

What changes are proposed in this pull request?

For easier and cleaner multiplexing for arrow streams I extract the existing multiplexed-stream routing and lifecycle logic into a private MuxCore. I also add a generic MuxLane with the main methods needed for multiplexing. Then we maintain very small implementations for grpc and arrow streams.

MultiplexedStream            public API (thin wrapper, gRPC-specific bits)
   │  core: MuxCore<ZerobusStream>
   ▼
MuxCore<S: MuxLane>          policy: spreading work over N lanes, poisoning, lifecycle
   │  streams: Vec<S>
   ▼
MuxLane (trait)              mechanism: what a single transport stream must provide
   └─ impl for ZerobusStream (gRPC); Arrow lane comes in #866

Stack base: main, including merged Avro PR #857. Follow-ups: #866 (Arrow API), then #867 (lifecycle tests).

How is this tested?

  • Existing gRPC mux integration suite: 24 passed.
  • SDK Clippy with all features and all targets, warnings denied.
  • Full Rust make test and make lint matrices passed on the combined implementation.
  • Rust formatting and diff whitespace checks.

@danilonajkov-db
danilonajkov-db force-pushed the danilo-najkov_data/effort/mux-streams-avro branch from aafe33f to 06b8474 Compare September 22, 2026 15:02
Base automatically changed from danilo-najkov_data/effort/mux-streams-avro to main September 23, 2026 08:14
@danilonajkov-db
danilonajkov-db force-pushed the danilo-najkov_data/effort/mux-streams-core branch 3 times, most recently from fe13be4 to 9328853 Compare September 23, 2026 14:54
@danilonajkov-db danilonajkov-db changed the title [Rust] Extract shared mux core [Rust] Share mux core and order close errors Sep 23, 2026
@danilonajkov-db
danilonajkov-db force-pushed the danilo-najkov_data/effort/mux-streams-core branch from 9328853 to fe13be4 Compare September 23, 2026 15:18
@danilonajkov-db danilonajkov-db changed the title [Rust] Share mux core and order close errors [Rust] Extract shared mux core Sep 23, 2026
@danilonajkov-db
danilonajkov-db marked this pull request as ready for review September 23, 2026 16:05
@danilonajkov-db
danilonajkov-db force-pushed the danilo-najkov_data/effort/mux-streams-core branch from ebeb366 to 85a48bb Compare September 25, 2026 13:32
Signed-off-by: danilo-najkov-db <danilo.najkov@databricks.com>
Signed-off-by: danilo-najkov-db <danilo.najkov@databricks.com>
@danilonajkov-db
danilonajkov-db force-pushed the danilo-najkov_data/effort/mux-streams-core branch from 85a48bb to f55b303 Compare September 25, 2026 15:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant