Skip to content

[Rust] Expose persistent stream API - #763

Open
elenagaljak-db wants to merge 1 commit into
mainfrom
stack/eos-public-api
Open

elenagaljak-db wants to merge 1 commit into
mainfrom
stack/eos-public-api

Conversation

@elenagaljak-db

@elenagaljak-db elenagaljak-db commented Aug 20, 2026 •

Copy link
Copy Markdown
Collaborator

What changes are proposed in this pull request?

This PR exposes the feature-gated Rust API for creating and resuming persistent streams on top of #762.

It adds:

  • ZerobusSdk::persistent_stream_builder().
  • PersistentStreamBuilder with the existing table, authentication, format, recovery, timeout, backpressure, and callback configuration.
  • build() for a new durable stream and resume(stream_id) for reconnecting.
  • PersistentStream, exposing stream_id, the resume watermark, ingestion, batch ingestion, flush, offset waiting, close, and unacknowledged-record retrieval.
  • Shared builder preparation so ephemeral and persistent streams resolve identical gRPC configuration.

JSON and protobuf are supported; Arrow Flight and a public retirement API are intentionally deferred. Documentation follows the pipelined loop-then-flush() pattern. The new public feature is recorded in rust/NEXT_CHANGELOG.md.

Stack

  1. [Rust] Add persistent stream proto #761 — protobuf contract
  2. [Rust] Add persistent gRPC transport #762 — gRPC transport and recovery engine
  3. [Rust] Expose persistent stream API #763 — public Rust API
  4. [Rust] Test persistent stream resume #764 — stateful mock and integration tests
  5. [Rust] Document persistent streams #765 — documentation and runnable example

How is this tested?

  • cargo check -p databricks-zerobus-ingest-sdk --features eos
  • cargo test -p databricks-zerobus-ingest-sdk --features eos --lib — 156 tests passed
  • cargo fmt --all -- --check
  • Public create/resume behavior is exercised in [Rust] Test persistent stream resume #764.

@elenagaljak-db
elenagaljak-db force-pushed the stack/eos-grpc-core branch 2 times, most recently from 18a643f to abf5a92 Compare August 20, 2026 13:09
@elenagaljak-db
elenagaljak-db force-pushed the stack/eos-public-api branch 3 times, most recently from 0ecfe35 to c202abf Compare August 28, 2026 16:41
@elenagaljak-db
elenagaljak-db force-pushed the stack/eos-public-api branch 2 times, most recently from 8a1eda0 to c722ab2 Compare September 25, 2026 14:03
Base automatically changed from stack/eos-grpc-core to main September 25, 2026 15:34
Signed-off-by: elenagaljak-db <elena.galjak@databricks.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant