Skip to content

Expose an IO-like, Enumerable response body #117

Description

@ZilvinasKucinskas

Priority: P1

Related API contract: Provide a stable error hierarchy and safe, class-specific metadata

Describe the issue

Wreq::Response currently mixes response metadata and body ownership. bytes, text, and json fully buffer the body, while chunks is a one-shot operation that requires a block. There is no body object with the ordinary Ruby protocols for incremental reading, enumeration, and cleanup.

After chunks or close, another body read raises Wreq::MemoryError with a long explanation of Rust ownership. That exposes an implementation detail instead of a clear Ruby stream state.

Proposed Ruby API

response = client.get("https://example.com/archive")
body = response.body

body.read(16_384)       # => binary String or nil at EOF
body.readpartial(16_384)
body.each { |chunk| consume(chunk) }
body.each               # => Enumerator
body.close
body.closed?            # => true

Response#each_chunk can be a convenience delegate:

response.each_chunk { |chunk| consume(chunk) }
response.each_chunk # => Enumerator

Keep Response#chunks as a compatibility wrapper. bytes, text, and json can remain response conveniences implemented on top of the body.

Body semantics

  • The body is IO-like; it does not need to subclass IO.
  • read(length = nil, out_buffer = nil) and readpartial(max_length, out_buffer = nil) follow Ruby IO return/EOF conventions: a positive-length read returns nil when already at EOF, while readpartial raises EOFError.
  • each/each_chunk yield binary strings and return an Enumerator without a block.
  • Only one consumer owns an unbuffered network stream. Concurrent reads raise a concise state error.
  • close is idempotent, closed? is thread-safe, and post-close reads raise IOError or a typed Wreq::BodyStateError < Wreq::BodyError.
  • Breaking enumeration early releases or discards the connection according to documented pool rules.
  • If convenience methods memoize a fully buffered body, document whether later reads are repeatable and whether rewind becomes available. A live network stream need not be rewindable.
  • Caller block exceptions propagate unchanged.
  • The body must make it possible for a future Rust wreq decoded-byte limit to stop reads at the same boundary; that native prerequisite should be filed separately in Rust wreq.

Acceptance criteria

  • Response#body returns a stable body object.
  • The body supports read, readpartial, each, close, and closed? with standard Ruby behavior.
  • Blockless each and Response#each_chunk return Enumerators.
  • EOF, explicit close, double close, early break, block failure, and competing-reader states are tested.
  • No normal body state raises Wreq::MemoryError or exposes Rust ownership terminology.
  • Streaming does not buffer the full response and continues to release the GVL while waiting for network data.

Ruby ecosystem precedent

  • HTTPX response handling treats a response body as both an IO-like reader and an enumerable, with explicit close semantics.
  • HTTP::Response::Body supports incremental reads, enumeration, and conversion to a complete string.
  • Net::HTTPResponse#read_body provides incremental block-based reads.
  • wreq-python exposes the same native response stream through iterable/context-managed Streamer and explicit response close APIs.

Relevant wreq-ruby source

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions