Critical: HTTP2 stream will load whole response into memory

[hissssst]

Mix.install [:plug, :bandit, :finch]

defmodule P do
  @behaviour Plug
  import Plug.Conn

  def init(_ \\ []) do
    {:ok, []}
  end

  def call(conn, _) do
    conn = send_chunked(conn, 200)
    Enum.reduce(1..1_000_000_000, conn, fn _, conn ->
      {:ok, conn} = chunk(conn, String.duplicate("0", 1_000_000))
      conn
    end)
  end
end

Bandit.start_link(ip: {0, 0, 0, 0}, plug: P, port: 8000, http_1_options: [enabled: true])

Finch.start_link(name: TestFinch, pools: %{default: [size: 100, count: 1, protocols: [:http2]]})

r = Finch.build("GET", "http://localhost:8000/")
Finch.stream(r, TestFinch, [], fn _, _ ->
  IO.inspect Process.info(self(), :message_queue_len)
  Process.sleep(1_000)
end)

Run this and see in logs

{:message_queue_len, 1}
{:message_queue_len, 15902}
{:message_queue_len, 33221}
{:message_queue_len, 50260}
{:message_queue_len, 67273}
{:message_queue_len, 84020}

Current implementation of streaming in HTTP2 is extremely dangerous since it may just OOM the caller, which is a bad thing to do. I'd mark http2 adapter as not ready for production use and raise in any client using it, until this gets fixed

[hissssst]

I've reproduced it until the program got killed by OOM-killer, consuming 16gb (max on my machine) total

[hissssst]

@sneako , I don't think that it is a good thing to consider this issue as completed when there's no solution, but only a warning

[TheOneric]

For reference, is this an issue in FInch’s HTTP2 implementation, or something which needs to be addressed directly in Mint?

[sneako]

The issue is in Finch