Skip to content

Connecting to Kafka with 3.4.0 is very slow #1128

Open
Open
Connecting to Kafka with 3.4.0 is very slow#1128

Description

@kirrg001
kirrg001
opened on May 6, 2025

Environment Information

  • OS [e.g. Mac, Arch, Windows 10]: Mac
  • Node Version [e.g. 8.2.1]: v20
  • NPM Version [e.g. 5.4.2]: v10
  • C++ Toolchain [e.g. Visual Studio, llvm, g++]:
  • node-rdkafka version [e.g. 2.3.3]: 3.4.0

Steps to Reproduce

Download the zip, unzip, ensure Kafka is running on 9020 and run:

npm i

// uses 3.4.0 (latest, slow)
npm run start

// uses 3.3.1 (previous, fast)
npm run start-prev

Run it a couple of times, you will see that the start command is very slow (up to 20s) and the start-prev is always < 50ms.

I have tried it with:

  • Kafka v2 + Zookeeper v3
  • Kafka v3 + Zookeeper v3

With the update from 3.3.1 to 3.4.0 we are not just experiencing a slow connection. We are in general getting suddenly random timeout errors in our tests and the delivery-report is completely broken for us. I will post my findings on #1123

Thanks!

reproduce.zip

Activity

  1. kirrg001 commented on May 6, 2025

    @kirrg001
    kirrg001
    on May 6, 2025
    Author

    Further discovery: Consuming messaging is randomly not working in our test env. The message is sent from a producer process and not consumed from a consumer process (standard, no stream). Its simple code from the README.md. Nothing special. Works stable with 3.3.1 - is flaky with 3.4.0.

  2. nottisa commented on Jun 7, 2025

    @nottisa
    nottisa
    on Jun 7, 2025

    Experiencing the same issue, downgraded to 3.3.1 and we stopped experiencing it.

  3. yduman commented on Dec 10, 2025

    @yduman
    yduman
    on Dec 10, 2025

    I was looking into updating to the latest version since I want migrate towards Node.js 24 and I was keeping an eye on this issue. I did some research myself and this issue should not happen anymore. The bug originates from librdkafka 2.10.0 which was introduced with node-rdkafka 3.4.0.

    The bug was tracked on these issues:
    confluentinc/librdkafka#5100
    confluentinc/librdkafka#5142

    This was fixed with librdkafka 2.11.1. and additional confidence added with 2.12.0, meaning node-rdkafka versions 3.5.0 and above should behave correctly now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions