Kafka Stage fails with error "Request joining group due to: rebalance failed due to 'null' (DisconnectException)"

Issue

A Kafka stage fails to connect to the broker due to the follow error stack trace:

INFO  ConsumerCoordinator - (Re-)joining group
INFO  NetworkClient - Disconnecting from node xxx due to socket connection setup timeout. The timeout value is 8424 ms.
INFO  ConsumerCoordinator - Group coordinator is unavailable or invalid due to cause: null. isDisconnected: true. Rediscovery will be attempted.
2023-09-27 13:07:58,776 INFO  ConsumerCoordinator - Request joining group due to: rebalance failed due to 'null' (DisconnectException)

Background

This specific Kafka Client exception appears when the Kafka Client’s socket-connection-setup-timeout-max-ms is set too low, causing it to attempt to re-join the group.

The maximum amount of time the client will wait for the socket connection to be established. The connection setup timeout will increase exponentially for each consecutive connection failure up to this maximum.

Solution

When this property is set too low, the Kafka Client will time out during the initial socket connection. To resolve this issue, simply add this property socket-connection-setup-timeout-max-ms to your Additional Kafka Properties in your Kafka stage with a value large enough to allow the initial socket connection to complete on the first try.