Product: StreamSets Data Collector
Issue:
WARN Can't commit offset to Kafka: org.apache.kafka.clients.consumer.CommitFailedException: Commit cannot be completed since the group has already re-balanced and assigned the partitions to another member. This means that the time between subsequent calls to poll() was longer than the configured max.poll.interval.ms, which typically implies that the poll loop is spending too much time message processing. You can address this either by increasing the session timeout or by reducing the maximum size of batches returned in poll() with max.poll.records.
Solution:
Check if there is any bottleneck in the destination which might block Kafka to process faster and try to increase the following parameters:
max.poll.interval.ms = 1500000 session.timeout.ms = 30000 heartbeat.interval.ms = 10000
