Issue:
Getting below error while running Kafka consumer and Kafka multitopic consumer.
caught throwable from com.streamsets.datacollector.execution.runner.common.AsyncRunner$$Lambda$267/2146734391@63b45c46: java.lang.NoClassDefFoundError: Could not initialize classorg.xerial.snappy.Snappykafka.clients.consumer.internals.Fetcher.fetchedRecords(Fetcher.java:523)
at org.apache.kafka.clients.consumer.KafkaConsumer.pollForFetches(KafkaConsumer.java:1259)
at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1187)
at
org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1115)
at com.streamsets.pipeline.stage.origin.multikafka.v0_10.loader.Kafka0_10ConsumerLoader$WrapperKafkaConsumer.poll(Kafka0_10ConsumerLoader.java:166)
at com.streamsets.pipeline.stage.origin.multikafka.MultiKafkaSource$MultiTopicCallable.call(MultiKafkaSource.java:184)
at com.streamsets.pipeline.stage.origin.multikafka.MultiKafkaSource$MultiTopicCallable.call(MultiKafkaSource.java:131)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Solution:
The above error occurs when (for security reasons) /tmp is mounted non-exec, So to resolve this we need to make /tmp as exec or we need to add -Dorg.xerial.snappy.tempdir=<folder> in sdc-env.sh and restart the service