Issue:
While trying to execute a pipeline that makes use of a Kafka origin or destination, the pipeline fails. The following error message is seen in the SDC logs:
ERROR SafeScheduledExecutorService - Uncaught throwable from com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable@8535ae4: com.streamsets.datacollector.util.PipelineException: PREVIEW_0003 - Encountered error while previewing : java.lang.IllegalArgumentException: Could not find a 'KafkaClient' entry in the JAAS configuration. System property 'java.security.auth.login.config' is not set
com.streamsets.datacollector.util.PipelineException: PREVIEW_0003 - Encountered error while previewing : java.lang.IllegalArgumentException: Could not find a 'KafkaClient' entry in the JAAS configuration. System property 'java.security.auth.login.config' is not set
at com.streamsets.datacollector.execution.preview.sync.SyncPreviewer.start(SyncPreviewer.java:259)
at com.streamsets.datacollector.execution.preview.async.AsyncPreviewer.lambda$start$1(AsyncPreviewer.java:98)
at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.lambda$call$0(SafeScheduledExecutorService.java:226)
at com.streamsets.datacollector.security.GroupsInScope.execute(GroupsInScope.java:33)
at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.call(SafeScheduledExecutorService.java:222)
at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.lambda$call$0(SafeScheduledExecutorService.java:226)
at com.streamsets.datacollector.security.GroupsInScope.execute(GroupsInScope.java:33)
at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.call(SafeScheduledExecutorService.java:222)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at com.streamsets.datacollector.metrics.MetricSafeScheduledExecutorService$MetricsTask.run(MetricSafeScheduledExecutorService.java:100)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IllegalArgumentException: Could not find a 'KafkaClient' entry in the JAAS configuration. System property 'java.security.auth.login.config' is not set
Solution:
The pipeline failure is caused by a configuration property not being set for the Data Collector instance on which the pipeline is running. Referring to the documentation for enabling the Kafka stages for Kerberos[1], there is an explicit step to define the 'java.security.auth.login.config' property which controls the explicit path to the jaas.conf file required for Kerberos authentication.
In order to resolve the issue, please verify that all necessary steps have been completed to enable Kerberos authentication for the Kafka stages as listed in the documentation. Specifically, verify that the 'java.security.auth.login.config' property has been defined in the SDC_JAVA_OPTS parameter and that it points to a valid jaas.conf file for your SDC instance.