Hello,
We performed mapr-setup as per documentation, but unfortunatelly we are facing with pretty limited functionality of sdc and following entries in sdc.log:
#####
4-9963-6352a0fc023c-tsystems_maprfs_test] ]stage:] ERROR ProductionPipelineRunnable - An exception occurred while running the pipeline, com.streamsets.datacollector.runner.PipelineRuntimeException: CONTAINER_0702 - Pipeline initialization error: java.lang.NoClassDefFoundError: Could not initialize class org.apache.hadoop.security.UserGroupInformation
com.streamsets.datacollector.runner.PipelineRuntimeException: CONTAINER_0702 - Pipeline initialization error: java.lang.NoClassDefFoundError: Could not initialize class org.apache.hadoop.security.UserGroupInformation
at com.streamsets.datacollector.execution.runner.common.ProductionPipeline.run(ProductionPipeline.java:106)
at com.streamsets.datacollector.execution.runner.common.ProductionPipelineRunnable.run(ProductionPipelineRunnable.java:75)
at com.streamsets.datacollector.execution.runner.standalone.StandaloneRunner.start(StandaloneRunner.java:726)
at com.streamsets.datacollector.execution.AbstractRunner.lambda$scheduleForRetries$0(AbstractRunner.java:363)
at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.lambda$call$0(SafeScheduledExecutorService.java:226)
at com.streamsets.datacollector.security.GroupsInScope.execute(GroupsInScope.java:34)
at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.call(SafeScheduledExecutorService.java:222)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at com.streamsets.datacollector.metrics.MetricSafeScheduledExecutorService$MetricsTask.run(MetricSafeScheduledExecutorService.java:100)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NoClassDefFoundError: Could not initialize class org.apache.hadoop.security.UserGroupInformation
at com.streamsets.datacollector.security.MapRLoginUgiProvider.getLoginUgi(MapRLoginUgiProvider.java:48)
at com.streamsets.datacollector.security.HadoopSecurityUtil.getLoginUser(HadoopSecurityUtil.java:35)
at com.streamsets.pipeline.lib.hdfs.common.HdfsBaseConfigBean.validateHadoopFS(HdfsBaseConfigBean.java:286)
at com.streamsets.pipeline.stage.origin.hdfs.HdfsSourceConfigBean.init(HdfsSourceConfigBean.java:40)
at com.streamsets.pipeline.stage.origin.hdfs.HdfsSource.init(HdfsSource.java:59)
at com.streamsets.pipeline.api.base.BaseStage.init(BaseStage.java:48)
at com.streamsets.pipeline.api.base.configurablestage.DStage.init(DStage.java:36)
at com.streamsets.datacollector.runner.StageRuntime.lambda$init$0(StageRuntime.java:220)
at com.streamsets.datacollector.util.LambdaUtil.withClassLoaderInternal(LambdaUtil.java:148)
at com.streamsets.datacollector.util.LambdaUtil.withClassLoader(LambdaUtil.java:44)
at com.streamsets.datacollector.runner.StageRuntime.init(StageRuntime.java:218)
at com.streamsets.datacollector.runner.StagePipe.init(StagePipe.java:109)
at com.streamsets.datacollector.runner.StagePipe.init(StagePipe.java:44)
at com.streamsets.datacollector.runner.Pipeline.initPipe(Pipeline.java:396)
at com.streamsets.datacollector.runner.Pipeline.init(Pipeline.java:301)
at com.streamsets.datacollector.execution.runner.common.ProductionPipeline.run(ProductionPipeline.java:98)
... 13 more
#####
Do you have any idea how to solve it? We checked the libraries and symbolic link were created successfully for new version of MapR beside old ones. We set also correct ownership and right, but still the same.
Current versions:
MapR core 6.1.1
MEP 6.3.6
SDC 3.16
Thank you!
michal.kertes