Skip to main content

the pipeline is being started by control-m

 

following the execution of the pipeline by the streamsets in the option "view logs" we verify that the pipeline starts in STANDALONE mode and then presents a ERROR THROWABLE in log, as shown in the logs below:

 

2022-07-07 03:23:54,937    vilage_secret/vilagesecret9922xdfc-e39e-4235-a85f-a9f9b83a04f2    INFO    Pipeline execution mode is: STANDALONE    StandaloneAndClusterRunnerProviderImpl    admin        webserver-14858

2022-07-07 03:23:54,937    vilage_secret/vilagesecret9922xdfc-e39e-4235-a85f-a9f9b83a04f2    INFO    Preparing to start pipeline 'vilagesecret9922xdfc-e39e-4235-a85f-a9f9b83a04f2::0'    StandaloneRunner    admin        webserver-14858

2022-07-07 03:23:58,412    vilage_secret/vilagesecret9922xdfc-e39e-4235-a85f-a9f9b83a04f2    ERROR    Uncaught throwable from com.streamsets.datacollector.execution.runner.common.AsyncRunner$$Lambda$234/990753129@290e6c97: java.lang.IllegalStateException: Cannot start the pipeline 'vilagesecret9922xdfc-e39e-4235-a85f-a9f9b83a04f2::0' as the runner is already closed    SafeScheduledExecutorService    *admin        runner-pool-2-thread-148

java.lang.IllegalStateException: Cannot start the pipeline 'vilagesecret9922xdfc-e39e-4235-a85f-a9f9b83a04f2::0' as the runner is already closed
    at com.streamsets.pipeline.api.impl.Utils.checkState(Utils.java:87)
    at com.streamsets.datacollector.execution.runner.standalone.StandaloneRunner.startPipeline(StandaloneRunner.java:725)
    at com.streamsets.datacollector.execution.runner.standalone.StandaloneRunner.start(StandaloneRunner.java:717)
    at com.streamsets.datacollector.execution.runner.common.AsyncRunner.lambda$start$3(AsyncRunner.java:151)
    at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.lambda$call$0(SafeScheduledExecutorService.java:226)
    at com.streamsets.datacollector.security.GroupsInScope.execute(GroupsInScope.java:33)
    at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.call(SafeScheduledExecutorService.java:222)
    at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.lambda$call$0(SafeScheduledExecutorService.java:226)
    at com.streamsets.datacollector.security.GroupsInScope.execute(GroupsInScope.java:33)
    at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.call(SafeScheduledExecutorService.java:222)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
    at com.streamsets.datacollector.metrics.MetricSafeScheduledExecutorService$MetricsTask.run(MetricSafeScheduledExecutorService.java:100)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
    
2022-07-07 06:17:14,707    vilage_secret/vilagesecret9922xdfc-e39e-4235-a85f-a9f9b83a04f2    INFO    Pipeline execution mode is: STANDALONE

2022-07-07 09:00:23,578    vilage_secret/vilagesecret9922xdfc-e39e-4235-a85f-a9f9b83a04f2    INFO    Preparing to stop pipeline    StandaloneRunner    admin        webserver-19680


when we STOP this pipeline, the pipeline status is STOPPING and when we try to FORCE STOP it gives the error below:

 

2022-07-07 09:00:35,946    vilage_secret/vilagesecret9922xdfc-e39e-4235-a85f-a9f9b83a04f2    INFO    Ignoring force quit request because pipeline is in STOPPING state

Hi @Yago Aparecido,

Which SDC version are you using?
 

The error looks familiar to me and I feel it got solved in newer versions.

Does the pipeline get stuck in “Stopping” state? Or it is just taking time there?

hi @alex.sanchez 

 

We are using sdc in version 3.12.0

 

The pipeline gets stuck in the "stopping" state

My recommendation will be to upgrade to 5.0, 3.12 is almost 3 years old, and we have improved a lot of things over that time

@alex.sanchez 

 

wow the jump from 3.12 to 5.0 is too big and also sdc is being used in cloudera manager

 

from what we can see, the latest version of sdc released for cloudera manager is version 3.22.3

 

Could this version 3.22.3 fix these issues?

Any 4.x and 5.x versions are available as a Cloudera Parcel, I’m not sure why does not show in the Cloudera Manager, but they should work without any problem, we have deprecated some old Cloudera versions, but they are still supported.

@alex.sanchez 

 

could you send us the download link of the cloudera sdc parcels?

 

do you think version 3.22.3 can solve this pipeline problem?

 

I think the error was solved for 4.0 or some newer version.

Please check the docs here (https://docs.streamsets.com/portal/datacollector/latest/help/datacollector/UserGuide/Installation/CMInstall-Overview.html?hl=cloudera) I feel this is exactly what you are looking for
 

@alex.sanchez 

 

checking the version from the link, unfortunately the last released version of sdc for cloudera manager was 3.22.3

 

 

Reply


Terms & Conditions