Question

pipeline in StreamSets Data Collector 3.12.0 is not stopping normally, even clicking "force stop"


pipeline in StreamSets Data Collector 3.12.0 is not stopping normally, even clicking "force stop", we can only stop the pipeline if we restart the streamsets service

 

why is StreamSets Data Collector 3.12.0 having this issue and how to fix it?

 

PS: StreamSets Data Collector 3.12.0 is installed in Cloudera Express 6.3.


8 replies

Userlevel 2
Badge

Hi @Yago Aparecido. Can you provide details about the stages that your pipeline runs? Does this happen with any pipeline and any stage, or whe you run this specific pipeline?

Userlevel 3
Badge

One thing worth noting is that SDC 3.12.0 came out back in 2019 and we've made over a dozen releases with hundreds of bug fixes since then. Before spending too much time trying to debug things, it might be worth upgrading to a more modern version of the software to see if your issue has already been solved.

hi @Dimas Cabré i Chacón 

 

the problem is when we are going to manually stop the pipeline

 

we select the pipeline, click on "stop", the status stays on "stopping" and does not disappear, when we click on "force stop" nothing happens either, when we restart the streamsets it goes back to normal, so we can stop without any problem.

hi @dima 

 

I understood!

 

as streamsets is installed and managed by cloudera manager, is there any procedure for updating to a newer version?

 

we are thinking of upgrading to version 3.22.3, doing the whole process through cloudera manager.

 

Can this update process have any impact on pipelines? such as, for example, some component used in the pipeline stops working, some component becomes inaccessible, etc.

What would be the best practices to perform this update?

Userlevel 3
Badge

Hi @Yago Aparecido,

 

Please check out our docs for details.

hi @dima 

So, this behavior that I passed earlier that the streamsets is presenting can we consider a "bug" and only the update can solve it?

Userlevel 3
Badge

I think it's fair to say that having to restart Data Collector to stop a pipeline is not normal behavior. :)

@dima 

as this is a "bug", we will align the update to version 3.22.3 that is available in cloudera manager

Reply