Solved

running error pipeline couldn`t force stop

  • 28 December 2021
  • 9 replies
  • 528 views

 

Running Error pipeline couldn`t stop, when I click the button of force stop, nothing happened, what should i do?

icon

Best answer by Rishi 17 January 2022, 07:16

View original

9 replies

Userlevel 4
Badge

Hii @levanye ,

FORCE QUIT:

  • When one does the force quit, the batch which is processing could be stopped abruptly and the offset for the batch might not be stored. When the pipeline is restarted the incomplete batch will start from the beginning which could cause duplication of data.
  • Generally, If the force quit doesn’t work that could be because of lack of resources in SDC to stop the pipeline then you need to change the pipeline state manually.

MANUALLY CHANGING THE PIPELINE STATE:

  • One could manually stop the pipeline by going to   

    $SDC_DATA/runInfo/<pipeline_name>/0/pipelineState.json  and changing the value of  status to STOPPED and restarting the Data Collector:

    "status":"STOPPED"

Userlevel 5
Badge

@levanye did Rishi’s suggestions help? If so, Please make sure to select the “best answer” on ones correct response. This helps with other users quickly searching. :) 

get it !!!  thanks very much ! @Rishi 

 

Userlevel 5
Badge

@levanye Awesome! Thanks for selecting the best answer. Have a wonderful day. 

thanks very much :)

Userlevel 3
Badge +1

@Rishi , Thanks for the explanation. I have encountered this situation multiple times and I was aware of the “pipelinestate.json” file but I never tried to change that file manually. Good to know the trick.

get it !!!  thanks very much ! @Rishi 

 

I`m follow guide change the `piplineState.json`, but it is invaild. 

Did I miss any steps?

Version:

StreamSets Data Collector 3.17.1 

Hello everybody, 

I faced the same issue, I tried to manually stop , it works , but when I wanted to restart it took a long time starting, then I faced the same issue again. Please, I want to understand what does it mean a lack of ressource like you mentionned in the first comment, do we need to add some memory capacity in the config?


Thanks 

Userlevel 4
Badge

@hennaji please check your java configs to see that you are making full use of the available memory. 

Reply