Skip to main content
Solved

running error pipeline couldn`t force stop

  • December 28, 2021
  • 9 replies
  • 644 views
L
Fan

 

Running Error pipeline couldn`t stop, when I click the button of force stop, nothing happened, what should i do?

Best answer by Rishi

Hii @levanye ,

FORCE QUIT:

  • When one does the force quit, the batch which is processing could be stopped abruptly and the offset for the batch might not be stored. When the pipeline is restarted the incomplete batch will start from the beginning which could cause duplication of data.
  • Generally, If the force quit doesn’t work that could be because of lack of resources in SDC to stop the pipeline then you need to change the pipeline state manually.

MANUALLY CHANGING THE PIPELINE STATE:

  • One could manually stop the pipeline by going to   

    $SDC_DATA/runInfo/<pipeline_name>/0/pipelineState.json  and changing the value of  status to STOPPED and restarting the Data Collector:

    "status":"STOPPED"

View original
Did this topic help you find an answer to your question?

9 replies

Rishi
StreamSets Employee
Forum|alt.badge.img
  • RishiStreamSets Employee
  • StreamSets Employee
  • 96 replies
  • Answer
  • January 17, 2022

Hii @levanye ,

FORCE QUIT:

  • When one does the force quit, the batch which is processing could be stopped abruptly and the offset for the batch might not be stored. When the pipeline is restarted the incomplete batch will start from the beginning which could cause duplication of data.
  • Generally, If the force quit doesn’t work that could be because of lack of resources in SDC to stop the pipeline then you need to change the pipeline state manually.

MANUALLY CHANGING THE PIPELINE STATE:

  • One could manually stop the pipeline by going to   

    $SDC_DATA/runInfo/<pipeline_name>/0/pipelineState.json  and changing the value of  status to STOPPED and restarting the Data Collector:

    "status":"STOPPED"

Drew Kreiger
Rock star
Forum|alt.badge.img
  • Drew KreigerRock star
  • Senior Community Builder at StreamSets
  • 95 replies
  • February 7, 2022

@levanye did Rishi’s suggestions help? If so, Please make sure to select the “best answer” on ones correct response. This helps with other users quickly searching. :) 

L
Fan
  • levanyeFan
  • Author
  • Fan
  • 1 reply
  • February 15, 2022

get it !!!  thanks very much ! @Rishi 

 

Drew Kreiger
1 person likes this
  • Drew Kreiger
    Drew Kreiger
Drew Kreiger
Rock star
Forum|alt.badge.img
  • Drew KreigerRock star
  • Senior Community Builder at StreamSets
  • 95 replies
  • February 15, 2022

@levanye Awesome! Thanks for selecting the best answer. Have a wonderful day. 

songkunhuang
Fan

thanks very much :)

S
Headliner
Forum|alt.badge.img+1

@Rishi , Thanks for the explanation. I have encountered this situation multiple times and I was aware of the “pipelinestate.json” file but I never tried to change that file manually. Good to know the trick.

Srinivasan Sankar
Shawn
Fan
  • ShawnFan
  • Fan
  • 1 reply
  • August 4, 2022
levanye wrote:

get it !!!  thanks very much ! @Rishi 

 

I`m follow guide change the `piplineState.json`, but it is invaild. 

Did I miss any steps?

Version:

StreamSets Data Collector 3.17.1 

hennaji
Fan
  • hennajiFan
  • Fan
  • 1 reply
  • October 9, 2023

Hello everybody, 

I faced the same issue, I tried to manually stop , it works , but when I wanted to restart it took a long time starting, then I faced the same issue again. Please, I want to understand what does it mean a lack of ressource like you mentionned in the first comment, do we need to add some memory capacity in the config?


Thanks 

hennaji
1 person likes this
  • hennaji
    hennaji
saleempothiwala
Headliner
Forum|alt.badge.img

@hennaji please check your java configs to see that you are making full use of the available memory. 

Reply


Terms & Conditions