Solved

Update “globalMaxRetries” job config using REST API Call

  • 8 November 2022
  • 4 replies
  • 76 views

Hello Team,

I need to update one job config which is “globalMaxRetries” using rest call.

I refer Restful section in Control Hub but that Rest call for “updateJob” requires many other configs also needs to mention in Request Body.

 

could you please assist me with this ?

icon

Best answer by Rishi 8 November 2022, 07:02

View original

4 replies

Userlevel 4
Badge

Hello @yogesh0590 

Step1 : Get the Job Json  which will return Job for given Job ID

GET https://XXX.hub.streamsets.com/jobrunner/rest/v1/job/{jobId}

Step2: Modify the ‘globalMaxRetries’  parameter to desired number in received json blob from step

Step 3: Call the POST API to Update Job for given Job ID

POST https://XXX.hub.streamsets.com/jobrunner/rest/v1/job/{jobId}
 curl -X POST https://XXX.hub.streamsets.com/jobrunner/rest/v1/job/72b8200c-0e3b-426e-b564-bceb17220b1e:8c2c652f-e3d9-11eb-9fb3-b974ac4c3f67  -d '{
"id": "72b8200c-0e3b-426e-b564-bceb17220b1e:8c2c652f-e3d9-11eb-9fb3-b974ac4c3f67",
"name": "Job for Date Conversions",
"description": "sd",
"organization": "8c2c652f-e3d9-11eb-9fb3-b974ac4c3f67",
"creator": "8072136c-e3d9-11eb-9fb3-8929bd105d08@8c2c652f-e3d9-11eb-9fb3-b974ac4c3f67",
"createTime": 1666862894707,
"lastModifiedBy": "8072136c-e3d9-11eb-9fb3-8929bd105d08@8c2c652f-e3d9-11eb-9fb3-b974ac4c3f67",
"lastModifiedOn": 1666862894707,
"destroyer": null,
"deleteTime": 0,
"jobDeleted": false,
"pipelineName": "Date Conversions",
"pipelineId": "589c51ce-3df6-49aa-ab0a-f6860471ed46:8c2c652f-e3d9-11eb-9fb3-b974ac4c3f67",
"pipelineCommitId": "a096d193-bdb1-473b-b10d-4c2ef7863bf8:8c2c652f-e3d9-11eb-9fb3-b974ac4c3f67",
"rulesId": "2dbb7682-ba5e-4576-9deb-26cf6863d8ec:8c2c652f-e3d9-11eb-9fb3-b974ac4c3f67",
"pipelineCommitLabel": "v1",
"labels": [
"aws"
],
"currentJobStatus": {
"id": "5f71cc74-6567-445b-bb35-e7814d53cb3a:8c2c652f-e3d9-11eb-9fb3-b974ac4c3f67",
"jobId": "72b8200c-0e3b-426e-b564-bceb17220b1e:8c2c652f-e3d9-11eb-9fb3-b974ac4c3f67",
"pipelineCommitId": "a096d193-bdb1-473b-b10d-4c2ef7863bf8:8c2c652f-e3d9-11eb-9fb3-b974ac4c3f67",
"runCount": 0,
"user": "8072136c-e3d9-11eb-9fb3-8929bd105d08@8c2c652f-e3d9-11eb-9fb3-b974ac4c3f67",
"startTime": 1667886900675,
"finishTime": 0,
"errorMessage": null,
"color": "GRAY",
"sdcIds": [],
"status": "INACTIVE",
"errorInfos": [],
"warnings": [],
"ackTrackers": [],
"pipelineOffsets": [],
"pipelineStatus": [],
"upgradeStatus": null,
"pipelineCommitLabel": null,
"edge": false,
"inputRecordCount": 0,
"outputRecordCount": 0,
"errorRecordCount": 0,
"lastReportedMetricTime": 0,
"enginePipelineId": null,
"runtimeAttributes": []
},
"currentUpgradeStatus": null,
"parentJobId": null,
"systemJobId": null,
"numInstances": 1,
"migrateOffsets": true,
"statsRefreshInterval": 60000,
"runtimeParameters": "{}",
"edge": false,
"timeSeries": false,
"forceStopTimeout": 120000,
"maxRetriesForFailedJob": -1,
"provenanceMetaData": {},
"jobTemplate": false,
"templateJobId": null,
"cdConfig": null,
"rawJobTags": [],
"jobTags": null,
"executorType": "COLLECTOR",
"globalMaxRetries": 2,
"needsManualAck": true,
"staticParameters": [],
"archived": false,
"deleteAfterCompletion": false,
"draftRun": false
}' -H "Content-Type:application/json" -H "X-Requested-By:curl" -H "X-SS-REST-CALL:true" -H "X-SS-App-Component-Id: $CRED_ID" -H "X-SS-App-Auth-Token: $CRED_TOKEN" -i

 

@Rishi , Do I need to store JSON first or I can change the param on the fly ?

And also will it impact any other config as we only and only need to change that configuration ?

Userlevel 4
Badge

You need to pass this json as part of  POST API call, so first you get this json and then modify and then update.

It will not impact other parameter or config of the job.

 

How to pass the modified content in a variable? I have the need to run this as automated with no manual intervention. I tried below two ways but none worked. I dont see any other viable option to pass modified content in the curl command.

var1=`cat /home/script/modified.json`

curl -X POST https://XXX.hub.streamsets.com/jobrunner/rest/v1/job/72b8200c-0e3b-426e-b564-bceb17220b1e:8c2c652f-e3d9-11eb-9fb3-b974ac4c3f67 -d '{ @var1}’ -H "Content-Type:application/json" -H "X-Requested-By:curl" -H "X-SS-REST-CALL:true" -H "X-SS-App-Component-Id: $CRED_ID" -H "X-SS-App-Auth-Token: $CRED_TOKEN" -i

 

curl -X POST https://XXX.hub.streamsets.com/jobrunner/rest/v1/job/72b8200c-0e3b-426e-b564-bceb17220b1e:8c2c652f-e3d9-11eb-9fb3-b974ac4c3f67 -d '{ /home/script/modified.json}’ -H "Content-Type:application/json" -H "X-Requested-By:curl" -H "X-SS-REST-CALL:true" -H "X-SS-App-Component-Id: $CRED_ID" -H "X-SS-App-Auth-Token: $CRED_TOKEN" -i

 

Reply