Hello Team,
I need to update one job config which is “globalMaxRetries” using rest call.
I refer Restful section in Control Hub but that Rest call for “updateJob” requires many other configs also needs to mention in Request Body.
could you please assist me with this ?
Best answer by Rishi
Hello
Step1 : Get the Job Json which will return Job for given Job ID
| GET https://XXX.hub.streamsets.com/jobrunner/rest/v1/job/{jobId} |
Step2: Modify the ‘globalMaxRetries’ parameter to desired number in received json blob from step
Step 3: Call the POST API to Update Job for given Job ID
| POST | https://XXX.hub.streamsets.com/jobrunner/rest/v1/job/{jobId} |
curl -X POST https://XXX.hub.streamsets.com/jobrunner/rest/v1/job/72b8200c-0e3b-426e-b564-bceb17220b1e:8c2c652f-e3d9-11eb-9fb3-b974ac4c3f67 -d '{
"id": "72b8200c-0e3b-426e-b564-bceb17220b1e:8c2c652f-e3d9-11eb-9fb3-b974ac4c3f67",
"name": "Job for Date Conversions",
"description": "sd",
"organization": "8c2c652f-e3d9-11eb-9fb3-b974ac4c3f67",
"creator": "8072136c-e3d9-11eb-9fb3-8929bd105d08@8c2c652f-e3d9-11eb-9fb3-b974ac4c3f67",
"createTime": 1666862894707,
"lastModifiedBy": "8072136c-e3d9-11eb-9fb3-8929bd105d08@8c2c652f-e3d9-11eb-9fb3-b974ac4c3f67",
"lastModifiedOn": 1666862894707,
"destroyer": null,
"deleteTime": 0,
"jobDeleted": false,
"pipelineName": "Date Conversions",
"pipelineId": "589c51ce-3df6-49aa-ab0a-f6860471ed46:8c2c652f-e3d9-11eb-9fb3-b974ac4c3f67",
"pipelineCommitId": "a096d193-bdb1-473b-b10d-4c2ef7863bf8:8c2c652f-e3d9-11eb-9fb3-b974ac4c3f67",
"rulesId": "2dbb7682-ba5e-4576-9deb-26cf6863d8ec:8c2c652f-e3d9-11eb-9fb3-b974ac4c3f67",
"pipelineCommitLabel": "v1",
"labels": [
"aws"
],
"currentJobStatus": {
"id": "5f71cc74-6567-445b-bb35-e7814d53cb3a:8c2c652f-e3d9-11eb-9fb3-b974ac4c3f67",
"jobId": "72b8200c-0e3b-426e-b564-bceb17220b1e:8c2c652f-e3d9-11eb-9fb3-b974ac4c3f67",
"pipelineCommitId": "a096d193-bdb1-473b-b10d-4c2ef7863bf8:8c2c652f-e3d9-11eb-9fb3-b974ac4c3f67",
"runCount": 0,
"user": "8072136c-e3d9-11eb-9fb3-8929bd105d08@8c2c652f-e3d9-11eb-9fb3-b974ac4c3f67",
"startTime": 1667886900675,
"finishTime": 0,
"errorMessage": null,
"color": "GRAY",
"sdcIds": [],
"status": "INACTIVE",
"errorInfos": [],
"warnings": [],
"ackTrackers": [],
"pipelineOffsets": [],
"pipelineStatus": [],
"upgradeStatus": null,
"pipelineCommitLabel": null,
"edge": false,
"inputRecordCount": 0,
"outputRecordCount": 0,
"errorRecordCount": 0,
"lastReportedMetricTime": 0,
"enginePipelineId": null,
"runtimeAttributes": []
},
"currentUpgradeStatus": null,
"parentJobId": null,
"systemJobId": null,
"numInstances": 1,
"migrateOffsets": true,
"statsRefreshInterval": 60000,
"runtimeParameters": "{}",
"edge": false,
"timeSeries": false,
"forceStopTimeout": 120000,
"maxRetriesForFailedJob": -1,
"provenanceMetaData": {},
"jobTemplate": false,
"templateJobId": null,
"cdConfig": null,
"rawJobTags": [],
"jobTags": null,
"executorType": "COLLECTOR",
"globalMaxRetries": 2,
"needsManualAck": true,
"staticParameters": [],
"archived": false,
"deleteAfterCompletion": false,
"draftRun": false
}' -H "Content-Type:application/json" -H "X-Requested-By:curl" -H "X-SS-REST-CALL:true" -H "X-SS-App-Component-Id: $CRED_ID" -H "X-SS-App-Auth-Token: $CRED_TOKEN" -i
Hello
Step1 : Get the Job Json which will return Job for given Job ID
| GET https://XXX.hub.streamsets.com/jobrunner/rest/v1/job/{jobId} |
Step2: Modify the ‘globalMaxRetries’ parameter to desired number in received json blob from step
Step 3: Call the POST API to Update Job for given Job ID
| POST | https://XXX.hub.streamsets.com/jobrunner/rest/v1/job/{jobId} |
curl -X POST https://XXX.hub.streamsets.com/jobrunner/rest/v1/job/72b8200c-0e3b-426e-b564-bceb17220b1e:8c2c652f-e3d9-11eb-9fb3-b974ac4c3f67 -d '{
"id": "72b8200c-0e3b-426e-b564-bceb17220b1e:8c2c652f-e3d9-11eb-9fb3-b974ac4c3f67",
"name": "Job for Date Conversions",
"description": "sd",
"organization": "8c2c652f-e3d9-11eb-9fb3-b974ac4c3f67",
"creator": "8072136c-e3d9-11eb-9fb3-8929bd105d08@8c2c652f-e3d9-11eb-9fb3-b974ac4c3f67",
"createTime": 1666862894707,
"lastModifiedBy": "8072136c-e3d9-11eb-9fb3-8929bd105d08@8c2c652f-e3d9-11eb-9fb3-b974ac4c3f67",
"lastModifiedOn": 1666862894707,
"destroyer": null,
"deleteTime": 0,
"jobDeleted": false,
"pipelineName": "Date Conversions",
"pipelineId": "589c51ce-3df6-49aa-ab0a-f6860471ed46:8c2c652f-e3d9-11eb-9fb3-b974ac4c3f67",
"pipelineCommitId": "a096d193-bdb1-473b-b10d-4c2ef7863bf8:8c2c652f-e3d9-11eb-9fb3-b974ac4c3f67",
"rulesId": "2dbb7682-ba5e-4576-9deb-26cf6863d8ec:8c2c652f-e3d9-11eb-9fb3-b974ac4c3f67",
"pipelineCommitLabel": "v1",
"labels": [
"aws"
],
"currentJobStatus": {
"id": "5f71cc74-6567-445b-bb35-e7814d53cb3a:8c2c652f-e3d9-11eb-9fb3-b974ac4c3f67",
"jobId": "72b8200c-0e3b-426e-b564-bceb17220b1e:8c2c652f-e3d9-11eb-9fb3-b974ac4c3f67",
"pipelineCommitId": "a096d193-bdb1-473b-b10d-4c2ef7863bf8:8c2c652f-e3d9-11eb-9fb3-b974ac4c3f67",
"runCount": 0,
"user": "8072136c-e3d9-11eb-9fb3-8929bd105d08@8c2c652f-e3d9-11eb-9fb3-b974ac4c3f67",
"startTime": 1667886900675,
"finishTime": 0,
"errorMessage": null,
"color": "GRAY",
"sdcIds": [],
"status": "INACTIVE",
"errorInfos": [],
"warnings": [],
"ackTrackers": [],
"pipelineOffsets": [],
"pipelineStatus": [],
"upgradeStatus": null,
"pipelineCommitLabel": null,
"edge": false,
"inputRecordCount": 0,
"outputRecordCount": 0,
"errorRecordCount": 0,
"lastReportedMetricTime": 0,
"enginePipelineId": null,
"runtimeAttributes": []
},
"currentUpgradeStatus": null,
"parentJobId": null,
"systemJobId": null,
"numInstances": 1,
"migrateOffsets": true,
"statsRefreshInterval": 60000,
"runtimeParameters": "{}",
"edge": false,
"timeSeries": false,
"forceStopTimeout": 120000,
"maxRetriesForFailedJob": -1,
"provenanceMetaData": {},
"jobTemplate": false,
"templateJobId": null,
"cdConfig": null,
"rawJobTags": [],
"jobTags": null,
"executorType": "COLLECTOR",
"globalMaxRetries": 2,
"needsManualAck": true,
"staticParameters": [],
"archived": false,
"deleteAfterCompletion": false,
"draftRun": false
}' -H "Content-Type:application/json" -H "X-Requested-By:curl" -H "X-SS-REST-CALL:true" -H "X-SS-App-Component-Id: $CRED_ID" -H "X-SS-App-Auth-Token: $CRED_TOKEN" -i
And also will it impact any other config as we only and only need to change that configuration ?
You need to pass this json as part of POST API call, so first you get this json and then modify and then update.
It will not impact other parameter or config of the job.
How to pass the modified content in a variable? I have the need to run this as automated with no manual intervention. I tried below two ways but none worked. I dont see any other viable option to pass modified content in the curl command.
var1=`cat /home/script/modified.json`
curl -X POST https://XXX.hub.streamsets.com/jobrunner/rest/v1/job/72b8200c-0e3b-426e-b564-bceb17220b1e:8c2c652f-e3d9-11eb-9fb3-b974ac4c3f67 -d '{ @var1}’ -H "Content-Type:application/json" -H "X-Requested-By:curl" -H "X-SS-REST-CALL:true" -H "X-SS-App-Component-Id: $CRED_ID" -H "X-SS-App-Auth-Token: $CRED_TOKEN" -i
curl -X POST https://XXX.hub.streamsets.com/jobrunner/rest/v1/job/72b8200c-0e3b-426e-b564-bceb17220b1e:8c2c652f-e3d9-11eb-9fb3-b974ac4c3f67 -d '{ /home/script/modified.json}’ -H "Content-Type:application/json" -H "X-Requested-By:curl" -H "X-SS-REST-CALL:true" -H "X-SS-App-Component-Id: $CRED_ID" -H "X-SS-App-Auth-Token: $CRED_TOKEN" -i
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.
