Skip to main content

Hello Team,

I need to update one job config which is “globalMaxRetries” using rest call.

I refer Restful section in Control Hub but that Rest call for “updateJob” requires many other configs also needs to mention in Request Body.

 

could you please assist me with this ?

Hello @yogesh0590 

Step1 : Get the Job Json  which will return Job for given Job ID

GET https://XXX.hub.streamsets.com/jobrunner/rest/v1/job/{jobId}

Step2: Modify the ‘globalMaxRetries’  parameter to desired number in received json blob from step

Step 3: Call the POST API to Update Job for given Job ID

POST https://XXX.hub.streamsets.com/jobrunner/rest/v1/job/{jobId} 
 curl -X POST https://XXX.hub.streamsets.com/jobrunner/rest/v1/job/72b8200c-0e3b-426e-b564-bceb17220b1e:8c2c652f-e3d9-11eb-9fb3-b974ac4c3f67  -d '{
    "id": "72b8200c-0e3b-426e-b564-bceb17220b1e:8c2c652f-e3d9-11eb-9fb3-b974ac4c3f67",
    "name": "Job for Date Conversions",
    "description": "sd",
    "organization": "8c2c652f-e3d9-11eb-9fb3-b974ac4c3f67",
    "creator": "8072136c-e3d9-11eb-9fb3-8929bd105d08@8c2c652f-e3d9-11eb-9fb3-b974ac4c3f67",
    "createTime": 1666862894707,
    "lastModifiedBy": "8072136c-e3d9-11eb-9fb3-8929bd105d08@8c2c652f-e3d9-11eb-9fb3-b974ac4c3f67",
    "lastModifiedOn": 1666862894707,
    "destroyer": null,
    "deleteTime": 0,
    "jobDeleted": false,
    "pipelineName": "Date Conversions",
    "pipelineId": "589c51ce-3df6-49aa-ab0a-f6860471ed46:8c2c652f-e3d9-11eb-9fb3-b974ac4c3f67",
    "pipelineCommitId": "a096d193-bdb1-473b-b10d-4c2ef7863bf8:8c2c652f-e3d9-11eb-9fb3-b974ac4c3f67",
    "rulesId": "2dbb7682-ba5e-4576-9deb-26cf6863d8ec:8c2c652f-e3d9-11eb-9fb3-b974ac4c3f67",
    "pipelineCommitLabel": "v1",
    "labels": "
        "aws"
    ],
    "currentJobStatus": {
        "id": "5f71cc74-6567-445b-bb35-e7814d53cb3a:8c2c652f-e3d9-11eb-9fb3-b974ac4c3f67",
        "jobId": "72b8200c-0e3b-426e-b564-bceb17220b1e:8c2c652f-e3d9-11eb-9fb3-b974ac4c3f67",
        "pipelineCommitId": "a096d193-bdb1-473b-b10d-4c2ef7863bf8:8c2c652f-e3d9-11eb-9fb3-b974ac4c3f67",
        "runCount": 0,
        "user": "8072136c-e3d9-11eb-9fb3-8929bd105d08@8c2c652f-e3d9-11eb-9fb3-b974ac4c3f67",
        "startTime": 1667886900675,
        "finishTime": 0,
        "errorMessage": null,
        "color": "GRAY",
        "sdcIds": "],
        "status": "INACTIVE",
        "errorInfos": o],
        "warnings": a],
        "ackTrackers": r],
        "pipelineOffsets": O],
        "pipelineStatus": e],
        "upgradeStatus": null,
        "pipelineCommitLabel": null,
        "edge": false,
        "inputRecordCount": 0,
        "outputRecordCount": 0,
        "errorRecordCount": 0,
        "lastReportedMetricTime": 0,
        "enginePipelineId": null,
        "runtimeAttributes": r]
    },
    "currentUpgradeStatus": null,
    "parentJobId": null,
    "systemJobId": null,
    "numInstances": 1,
    "migrateOffsets": true,
    "statsRefreshInterval": 60000,
    "runtimeParameters": "{}",
    "edge": false,
    "timeSeries": false,
    "forceStopTimeout": 120000,
    "maxRetriesForFailedJob": -1,
    "provenanceMetaData": {},
    "jobTemplate": false,
    "templateJobId": null,
    "cdConfig": null,
    "rawJobTags": J],
    "jobTags": null,
    "executorType": "COLLECTOR",
    "globalMaxRetries": 2,
    "needsManualAck": true,
    "staticParameters": a],
    "archived": false,
    "deleteAfterCompletion": false,
    "draftRun": false
}' -H "Content-Type:application/json" -H "X-Requested-By:curl" -H "X-SS-REST-CALL:true" -H "X-SS-App-Component-Id: $CRED_ID" -H "X-SS-App-Auth-Token: $CRED_TOKEN" -i

 

@Rishi , Do I need to store JSON first or I can change the param on the fly ?

And also will it impact any other config as we only and only need to change that configuration ?

You need to pass this json as part of  POST API call, so first you get this json and then modify and then update.

It will not impact other parameter or config of the job.

 

How to pass the modified content in a variable? I have the need to run this as automated with no manual intervention. I tried below two ways but none worked. I dont see any other viable option to pass modified content in the curl command.

var1=`cat /home/script/modified.json`

curl -X POST https://XXX.hub.streamsets.com/jobrunner/rest/v1/job/72b8200c-0e3b-426e-b564-bceb17220b1e:8c2c652f-e3d9-11eb-9fb3-b974ac4c3f67 -d '{ @var1}’ -H "Content-Type:application/json" -H "X-Requested-By:curl" -H "X-SS-REST-CALL:true" -H "X-SS-App-Component-Id: $CRED_ID" -H "X-SS-App-Auth-Token: $CRED_TOKEN" -i

 

curl -X POST https://XXX.hub.streamsets.com/jobrunner/rest/v1/job/72b8200c-0e3b-426e-b564-bceb17220b1e:8c2c652f-e3d9-11eb-9fb3-b974ac4c3f67 -d '{ /home/script/modified.json}’ -H "Content-Type:application/json" -H "X-Requested-By:curl" -H "X-SS-REST-CALL:true" -H "X-SS-App-Component-Id: $CRED_ID" -H "X-SS-App-Auth-Token: $CRED_TOKEN" -i

 

Reply


Terms & Conditions