Issue:
- Job failover is generating multiple subscription events and resulting in duplicate incidents.
Environment:
- SCH OnPrem
- SCH Cloud
Resolution:
- We can achieve this use-case by using "Max Global Failover Retries Exhausted" subscription and enabling the "Global Failover Retries" attribute on the job.
- https://docs.streamsets.com/portal/controlhub/latest/help/controlhub/UserGuide/Subscriptions/Parameters.html#concept_owq_pbm_xpb
- https://docs.streamsets.com/portal/controlhub/latest/help/controlhub/UserGuide/Jobs/PipelineFailover.html#concept_i1l_4vj_jfb
Steps to configure:
Job configuration:
1. Enable failover on the job
2. And set a value for the Global Failover Retries property (value > 0)
3. Save the job.
Subscription Configuration:
1. Go to Subscription UI --> Click + icon
2. In Step One: Define Trigger Event
--> Change the Event --> Max Global Failover Retries Exhausted
--> Select the condition --> JOB_NAME
--> Add the JOB NAME (Ex: XYZ)
3. In Step Two: Define Action, select Email OR Web hook and fill all the required details.
4. Save the configuration.
Description:
- Let’s say "Global Failover Retries" property is configured as 2.
- When the job failover from one SDC to another SDC and reaches the global failover.
- A event will be generated and "Max Global Failover Retries Exhausted" subscription will be triggered.
- This event will be triggered only once. So there won’t be any duplicate incidents in ServiceNow.
Troubleshooting:
Issue-1: Multiple incidents are generated.
- Check which subscription is configured and triggered.
- Generally multiple incidents are generated when the job status change subscription is triggered.
Issue-2: "Max Global Failover Retries Exhausted" subscription is not working
- Check if the "Global Failover Retries" property is configured on job or not.
- Check in the job history whether the job reached global failover.