Skip to main content
Solved

HTTP Client Loops Unceasingly

  • May 6, 2025
  • 1 reply
  • 25 views

Sperch
Opening Band
Forum|alt.badge.img

We have an HTTP Client origin that pulls from our ServiceNow site records that would be returned if we ran a report in ServiceNow directly. The records pulled appear at first blush to all be accounted for, but unfortunately the origin just keeps looping and pulling the every record over and over. I reduced the batch size to as low as 100 records without effect (i.e., it still loops endlessly).

The URL with parameters is:

https://<intentionally_redacted>.servicenowservices.com/api/now/table/task?sysparm_query=closed_atISNOTEMPTY%5Eclosed_at%3E%3Djavascript%3Ags.dateGenerate('2023-01-01'%2C'00%3A00%3A00')%5Eclosed_by.department.nameSTARTSWITHPBO%20-%20ETS%5Eassignment_group!%3Daeb1d3bc3772310057c29da543990ea2%5Eassignment_group!%3D4660e3fc3772310057c29da543990e0b%5EnumberNOT%20LIKEGAPRV%5EnumberNOT%20LIKERCC%5Esysparm_display_value=true%5Esysparm_limit=100%5Esysparm_offset=0

I have the stage set to pull in Batch mode, and for pagination I have tried all 5 modes including “None”. Since it doesn’t stop until manually stopped I am unable to get a snapshot and no errors are being logged. Without an event generation option I am not sure how to resolve this issue.

Note: I reduced the original record count from 23k+ records to 9k+ records due to restrictions prohibiting me from getting a timeout increase on the SN side improved, so I have addressed the connection timeout angle since those 9k+ records return once in Postman.

If anyone has any ideas please let me know.

Best Regards,

Sean Perch

Best answer by Sperch

The root cause was in formatting of the XML body being sent. What worked in Postman needed slight tweaking to work as desired in StreamSets.

1 reply

Sperch
Opening Band
Forum|alt.badge.img
  • Author
  • Opening Band
  • Answer
  • October 31, 2025

The root cause was in formatting of the XML body being sent. What worked in Postman needed slight tweaking to work as desired in StreamSets.