Skip to main content

The origin has a parameter ‘since’ that is a UNIX timestamp, which tells the API to give all records since the given timestamp. Looks something like https://……./v1/ssns?since=1667257200000. I would like this value to be dynamic. For example, if I set my pipeline to run once each day I would like the timestamp to give me only records from the last 24 hours.

I’ve tried variations of like ?since={time:dateTimeToMilliseconds(time:now())-86400000}, but I can’t find the right syntax. Is it possible to put this kind of statement in the resource URL that returns a unix timestamp integer? Or if not, any other way to do this?

@HåkonD 

 

You can use a simple Raw Data Source as origin to generate a single record. This can be your starting point. Next, add an Expression Evaluator to create a field with your date-time calculation. Let us call it calc_since. and also create a new field called http_url that creates the whole url for you as a field.

You can then add an HTTP client processor and pass ${record:value(‘/http_url’)} as the value for URL. This will pass on the URL. Also preview will give you a clear view of how your URL looks like.

Hope it helps.

Reply


Terms & Conditions