Question

Transfer of data

  • 5 November 2021
  • 4 replies
  • 72 views
Diana Lacatus
Rank

Hi everyone,

I would like to know if it is possible to introduce a delay between the transfer of the records, in order to not send the whole information at once. I want to transfer the data record by record.

Could you help me, please?

4 replies

alex.sanchez
Rank
Userlevel 2
Badge

Hi @Diana Lacatus 
You can change the batch size to 1 which will make all the batches of size 1. 

Apart from that we have a delay processor that can introduce any delay between batches. I think that this might help you

Alex - StreamSets Engineering Manager @ Collector
dima
Rank
Userlevel 3
Badge

Along with changing batch sizes at the origin, as @alex.sanchez pointed out, you can also set a rate limit at the pipeline level. These two in combination can probably help you get what you're looking for, @Diana Lacatus. 

Sanjeev
Rank
Userlevel 2
Badge

and just in case you need to add a delay in passing the records to the target or next stage, you can use the Delay processor. 

Drew Kreiger
Rank
Userlevel 5
Badge

@Diana Lacatus did the responses above help? Please select a “best answer” for others to see what helped with your specific question. :)  

Reply


Terms & Conditions