I have an http client origin which gives a json response. Pipeline uses pagination (by page number). When I am enabling ‘Keep all fields’ option in http client it is creates the duplicates of first record in every page. Say I have 10 records in the file, it writes first record 10 times in the same, 1st record in the second page 10 times and so on. basically it repeats first record for the entire page. Any way to fix this issue? We need to ‘Keep all fields’ enabled to get page properties for processing within the job.
Page 1 / 1
Hello
can you please try to select the option as “All values as a list” in HTTP origin , i believe the current select is “Split into multiple records” in HTTP origin , that’s the reason you are getting duplicate entries in pagination.
Reply
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.