we are ingesting data from oracle to s3 bucket and s3 to data bricks delta table using streamsets.
S3 to delta bricks pipeline runs every 15mins. we observed that at times s3 to delta pipeline is not picking up any records form s3 bucket even if has the records present.
Just wanted to no, what might be the issue?
is their any way to reset origin and start every time on scheduler level as the jobs are scheduled every 15mins?