Skip to main content

Hello, 

 

I am creating the most simplest pipeline:  a query (JDBC Query Consumer) whose output is written to files - Local FS.  I have broken the files to 200k records each but this job never stops running and keeps writing file after file of what I assume is duplicate data. 

I cannot for the life of me figure out what I did wrong.  Any insight would be greatly appreciated!

I have created complex pipelines in the past, so this is strange.

 

 

@lizzie 

can you please give a try with below configuration mentioned in the below snap.

Add the event as pipeline finisher with condition as ${record:eventType() == 'no-more-data'} in it and also uncheck the incremental mode.

 

Please let me know if it helps.

 

Reply


Terms & Conditions