Generating blank output files

  • 12 September 2023
  • 3 replies

I have a Kafka topic as a source and the sink as a delimited files. I want to generate a blank output file if there is no data in the topic. What is the best way to achieve this. 


3 replies

Userlevel 5
Badge +1


Kindly check if the topic is having record in it or not.

If record != null , then create a file having data in it.

If record is null then configure below code in jython and save the file as txt file.



Thanks Bikram for the quick reply. I will give that a try. I have another scenario. What happens if the Kafka topic goes through a complex ingestion and generate more than 100 tables where few tables would be empty. What would be the best option in this scenario to generate empty files. 



Userlevel 5
Badge +1


In that case you can create a metadata table which will contains the table names in it.

By using the JDBC consumer , read data from meta data table which contains only table names in it .


for example , the metadata table having tables names in it. Once tables name fetched from the metadata table , then use jdbc lookup processor to check number of records in the table.


Attached the pipeline for your reference .