Want a parquet file format in Local FS while reading from Kafka.
I tried whole file transformer but its not working as it need whole file but my data format in Kafka is SDC records.
I tried it to read into AVRO format then tried converting into Parque, This logic worked but it required 2 different pipelines.
Pipeline 1 : Kafka to Avro.
Pipeline 2: Avro to Parque using whole file transformer.
Is there is any way where I can achieve this in single pipeline.
Thanks in Advance.