Skip to main content

How to join 2 files using Kafka ?

@truder66.again Could you please elaborate this with more details, What do you mean by join 2 files ?


Taking 2 csv files and have to join using kafka.

 


I did not fully understand what exactly you mean by joining 2 file using Kafka ..can you share an example. ?

However, if you meant reading this 2 file and put this data to  kafka topic then this can be achieved 

Directory Origin -→ Kafka Producer Destination

You can read this 2 csv file via Directory origin and then dump this data to single kakfa topic.

 


how can we read csv from our desktop-->Download (path), actually im stuck at this step.

and im using strigo as my terminal

 

 


In case of Strigo, your data collector runs as a docker on ‘StreamSets Cluster’. You will have to copy the file onto your sdc docker or map your local linux drive as a volume on the docker, only then you will be able to read the file using Directory origin.

 

 

You can access your sdc file system using

sudo docker exec -it <sdc container name> bash


Reply