Question

Need to join datasets using kafka and join them


Hi team,

how to join datasets using kafka and how to join them?

Please help me this one. 


5 replies

Userlevel 4
Badge

The Directory origin reads data from files in a directory. You can put these two files in a directory and then configure origin to read this. 

ref : https://docs.streamsets.com/portal/platform-datacollector/latest/datacollector/UserGuide/Origins/Directory.html#concept_qcq_54n_jq

i have take two files in directory origin?

 

Userlevel 4
Badge

You can use Directory origin to read files and then use Kafka Producer as destination to write this file.

I have a requirement in which I have to join two files using kafka

Userlevel 4
Badge

@vasanthi Could you please elaborate your usecase with more details. What do you mean by join dataset using kafka? 

Reply