Skip to main content
Question

Need to join datasets using kafka and join them


Hi team,

how to join datasets using kafka and how to join them?

Please help me this one. 

5 replies

Rishi
StreamSets Employee
Forum|alt.badge.img
  • StreamSets Employee
  • 96 replies
  • July 25, 2022

@vasanthi Could you please elaborate your usecase with more details. What do you mean by join dataset using kafka? 


  • Author
  • Fan
  • 4 replies
  • July 25, 2022

I have a requirement in which I have to join two files using kafka


Rishi
StreamSets Employee
Forum|alt.badge.img
  • StreamSets Employee
  • 96 replies
  • July 25, 2022

You can use Directory origin to read files and then use Kafka Producer as destination to write this file.


  • Author
  • Fan
  • 4 replies
  • July 25, 2022

i have take two files in directory origin?

 


Rishi
StreamSets Employee
Forum|alt.badge.img
  • StreamSets Employee
  • 96 replies
  • July 25, 2022

The Directory origin reads data from files in a directory. You can put these two files in a directory and then configure origin to read this. 

ref : https://docs.streamsets.com/portal/platform-datacollector/latest/datacollector/UserGuide/Origins/Directory.html#concept_qcq_54n_jq


Reply