Skip to main content
Question

Need to join datasets using kafka and join them

  • July 25, 2022
  • 5 replies
  • 41 views

Hi team,

how to join datasets using kafka and how to join them?

Please help me this one. 

5 replies

Rishi
StreamSets Employee
Forum|alt.badge.img
  • StreamSets Employee
  • 96 replies
  • July 25, 2022

@vasanthi Could you please elaborate your usecase with more details. What do you mean by join dataset using kafka? 


  • Author
  • Fan
  • 4 replies
  • July 25, 2022

I have a requirement in which I have to join two files using kafka


Rishi
StreamSets Employee
Forum|alt.badge.img
  • StreamSets Employee
  • 96 replies
  • July 25, 2022

You can use Directory origin to read files and then use Kafka Producer as destination to write this file.


  • Author
  • Fan
  • 4 replies
  • July 25, 2022

i have take two files in directory origin?

 


Rishi
StreamSets Employee
Forum|alt.badge.img
  • StreamSets Employee
  • 96 replies
  • July 25, 2022

The Directory origin reads data from files in a directory. You can put these two files in a directory and then configure origin to read this. 

ref : https://docs.streamsets.com/portal/platform-datacollector/latest/datacollector/UserGuide/Origins/Directory.html#concept_qcq_54n_jq