Question

Join two files

  • 8 July 2022
  • 1 reply
  • 74 views
R
Rank

Hi,

I have a requirement in which I have to join two files(from two different API end points) using kafka and Groovy scripts. Can someone help me with this?

1 reply

saleempothiwala
Rank
Userlevel 4
Badge

Hi @Rohith 

SDC supports only 1 origin and also data is extracted in batches. You cannot join multiple datasets using data collector. You will have to use StreamSets Transformer to join data. In order to join files, you will have to use SDC to bring these files to some landing area like an S3 bucket and then join using Transformer. Share more details if you need help.

Reply


Terms & Conditions