Skip to main content
Question

Join two files

  • July 8, 2022
  • 1 reply
  • 89 views
R
Fan

Hi,

I have a requirement in which I have to join two files(from two different API end points) using kafka and Groovy scripts. Can someone help me with this?

    1 reply

    saleempothiwala
    Headliner
    Forum|alt.badge.img

    Hi @Rohith 

    SDC supports only 1 origin and also data is extracted in batches. You cannot join multiple datasets using data collector. You will have to use StreamSets Transformer to join data. In order to join files, you will have to use SDC to bring these files to some landing area like an S3 bucket and then join using Transformer. Share more details if you need help.

    Reply


    Terms & Conditions