Skip to main content
Question

Joining 2 files

  • July 18, 2022
  • 3 replies
  • 46 views

How to join 2 files via javascript evaluator (or) groovy (or) jython.

Can someone show simple usecase on this

3 replies

dima
StreamSets Employee
Forum|alt.badge.img
  • StreamSets Employee
  • 83 replies
  • July 18, 2022

Can you describe the complete use case? What specific business problem are you trying to solve? Data Collector is a record-based data ingestion engine so file joining is not a typical pattern.


  • Author
  • Fan
  • 4 replies
  • July 18, 2022

Given file1.txt and file2.txt with some content each.

Need to append content of these files in file3.txt

Is it possibe in streamsets using Kafka/customscripting approach


dima
StreamSets Employee
Forum|alt.badge.img
  • StreamSets Employee
  • 83 replies
  • July 18, 2022

Seems like you could simply use a Directory origin to read the folder containing the two files and parse their contents and then write to the Local FS destination. Are you simply concatenating the files without modification or are you trying to enrich data or something else along the way?