Question

Joining 2 files


How to join 2 files via javascript evaluator (or) groovy (or) jython.

Can someone show simple usecase on this


3 replies

Userlevel 3
Badge

Can you describe the complete use case? What specific business problem are you trying to solve? Data Collector is a record-based data ingestion engine so file joining is not a typical pattern.

Given file1.txt and file2.txt with some content each.

Need to append content of these files in file3.txt

Is it possibe in streamsets using Kafka/customscripting approach

Userlevel 3
Badge

Seems like you could simply use a Directory origin to read the folder containing the two files and parse their contents and then write to the Local FS destination. Are you simply concatenating the files without modification or are you trying to enrich data or something else along the way?

Reply