I have an usecase where we need to read from the source and write all the records into a text file using streamsets. Is it possible to write all the records as a text file?
Page 1 / 1
The Local FS destination lets you do this. See our docs.
Hey Dima,thanks for the answer.I ’d like know if we can write directly as a text file to NAS location(using SFTP) ?
The SFTP destination only writes directly via the Whole File format. Your best bet would be to write to Local FS and then have a separate process move the complete file via SFTP.
In this case , you can create the first file to read data from source and send to s3 bucket as txt file format.
In the second pipeline , read file from s3 bucket as whole file and send sftp server.
Reply
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.