Skip to main content

im using oracle as a source and s3 as destination.

im ingesting the records from the source and adding table name as a column through expression evalutaor. i want to use this table name and create a folder in s3 dynamically before dropping those records in s3 bucket

so the folder name should be created dynamically through streamsets fetching the name of the table. what should be the approach for the same?

for ex: if I'm fetching records from table abc , I need to create a folder called “abc” and drop all the records inside that folder.

If I’m not mistaken, S3 doesn't have a separate abstractions for folders; the name of an object delimited by slashes is what's interpreted as a folder hierarchy. You can set this name by setting the "Common prefix” or the “Object name prefix,” as appropriate for your use case. 

Reply


Terms & Conditions