Configuring AWS S3 Bucket

@pkandra , would you please let us know:

1. what error/s are you seeing in the stage error?
2. The link to the documentation you are referring to please?

1: I’m new to StreamSets, so I’m unsure of how to see the exact error but have attached a screenshot.

2:  

Here’s the documentation I’m referring to, there’s no reference of what values to use for the bucket...is it the URL, the URI or something else?

Portal - StreamSets Docs

@pkandra ,

1. For the bucket: here is the documentation https://docs.streamsets.com/portal/platform-datacollector/latest/datacollector/UserGuide/Destinations/AmazonS3.html#concept_bnp_gwp_f1b
2. Not sure where you are seeing the path (in s3 destination -stage configs)
3. Related to stage errors: I am not sure how you have configured error-handling: depending on that you need to see errors: Here is documentation on the same https://docs.streamsets.com/portal/platform-datacollector/latest/datacollector/UserGuide/Pipeline_Design/ErrorHandling.html#concept_pm4_txm_vq

Thanks Kirti, but that documentation doesn’t tell me if I should use the bucket URL, URI or something else.  I’ve tried every combination and it doesn’t seem to be working.

@pkandra ,

you can put the name of the bucket e.g. streamsets-bucket

Have you tried that? If yes, what is happening?

@pkandra 

Yes , you need to provide the bucket name and prefix and postfix details.

If you want the prefix is csv then dataformat should be csv , if json then data format format should be json etc.

Attached screenshot for you reference.