Question

How to enable the acl for aws s3 destination bucket in streamsets data collector

  • 25 April 2023
  • 1 reply
  • 25 views
Victor01
Rank

Hi Team,

 

I was facing the issue to keep my whole file to s3 destination bucket using private endpoint with acl. I am using streamsets datacollector 5.3 version. The customer s3 destination bucket requires canned acl to put file on that destination bucket path. The required canned acl would be bucket-owner-full-control. 

Can you please help me with detailed steps how to configure acl with s3 private endpoint link on destination bucket. We use Instance Profile without access keys and secret keys.

 

Thanks.

1 reply

saleempothiwala
Rank
Userlevel 4
Badge

@Victor01 

 

Bucket: The name of your S3 bucket
Common Prefix
Object Name Suffix
Data Format: 

Configure the S3 private endpoint link:

To access your S3 bucket through a VPC endpoint, follow these steps:

Go to your AWS Management Console and navigate to the VPC Dashboard.
Create a new VPC Endpoint and choose "S3" as the service. Make sure the VPC endpoint is associated with the correct VPC and subnets.
Modify the route tables of your subnets to include the VPC endpoint.
Next, in the StreamSets Data Collector, go to the "Advanced" tab in the Amazon S3 destination component properties. In the "Endpoint" field, enter the private endpoint link (

Configure the Access Control List (ACL):

Go to the AWS Management Console, navigate to the S3 Dashboard, and select your destination bucket. Click on the "Permissions" tab, then click on "Access Control List". From here, you can add or remove permissions for specific AWS accounts or public access.

Reply


Terms & Conditions