Question

Amazon SQS Consumer

  • 8 February 2023
  • 2 replies
  • 43 views

Hello,

I am trying to setup an amazon SQS consumer stage on streamsets. I only want it to receive messages no need to write. Currently the IAM user has these permissions : RecieveMessage, ListQueues, GetQueueAttributes. Are there any specifc permissions that streamsets requires? I attached a picture of what the error looks like.


2 replies

Userlevel 4
Badge

@EIS-EdgarP 

The permission needed depends on other configurations too like DLQ, delete message after reading etc. I would recommend going with more liberal permission to start with and then start taking it away, one step at a time until you reach your optimal access.

Userlevel 5
Badge +1

@EIS-EdgarP 

I believe you need to whitelist StreamSets ARN  in your SQS policy.

 

Currently I am consuming SQS messages on the event in S3 bucket(i.e when there are file delivery in s3) ,it will trigger the message .

Please have a look into the below policy and do necessary changes in your policy to consume messages from SQS in StreamSets.

The policy looks like below .

{
  "Version": "2008-10-17",
  "Id": "__default_policy_ID",
  "Statement": [
    {
      
      "Effect": "Allow",
      "Principal": {
        "Service": "s3.amazonaws.com",
        "AWS": "streamsetsARN"
      },
      "Action": "SQS:*",
      "Resource": " SQS Pipeline ARN",
      "Condition": {
        "ArnLike": {
          "aws:SourceArn": "AWS S3 bucket ARN*"
        }
      }
    }
  ]
}

 

Thanks & Regards

Bikram_

Reply