Hello Everyone
I am facing following issue. Any inputs will be helpful.
- We are using Streamsets to load data into Snowflake.
- Our Snowflake instance is hosted in the region [EU-WEST-1] and S3 (for Stage) is in [EU-WEST-2]
- We configured Snowflake connector [Staging] in the StreamSets to use S3 region same as Snowflake. With this setting in place, we see following errors getting logged in S3 bucket (which is meant to capture the errors):
"errorMessage": "SNOWFLAKE_59 - Could not get a Stage File Writer instance to write the records: 'The bucket is in this region: eu-west-2. Please use this region to retry the request",
- If we change S3 Region for the Snowflake connector in the Streamsets to [EU-WEST-2], we see following error in the Streamsets UI.
SNOWFLAKE_19 - Could not access the S3 bucket associated with the stage: com.amazonaws.SdkClientException: Unable to execute HTTP request: s3.eu-west-2.aws.amazonaws.com: Name or service not known
Streamset documentation says following:
“To stage data in an Amazon S3 external stage, create a Snowflake external stage in a bucket in the same S3 region that hosts your Snowflake virtual warehouse. For example, if your Snowflake warehouse is in AWS US West, then create the Snowflake external stage in a bucket in the AWS US West region.”
Does this mean there is a limitation on Streamsets side though Snowflake does not have such limitation? If yes, then is there a work-around for this please?
Thank you.