I have a data collector pipeline that runs on data collector version 3.21.x
This pipeline reads data using a JDBC Multi-table Consumer Origin and writes to a Snowflake DB/table.
The pipeline used to run successfully. But in the past two days we encounter the following error,
"errorMessage": "SNOWFLAKE_59 - Could not get a Stage File Writer instance to write the records: 'SNOWFLAKE_65 - File upload to Snowflake failed: net.snowflake.client.jdbc.SnowflakeSQLLoggedException: S3 operation failed: Operation=listObjects, Error type=Client, Error code=AccessDenied, Error message=Access Denied (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied; Request ID: DK041FE70QRW4B85; S3 Extended Request ID: DCSX2ikBaa3k/Vwcsc5+Cbb+pH+7KkLITtJYr+e3ea7cHlOVvVeDG9xhpZtQnbc/6FrAsef4C/w=), Request id=DK041FE70QRW4B85, Extended request id=DCSX2ikBaa3k/Vwcsc5+Cbb+pH+7KkLITtJYr+e3ea7cHlOVvVeDG9xhpZtQnbc/6FrAsef4C/w='",
Note: I am using the Snowflake Internal Stage NOT AWS S3 and hence unsure what permission is required. I’ve already created the Stage object in Snowflake and have loaded the data successfully previously.
Any help would be appreciated.
DEBUG Checking if any active runner is idle
Any clue and what this mean?
Best answer by Srinivasan SankarView original