Skip to main content

SDC data does not reach Snowflake due to PIPE_EXECUTION_PAUSED


pedro
StreamSets Employee

When working with Snowflake, if a pipe object has the parameter PIPE_EXECUTION_PAUSED enabled, it will cause data to not get to the Snowflake destination.

 

The Snowflake logs will show the PUT commands coming from SDC but the data will get rejected. This is the expected behavior of Snowflake when having PIPE_EXECUTION_PAUSED enabled. However, to a user this will look as though SDC processed the data, did not report any problem but it never moved the data to Snowflake.

 

If you determine that data is not getting to Snowflake because this parameter is enabled, you simply need to run the following command and Snowflake should start accepting the data:

 

ALTER PIPE <pipe_name> PIPE_EXECUTION_PAUSED=false;

 

0 replies

Be the first to reply!

Reply