Question

Unable to run snowflake multiple queries in Parallel using Snowflake Executor

  • 22 July 2022
  • 3 replies
  • 135 views

Hi All,

 

I am trying to run multiple queries in parallel using snowflake executor. I have set the property to run all the queries in parallel. but still on snowflake it is running one at a time.

 

also increased connection pool size to 10.

 

Thanks,

Anupam


3 replies

Userlevel 2
Badge

Hi @Anupam Kushwah,

You are using the JDBC Executor to execute those, which is not supported, we release the Snowflake Executor which takes care of some specifics of Snowflake.


Sadly, it is currently disabled since it is not possible to submit multiple queries to Snowflake in a performant way and keeping track of them as we need.

Thanks

Thanks Alex, is there any way we can run snowflake queries in parallel?

Userlevel 2
Badge

If you are using a multithreaded origin, each thread will create instances of the rest of the stages, so you will virtually have multiple instances of the Snowflake Executor

Reply