I am trying to load data from Teradata to Snowflake. I am using Transformer for spark as engine.
Origin:
JDBC query consumer.
Destination:
Snowflake
I am able to load 12 million data in 1 hour.
but I want to improve the performance.
Below is the configuration of my pipeline.
+1
can you please try to enable the cache data and also try the option to enable the lubricrous mode as well.
Generally cache data enabling increase the performance .
Enter your username or e-mail address. We'll send you an e-mail with instructions to reset your password.
