Question

is there a way to make the sql server cdc client to pull changes from source at less frequency?

  • 8 January 2023
  • 1 reply
  • 29 views

We have huge amount of load at sql server db side due to streamsets and we want to reduce the same. We are using Streamsets sql server cdc client to pull data from 500 sql server tables to Snowflake. Can you please help us below questions:

  1. Is there a way to reduce frequency with which stream sets cdc client connect to sql server? We have used queries per second parameter as 1 to reduce the number of queries running at source. please let me know if we can do anything?
  2. And we see huge number of select max($__$seqval) running. Is this expected? Thanks!

1 reply

Userlevel 2
Badge

Queries per Second is a correct way to to control throughput. You can use also positive values smaller then 1, like 0.1. The queries for seqval are necessary, yes.

Reply