Skip to main content

Hi Streamsets Community,

 

just started out and tried the JDBC Query Consumer in DataCollector. My query executes correctly (offset etc.) but I is continuously executed regardless what I put into the query interval. I tried different values like ${5 * MINUTES} ${5 * HOURS} and also just plain integers like 10000. Any Idea what I’m doing wrong?

 

Thanks Marco

Hi Marco,

 

When you say it is continuously executed, what do you mean (and how did you determine that)? Also, can you share some details about your pipeline and what environments it’s connecting to?


I execute the pipeline in a draft run and I see Processing Timer refresh all the time with now records fetched (correct due to offset). I connect to a Postgres demo db and write to a Kafka producer. Environment is self managed docker.

 


This was my misinterpretation, I saw the yellow circle updating all the time in monitoring mode and thought that this was an execution.


Reply