Hi Friends,

I am new to StreamSets and trying to build a pipeline to handle event data coming from Google Cloud pub/sub to Google Cloud BigQuery. I am stuck with a problem and need your help.

Whenever an event enters the pipeline, I will have to check in the Bigquery tables whether the event already exists in the tables. We have a unique id in the event against which i can query the BigQuery table to check and further continue in the pipeline. I tried various options(including jdbc lookup) but no luck.

Can you please help?

Thanks in Advance.

Regards,

Ravi