Hi Friends,
I am new to StreamSets and trying to build a pipeline to handle event data coming from Google Cloud pub/sub to Google Cloud BigQuery. I am stuck with a problem and need your help.
Whenever an event enters the pipeline, I will have to check in the Bigquery tables whether the event already exists in the tables. We have a unique id in the event against which i can query the BigQuery table to check and further continue in the pipeline. I tried various options(including jdbc lookup) but no luck.
Can you please help?
Thanks in Advance.
Regards,
Ravi