Let us say we have a dataset like below. Requirement is to use JDBC Query Consumer Origin to load id, title, ratings based on whatever is the latest from created_date or updated_date. It means we want to do incremental load but not on primary key ‘id’.
![](https://uploads-us-west-2.insided.com/streamsets-en/attachment/d0c9b448-d3e6-488f-bda2-dc9d0a9b550c.png)
Whenever the pipeline runs it should pull in the updated ratings with other fields. In order to incrementally load data based on our requirement we have written query like below.
![](https://uploads-us-west-2.insided.com/streamsets-en/attachment/ba871c7f-d4d1-4b7f-9110-a1de8813f722.png)
Query is failing by saying Invalid column name ‘offset_date’ though it is valid per SQL when we do validation.
Best answer by Pradeep
View original