Question

JDBC Query Consumer not populating jdbc.tables record header

  • 20 September 2022
  • 7 replies
  • 34 views

Hi! I have a pipeline that uses a JDBC multitable consumer origin and that populates the jdbc.tables header attribute, but when I change out the origin to a JDBC query consumer it is not longer populated. Any advice on why this might be happening?

 


7 replies

Userlevel 2
Badge

Hi @annbandy,

In JDBC Query Consumer, you have to check the "Create JDBC Header Attributes" configuration in order to get that produced.

Thanks

HI @alex.sanchez

Create JDBC Header Attributes is set to true! They’re all populated except for jdbc.tables.

Userlevel 2
Badge

So, we are getting the table names from the resultSet metadata, depends on your query, that can actually be empty, and I feel this is the case.

If you try with a simpler query: “select *  from table” it should work straight, but depending on the query it can be empty on the results sets, and we might not be able to read it.

As a workaround, given that you know the query in advance, you can also set it using some field processor.

I am using a simple select * from already. However I am using parameter to specify the schema and table. So the query looks like select * from ${schema}.${table}. Do you think this could be cause?

Userlevel 2
Badge

Which SDC version are you using? JDBC Multitable had that option long time back, but only newer versions from JDBC Query Consumer are populating those headers

I am using 4.1.0

Userlevel 2
Badge

Hi @annbandy,

 

I’ve asked someone in the Engineering team to take a deeper look and seems that you were always correct, we will investigate further and try to get that fixed on a future version, please keep an eye on the release notes, and thank you for reporting it

Reply