How to write to multiple tables in Hive or PostgreSQL Metadata Processor

  • 29 July 2023
  • 0 replies
  • 14 views

 

Issue: You want to write to multiple tables in Hive or PostgreSQL Metadata Processor instead of defining a specific table name.

Solution: When processing data from either of the JDBC origins, you can use a record header attribute for the table name. The JDBC origin writes the originating table name in the jdbc.tables record header attribute. If you want to write records to tables of the same name, you can use ${record:attribute('jdbc.tables')} for the table name property.

Below is an example in Hive Metadata Processor.

Link to the documentation: https://docs.streamsets.com/portal/datacollector/5.3.x/help/datacollector/UserGuide/Solutions/JDBC_DriftSyncSolution.html#concept_s2c_qht_pcb


0 replies

Be the first to reply!

Reply