Can we configure multiple tables in same pipeline when we use JDBC Query consumer 1 as origin and JDBC producer 1 as destination. in my case i need to send 20 tables data to send from on-premise DB (db2) to cloud DB (Azure postgresql)?. i am thinking to avoid multiple pipelines creation in streamsets.
Userlevel 5
+1
- Headliner
- 481 replies
-
21 October 2023
StreamSets Data Collector provides the flexibility to employ its JDBC Multi-Table Consumer, allowing users to set up configurations for fetching data from multiple tables and then transferring it to their desired destination tables.
When you need to execute a query, it's more suitable to employ multiple pipelines, whereas within a transformer, it can be accomplished more easily. Within StreamSets Transformer, it enables the retrieval of data from multiple origins and facilitates the forwarding of this data to destination processors.
Reply
Enter your username or e-mail address. We'll send you an e-mail with instructions to reset your password.