I am trying to build CDC pipeline to migrate MySQL database to MySQL database in different server.
Here’s the data collector pipeline I’ve created.
As per the documentation here, JDBC producers should be able to process binlog data. I’ve used Field remover to only use /Data and /Table fields.
When I run the pipeline I am getting error that input record has no data for <schema>.<table> . How can I create a pipeline to consume binlog records?
+1
Kindly just preview the pipeline and see if you manage to fetch CDC data from binary log.
In place of JDBC producer , please add jdbc tee processor to handle the CDC data.
Please find below the document for your reference and i hope it will help you.
Hi
I used the JDBC tee as you suggested and also tested om preview mode. The data is coming as a record from binlog as expected. But I’m still seeing an error in JDBC tee during processing mode (which was same as without JDBC TEE). : Record1-Error JDBC_90 - Record doesn't have any columns for table
+1
When you are using the jdbc producer or jdbc tee , please make sure the column names exactly match with the source column name else it will throw error.
Kindly check on it .
Hi
My objective is to migrate all tables from one database, is there a way to dynamically map this in jdbc tee?
This is my setup
And since I have flattened the record fields, how can I use it in column mapping?
Below is the input for JDBC tee, please see these columns will change based on table so how should I map it? Also see, that is homogeneous migraiton the destination column names are same as source.
+1
we have option for column mapping . May i know if you try this .
Enter your username or e-mail address. We'll send you an e-mail with instructions to reset your password.
