Hi team,
Example pipeline
Pipeline 1 -- origin--to--kafka producer
Pipeline 2 -- kafka consumer--to--destination
Need your support for how to create job to merge pipeline1 and pipeline2.
Please let me know the possibility.
- Headliner
- 258 replies
Hi
Pipeline 1 can run on its own and Pipeline 2 can run on its own. Since these are kafka pipelines, you just let them run independently. As soon as the Origin from Pipeline 1 creates a new record, it will generate a message in Kafka Producer. On the other hand, Pipeline 2 will continously look for new messages and as soon as it sees a new message, it will send it to the destination.
You dont have to manually make dependencies.
Reply
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.