Question

Kafka pipeline

  • 28 September 2022
  • 2 replies
  • 31 views
P
Rank

Hi Team

Can someone please suggest me a way to stop kafka topic using a jython or groovy-script

2 replies

P
Rank

I mean pipeline consists of kafka-multi-topic as origin, It’s running infinitely.

saleempothiwala
Rank
Userlevel 4
Badge

@Priyanka Mynepally 

Kafka origin is a streaming type of origin and does not have events like no-more-data. 

If you want to stop the pipeline on purpose then you can use

  1. Schedular Task: you can create a schedule to stop the job at a given time on schedule
  2. Use Orchestrator processor for calling StreamSets API to stop the job on some action
  3. Use an external system like Airflow, Control M, etc to call StreamSets API to stop the job.

Reply


Terms & Conditions