Skip to main content
Question

Kafka pipeline

  • September 28, 2022
  • 2 replies
  • 37 views

Hi Team

Can someone please suggest me a way to stop kafka topic using a jython or groovy-script

2 replies

  • Author
  • Discovered Fame
  • 16 replies
  • September 28, 2022

I mean pipeline consists of kafka-multi-topic as origin, It’s running infinitely.


saleempothiwala
Headliner
Forum|alt.badge.img
  • Headliner
  • 258 replies
  • September 29, 2022

@Priyanka Mynepally 

Kafka origin is a streaming type of origin and does not have events like no-more-data. 

If you want to stop the pipeline on purpose then you can use

  1. Schedular Task: you can create a schedule to stop the job at a given time on schedule
  2. Use Orchestrator processor for calling StreamSets API to stop the job on some action
  3. Use an external system like Airflow, Control M, etc to call StreamSets API to stop the job.