Solved

StreamSets Airflow Integration

  • 10 September 2021
  • 3 replies
  • 347 views

Userlevel 1

Does StreamSets Data Collector and Transformer support integration with Airlfow? Could you please share details on the integration? i.e. Prebuild StreamSets Operator for Airflow

icon

Best answer by Dash 11 September 2021, 19:54

View original

3 replies

Userlevel 6
Badge +3

Hi @srinivasanvar

There is no StreamSets Operator for Airflow. What’s your use case?

Userlevel 1

Dear @Dash

 

We use airflow as the orchestration tool and it needs to integrate with StreamSets Data Collector / Transformer for a set of steps. Wanted to understand approach for integrating airflow with StreamSets.

 

Regards

Varadha

Userlevel 6
Badge +3

Hey @srinivasanvar!

Got it. So, like I mentioned there’s no StreamSets Operator for Airflow, but you might be able to use our REST APIs or if you’re a customer the StreamSets SDK for Python.

Cheers,

Dash

 

Reply