Skip to main content
Solved

StreamSets Airflow Integration

  • September 10, 2021
  • 3 replies
  • 355 views
S
Fan

Does StreamSets Data Collector and Transformer support integration with Airlfow? Could you please share details on the integration? i.e. Prebuild StreamSets Operator for Airflow

Best answer by Dash

Hey @srinivasanvar!

Got it. So, like I mentioned there’s no StreamSets Operator for Airflow, but you might be able to use our REST APIs or if you’re a customer the StreamSets SDK for Python.

Cheers,

Dash

 

View original
Did this topic help you find an answer to your question?

3 replies

Dash
Headliner
Forum|alt.badge.img+3
  • DashHeadliner
  • Senior Technical Evangelist and Developer Advocate at Snowflake
  • 67 replies
  • September 10, 2021

Hi @srinivasanvar! 

There is no StreamSets Operator for Airflow. What’s your use case?

Dash | https://twitter.com/iamontheinet
S
Fan

Dear @Dash

 

We use airflow as the orchestration tool and it needs to integrate with StreamSets Data Collector / Transformer for a set of steps. Wanted to understand approach for integrating airflow with StreamSets.

 

Regards

Varadha

Dash
Headliner
Forum|alt.badge.img+3
  • DashHeadliner
  • Senior Technical Evangelist and Developer Advocate at Snowflake
  • 67 replies
  • Answer
  • September 11, 2021

Hey @srinivasanvar!

Got it. So, like I mentioned there’s no StreamSets Operator for Airflow, but you might be able to use our REST APIs or if you’re a customer the StreamSets SDK for Python.

Cheers,

Dash

 

Dash | https://twitter.com/iamontheinet

Reply


Terms & Conditions