Question

list all streamsets pipelines by rest api

  • 29 September 2022
  • 2 replies
  • 37 views

hi streamsets community,

 

we need to export the listing of all pipelines that exist in streamsets by rest api

 

How can we do that?


2 replies

Userlevel 2
Badge

@Yago Aparecido , have you looked into SDK for Python for this purpose? If not, I would highly recommend it.

If you are on DataOps Platform, https://docs.streamsets.com/platform-sdk/latest/index.html

else if you are on 3.x , 

https://docs.streamsets.com/sdk/latest/index.html

 

In fact, there is SDK tutorial which has similar code to loop over pipelines:

https://github.com/streamsets/tutorials/blob/master/sdk-tutorials/sch/tutorial-pipelines/edit-pipelines-and-stages/README.md#step-3--loop-over-pipelines-to-add-label-to-each

 

and for import/export of pipelines in:

https://github.com/streamsets/tutorials/tree/master/sdk-tutorials/sch/tutorial-pipelines/common-pipeline-methods#step-5--export-and-import-the-pipeline

hi @Kirti 

 

I used the "RESTful API" and apparently it's working for what we need.

 

if we need more details I will try to look at the SDK, thanks for the help.

Reply