Question

How to get the list of pipeline names from control hub using python sdk ?

  • 9 July 2023
  • 2 replies
  • 66 views
HariUmesh
Rank

Hi All,


i am building a unittest framework for validating the pipeline names from the streamsets control hub
For that i need to fetch pipeline names from streamsets using the python platform sdk as seen below


from streamsets.sdk import Controlhub

sch = Controlhub(credential_id="my_id", token ="Mytoken", use_webscoke_tunneling=False)

pipelines = sch.pipeline.get_all

the above code return list of attributes and name of pipeline with the type of class ‘Method’.  
i actually need to get only those pipelines that already deployed into the streamsets through our CI platform
so what would be the right approach to filter and get the pipeline names from control hub?

I don't see any example specific to this scenario

Can you please help on this?

 

2 replies

dima
Rank
Userlevel 3
Badge

Hi Hari,

No need for the `get_all` (which is only intended to filter the `ControlHub.pipelines` by some attribute). You should be able to just get that `ControlHub.pipelines` list and then iterate over it to get out `Pipeline.name`.

HariUmesh
Rank

Thanks Dima.. looks like the sdk takes long time to fetch the pipeline name from streamsets using pipeine.name method. it tooks around more than half minute to get all details
Is there any optimized approach or filter method to get all those pipeline from streamsets?

 

Thanks

Reply


Terms & Conditions