Skip to main content

I am trying to get all pipeline name with below code statement

 

from streamsets.sdk import Controlhub

sch = Controlhub(credential_id="my_id", token ="Mytoken", use_webscoke_tunneling=False)

pipelines = sch.pipeline.name

filterd_name =  name for name in pipelines if name.startswith('abc')]

 

i observed its taking more than an minute to fetch all pipeline names in to the list

 

Is the righ approach to filter pipeline names from control hub or is there any issue on streamset sdk?

 

 

Hey Hari,

Pipeline names are neither unique nor indexed, so `ControlHub.pipelines.get(name=<name>)` ends up being an O(n) operation. You'll get much better performance if you query by pipeline ID, for example, by passing `pipeline_id` into your `get`. 

Reply


Terms & Conditions