Skip to main content

I am just starting to use the SDK for python (version 3.12.1). I retrieve all pipelines. But, for each pipeline, I can get only the latest version (even though it may be a draft version).

I’d like to get all versions of a pipeline. This way, I can see which version is being used by a job (or multiple jobs). If I can only see the latest version, it is very likely that I can’t find any related job.

Can I retrieve all versions of a pipeline?

Thanks a lot for your attention!

Hi @jyuan,

You can easily get pipeline version for a job using below sample script.  

 

sdc_url = "https://sdcurl:18630"

#Create Control hub Objects
control_hub = ControlHub(control_hub_url,username=control_hub_user, password=control_hub_pw,)
print(control_hub)
sdc = DataCollector(server_url=sdc_url, control_hub=control_hub)
print(sdc)
jobs =  control_hub.jobs.get_all()
print(jobs)
for job in jobs:
    print(job.pipelineName)
    print(job.pipeline.version)
    print(job.pipeline.name)
    print(job.pipeline)
    print(job.pipeline.commit_id)
    print(job.pipeline.pipeline_id)

 

Hi @jyuan,

You can easily get pipeline version for a job using below sample script.  

 

sdc_url = "https://sdcurl:18630"

#Create Control hub Objects
control_hub = ControlHub(control_hub_url,username=control_hub_user, password=control_hub_pw,)
print(control_hub)
sdc = DataCollector(server_url=sdc_url, control_hub=control_hub)
print(sdc)
jobs =  control_hub.jobs.get_all()
print(jobs)
for job in jobs:
    print(job.pipelineName)
    print(job.pipeline.version)
    print(job.pipeline.name)
    print(job.pipeline)
    print(job.pipeline.commit_id)
    print(job.pipeline.pipeline_id)

 

You are looking from the job side, and it only tells you which version of the pipeline is used by the job. It doesn’t tell what other versions of the pipeline are present in the system.

Thanks a lot for your suggestion! I am still looking for ways to find all versions of each pipeline.

ok here is how you can get all versions using


pipeline = control_hub.pipelines.get(pipeline_id='***')
print(pipeline)
pipeline_commits = pipeline.commits.get_all()
print(pipeline_commits)
for pc in pipeline_commits:
    print(pc.version)

 

Reply


Terms & Conditions