Solved

how to export pipeline in json format using python sdk

  • 21 March 2023
  • 1 reply
  • 127 views
himanshu1234567
Rank

I am trying to export pipeline in json format using python sdk. I am able to do in zip format but not in json format. I am following below links but export_pipeline functions is not available in new version of stramsets. export_pipelines() is available.

 

https://docs.streamsets.com/sdk/latest/usage/sdc/importing_exporting_pipelines.html#:~:text=In%20order%20to%20export%20a,JSON%20representation%20of%20the%20pipeline. 

icon

Best answer by wilsonshamim 21 March 2023, 21:24

View original

1 reply

wilsonshamim
Rank
Userlevel 2
Badge +1

Hi @himanshu1234567,

 

Currently it exports the pipeline in archive folder only. You can use zipfile module to unzip the exported file

here is sample script

import zipfile

def export_pipelines(local_zipfolder,local_zipfile):

    pipelines = sch.pipelines.get_all(label='export')
    print(pipelines)

    # Export the pipelines from Control Hub
    pipeline_export_data = sch.export_pipelines(pipelines=pipelines)


    # Write the exported pipeline data to a local archive
    with open(local_zipfolder+'/'+local_zipfile, 'wb') as output_file:
        output_file.write(pipeline_export_data)


    with zipfile.ZipFile(local_zipfolder +"/"+ local_zipfile , "r") as zip_ref:
        zip_ref.extractall(local_zipfolder)


export_pipelines('./ExportedObjects','pipelines.zip')

 

Reply


Terms & Conditions