Question

Cannot Connect StreamSets with Python SDK, but Can connect via CURL command with same Credential ID and Token

  • 23 April 2024
  • 1 reply
  • 15 views
D
Rank

Hi Team,

I generate Credential ID and Token via StreamSets UI → Manage → API Credential, and with the generated ID and Token I can run Curl command, returned status is “HTTP/1.1 200 OK”, also return me a json format that show my organization id, email, extra..

 

However, When I want to connect with below code snippet (using same credential ID and Token), it returns me error of 403. paste code and error below:

>>> from streamsets.sdk import ControlHub
>>> sch = ControlHub(credential_id='absd_myid', token="abcd_mytoken.")
 

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/module/sfsdcvenv/lib/python3.8/site-packages/streamsets/sdk/sch.py", line 141, in __init__
    self.api_client = sch_api.ApiClient(
  File "/opt/module/sfsdcvenv/lib/python3.8/site-packages/streamsets/sdk/sch_api.py", line 96, in __init__
    raise ValueError('Encountered error while decoding auth token: {}'.format(e))
ValueError: Encountered error while decoding auth token: 403 Client Error: Forbidden for url: https://cloud.login.streamsets.com/api/security/public/v3/orgs/e892013-fe1345-11ee-87ac-05dc62art4c4/info
 

 

Paste curl return json for reference:
{"tokenStr":"","issuerUrl":null,"expires":-1713929547936,"principalId":"e3cb72c9-fe1a-11ee-87ac-bd50c19cc8de@e725089d-fe1b-11ee-87ac-05dc62a798c4","principalName":"API(SDC_CREDENTIAL) - xxx xx","organizationId":"e892013-fe1345-11ee-87ac-05dc62art4c4","organizationName":"Netsaee","email":"dfwer@123.com","roles":["connection:user","org-admin","datacollector:admin","datacollector:manager","jobrunner:operator","pipelinestore:pipelineEditor","org-user","pipelinestore:rulesEditor","datacollector:creator","user","pipelinestore:user"],"groups":["all@e456778-gh1b-11ee-87er-05dc62a798c4"],"app":false,"attributes":{"component.type":"api-user"}}[

1 reply

Bikram
Rank
Userlevel 5
Badge +1 
from streamsets.sdk import ControlHub
sch = ControlHub(credential_id='your_crediential_id', token='your_token_id')
sdc = sch.data_collectors.get(url='http://your_data_collector_hostname:18630')

builder = sch.get_pipeline_builder(engine_type='data_collector', engine_id=sdc.id)

dev_raw_data_source = builder.add_stage('Dev Raw Data Source')
trash = builder.add_stage('Trash')
dev_raw_data_source >> trash

pipeline = builder.build('My first pipeline')

sch.publish_pipeline(pipeline, commit_message='First commit of my first pipeline')

Try like this way.

Reply


Terms & Conditions