Skip to main content

I’m building a pipeline which data source will be the twitter API and data destination will be the confluent cloud. I have added the broker URL according to my Confluent Cloud Server endpoint. I’m bit stucked in configuring the security details in Streamsets. Currently I’m getting the below issue and I appreciate if anyone could help me on this. 

From confluent cloud, I have a API key and Secret to authenticate. 

 

 

@Marty Norris 

hi @tdeshappriya 

 

I can see that you have selected ‘Override Stage Configurations’ checkbox. This is used when you want to override the security options provided by StreamSets that you can choose in Security Tab. Since the ‘Override Stage Configurations’ checkbox is true, non of the settings you provide in Security will work. So uncheck and see if that sorts it out. Or provide kafka configuration as key value pair with Override option selected.

Hope that helps.

hi @saleempothiwala,

Thank you very much for your reply. Even when I have not selected the Override Stage Configarations checkbox, I have got the same issue as I have not added the security properly. From where I can add the key value pair in Streamsets? I couldn’t find any place to add it. 

@tdeshappriya, you can test it by using something like…
 

 

Provide values that are relevant to your configurations.

jaas.config can be created and stored in $SDC_CONF folder and referred in Java_Opts variable if needed.

Set Override Stage Configurations to true and select NONE in security tab

Hi @saleempothiwala ,

 

Thank you very much for the detailed explanation. Your solution worked and I was able to configure the details properly and send the data to the confluent without any issue. 

 

@tdeshappriya glad to be able to help :-)

Reply


Terms & Conditions