Question

how to parametrize field remover field names

  • 10 February 2023
  • 4 replies
  • 65 views

Hi team,
I am facing problem in parametrizing filed names in field remover. Actually individual field names you can provide like /type,/id,/name. But in my use case I want to parametrize these fields and read it from properties file that thing I am not able to do. 


4 replies

Userlevel 5
Badge +1

@himanshu1234567 

May i know if you want to keep the fields in sdc.properties file and wants to use it streamsets ?

If that is the case then declare the variables in the properties files and try to use  like ${id} .

 If that is not the case , kindly provide some example so i can have a look and try to find  resolution for the use case.

 

Thanks & Regards

Bikram_

@Bikram 
Ya bikram same thing I want to put field in sdc.properties file and wants to use in streamsets. I have tried also , I will give one example I have put variable like runtime.conf_p_artist_type=type and in streamsets I am providing name like /*[${runtime:conf(‘p_artist_type’)}]  as per documentation but it is not working. 

Userlevel 5
Badge +1

@himanshu1234567 

 

Kindly check if the parameters are still available in sdc.properties file after restarting data collector.

I noticed that after restart its replacing with new sdc.properties file in SDC and i believe that is reason the value couldn’t recognise in

${runtime:conf(<property name>)}
Userlevel 5
Badge +1

@himanshu1234567 

Please refer the solution provided in the below use case.

 

 

Reply