Skip to main content

as my origin is local director i want read multipile csv and process it by streamselector according to the file name =${record:value('/fileInfo/filename')=='submission_trackers_25523.csv'} but it shows error how can i handle this scenario.

cd

@lakshmi_narayanan_t 

Please read *.csv from directory as a whole file and add the filter using streamsets collector .

Then you need to use jython or java script process the fetch the file content and send to destination .

 

Please  design your pipeline as given   below and do let me know if it helps.

 

 

In this case i am reading all csv file from directory as whole file , then filter the filename and by using jython evaluator fetching the file content and sending to destination.

 

Attached the pipeline for your reference.

 

Thanks & Regards

Bikram_


@lakshmi_narayanan_t

 

Instead of ${record:value('/fileInfo/filename')}

 

use ${record:attribute('filename')}


@lakshmi_narayanan_t

Please read *.csv from directory as a whole file and add the filter using streamsets collector .

Then you need to use jython or java script process the fetch the file content and send to destination .

 

Please  design your pipeline as given   below and do let me know if it helps.

 

 

In this case i am reading all csv file from directory as whole file , then filter the filename and by using jython evaluator fetching the file content and sending to destination.

 

Attached the pipeline for your reference.

 

Thanks & Regards

Bikram_

as you mentioned  above  is possible to  write in  MYSQL database .


@lakshmi_narayanan_t 

Yes, By using script processor , you can read the file content and load to DB.

Any sample file which you need to load , so i can help you on it.


Reply