Question

active file ,file tail

  • 7 February 2023
  • 7 replies
  • 54 views
S
Rank

hi,

I want to use the file tail origin but i didn’t known the active file path usage  in sdc. anyone can provide the configuration of file tail and active file path . 

7 replies

AkshayJadhav
Rank
Userlevel 4
Badge

Hello @santosh - Thank you for reaching out to the StreamSets community. Here is the sample pipeline which is tailing the file akshay.csv available under the /tmp location.

You can follow the below link for more details.

 https://docs.streamsets.com/portal/datacollector/5.1.x/help//datacollector/UserGuide/Origins/FileTail.html#task_unq_wdw_yq


Let us know if you have any questions - AkshayJ

S
Rank

Hello @santosh - Thank you for reaching out to the StreamSets community. Here is the sample pipeline which is tailing the file akshay.csv available under the /tmp location.

You can follow the below link for more details.

 https://docs.streamsets.com/portal/datacollector/5.1.x/help//datacollector/UserGuide/Origins/FileTail.html#task_unq_wdw_yq


Let us know if you have any questions - AkshayJ

yeah thank you ,I used the same format of pipeline but what i required  postprocessing  process like Delete.and how it Processed.After the completion of pipeline i want to delete.but not happenes

saleempothiwala
Rank
Userlevel 4
Badge

@Santhosh Kumar 

Have you set the option to ‘Delete - Deletes the file.’ on the Post Processing tab?

Does it give any errors?

AkshayJadhav
Rank
Userlevel 4
Badge

@Santhosh Kumar - Could you please capture the sdc log and see if there is any error for post processing operation?

 

S
Rank

@Santhosh Kumar

Have you set the option to ‘Delete - Deletes the file.’ on the Post Processing tab?

Does it give any errors?

yes, i used that option .after the process finised i can verify the path to confirm the file exists or not ,it will show the file or not .

saleempothiwala
Rank
Userlevel 4
Badge

@Santhosh Kumar 

Yes, confirm and see if the post processing worked. If not, share the logs and any error messages that you received.

S
Rank

@Santhosh Kumar

Have you set the option to ‘Delete - Deletes the file.’ on the Post Processing tab?

Does it give any errors?

yes used that option delete ,but not work and also no error  .but I didn’t known how to ensure the delete option working or not  in correct way .Can you pls solve this ..

Reply


Terms & Conditions