Solved

Cloud Anti-virus Scan via ICAP in Streamsets DC

  • 30 October 2021
  • 2 replies
  • 28 views

Dear Experts, 

I have a simple SFTP data pipeline to take text files from an SFTP cloud server and push these files to a local file system.

However, I am required to get these files scanned by an anti-virus via ICAP call "icap://${SERVER_IP}:${SERVER_PORT}/virusScan" before writing to the target.

Can anyone guide me on how to get this done in Streamsets?

FYI: some other tools like Informatica have a built-in option for this. Please refer this link:

https://knowledge.informatica.com/s/article/613287?language=en_US&type=external

icon

Best answer by alex.sanchez 1 November 2021, 12:06

View original

2 replies

Userlevel 2
Badge

Hi @wasifkabeer,

I might not be catching up the full use case, but can the HTTP processor be the solution? 
https://docs.streamsets.com/portal/#datacollector/latest/help/datacollector/UserGuide/Processors/HTTPClient.html

Thanks

Hi @alex.sanchez,

Thank you for your response. I will give it a try.

Reply