Error using a directory created in IIS (Windows) and published via FTPS

Hii @Yago Aparecido,  Seems to be connectivity issue. Since you are using FTPS,  Make sure that FTPS truststore provider setting and relevant client certificate are configured correctly. In pipeline you can provide all this TLS details. 

https://docs.streamsets.com/portal/datacollector/3.18.x/help/datacollector/UserGuide/Origins/SFTP.html#task_lfx_fzd_5v 

Hi @Rishi ,

thanks for the feedback.

but, when we insert the following directory after the "/PLV" the connection is OK

example: ftps://ftps.hostname.net:921/PLV/Input

with that, we believe that the problem might be in the streamsets or something.

why when we use the address "ftps://ftps.hostname.net:921/PLV" it gives the error.

@Yago Aparecido  could you please share your pipeline (please strip sensitive information) and we can have a detailed look at this.  Also, having the logs when running it at debug log level would be very helpful.
you can set the following debug :

log4j.logger.com.streamsets.pipeline.stage.origin.remote=DEBUG
log4j.logger.com.streamsets.pipeline.lib.remote=DEBUG

Run against ftps://ftps.hostname.net:921/PLV which should result in error, also run against ftps://ftps.hostname.net:921/PLV/Input which should be successful 
And share this logs. 

hi @Rishi  / @Drew Kreiger 

below are the logs we collect when we use the address "ftps://hostname_ftps/PLV":

{"log":"2022-02-14 11:02:01,000 [user:*admin] [pipeline:MoveFTPSPLVInput/MoveFTPSPLe88597d8-2442-4e31-bd63-3cf25197387d] [runner:] [thread:preview-pool-364-thread-1] [stage:] INFO  FTPRemoteConnector - Connecting to ftps://hostname_ftps/PLV/\n","stream":"stdout","time":"2022-02-14T14:02:01.000229873Z"}
{"log":"2022-02-14 11:02:01,059 [user:*admin] [pipeline:MoveFTPSPLVInput/MoveFTPSPLe88597d8-2442-4e31-bd63-3cf25197387d] [runner:] [thread:preview-pool-364-thread-1] [stage:] ERROR FTPRemoteConnector - Unable to connect to remote host 'ftps://hostname_ftps/PLV/' with given credentials. Please verify if the host is reachable, and the credentials and other configuration are valid. The logs may have more details. Message: Could not list the contents of \"ftps://hostname_ftps/PLV\" because it is not a folder.\n","stream":"stdout","time":"2022-02-14T14:02:01.060126096Z"}
{"log":"org.apache.commons.vfs2.FileNotFolderException: Could not list the contents of \"ftps://hostname_ftps/PLV\" because it is not a folder.\n","stream":"stdout","time":"2022-02-14T14:02:01.060163332Z"}
{"log":"Caused by: org.apache.commons.vfs2.FileNotFolderException: Could not list the contents of \"ftps://hostname_ftps/PLV\" because it is not a folder.\n","stream":"stdout","time":"2022-02-14T14:02:01.060288757Z"}

below are the logs we collect when we use the address "ftps://hostname_ftps/PLV/Input":

{"log":"2022-02-14 11:02:45,584 [user:*admin] [pipeline:MoveFTPSPLVInput/MoveFTPSPLe88597d8-2442-4e31-bd63-3cf25197387d] [runner:] [thread:preview-pool-364-thread-1] [stage:] INFO  FTPRemoteConnector - Connecting to ftps://hostname_ftps/PLV/Input\n","stream":"stdout","time":"2022-02-14T14:02:45.584619146Z"}
 

we are using the "validate" option of streamsets to validate the settings and collect these logs.

PS: this containerized streamsets in docker

hi @Rishi  / @Drew Kreiger 

would you have any feedback on this issue we are having?

@Yago Aparecido I am unable to reproduce this in my environment, So I would like you to run the following test to further narrow down this issue

• Create new dir in ftp and which does not have any subdirectory , use this directory path for your pipeline and check if the issue persist ?
• Does the issue only occurs with FTPS, what if you switch to other protocol FTP/SFTP ?

Let us know your finding and we can try further troubleshoot this.

Hi @Rishi / @Drew Kreiger 

We will perform the first test requested and will get back to you shortly with the results.

In the second test, even changing to FTP or SFTP, the following errors are returned:

SFTP/FTP/FTPS Client 1
REMOTE_17 - The prefix of the url ftps://ftps.hostname.net:921/PLV does not correspond to the selected protocol FTP. : conf.remoteConfig.remoteAddress

SFTP/FTP/FTPS Client 1
REMOTE_17 - The prefix of the url ftps://ftps.hostname.net:921/PLV does not correspond to the selected protocol SFTP. : conf.remoteConfig.remoteAddress

@Yago Aparecido Pardon me; for not clarifying the test-2 

• Regarding the second test, it expected to don’t work against your current FTP server until the other protocols are also configured at server level, and hence we can’t just change protocol at client level and expect it to work.
  
  What I was referring to :  if you can try setup a quick other test FTP environment (ref: here)  and use other protocol to test and verify the scenarios. 
   

Hi @Rishi / @Drew Kreiger 

unfortunately it is not possible to configure another environment for testing, we have to use SFTP itself, on port 921, as the address in the previous details shows.

@Yago Aparecido Did you get chance to try test-1 ?

Hi @Rishi 

apologies for the delay.

We performed the test using a new director created in IIS without any subdirectories, but still got the same error:

"REMOTE_11 - Unable to connect to remote host 'ftps://ftps.hostname.net:921/PLV_TST' with given credentials. Please verify if the host is reachable, and the credentials and other configuration are valid. The logs may have more details. Message: Could not list the contents of "ftps://ftps.hostname.net:921/PLV_TST" because it is not a . :

conf.remoteConfig.remoteAddress"

Hello @Yago Aparecido This seems to be environment specific issue and hence would require additional troubleshooting.  If you're a StreamSets enterprise customer, please open a support ticket.

 

Hi @Rishi / @Drew Kreiger,

Unfortunately we don't use StreamSets Enterprise, we use StreamSets Open Source.

Thanks for the support so far.