Azure ADLS Gen2 Destination pipeline fails on startup: Server failed to authenticate the request.


Userlevel 4
Badge

Issue:

When starting a pipeline with an Azure ADLS Gen2 Destination we can see the following exception causing the pipeline to start:

CONTAINER_0800 - Can't start pipeline due 10 validation error(s). First one: HADOOPFS_44 - Could not verify the base directory: 'HEAD https://<my-server>.dfs.core.windows.net/lines?resource=filesystem&timeout=90 StatusCode=403 StatusDescription=Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature. ErrorCode= ErrorMessage='

Solution:

After verifying that the connection to the ADLS server and the authentication used should also be successful, this issue can also be caused by a problem with the SDC server time. We would need to verify that our host has NTP installed and properly configured. We can verify that we have the service up and running with the following command:

systemctl status ntpd

We can run the following commands to verify NTP peers synchronization status and the system time.

ntpq -p
date -R

 


0 replies

Be the first to reply!

Reply