Skip to main content

Hadoop Proxy User for cluster pipelines with Hadoop FS origin.

  • February 19, 2022
  • 0 replies
  • 20 views

AkshayJadhav
StreamSets Employee
Forum|alt.badge.img

Issue:

SDC does not impersonate a proxy user set in Hadoop FS User in Hadoop FS origin. Instead, the pipeline submits the MapReduce job as the 'sdc' user.

 

Answer:

The impersonation is not working properly with HDFS origin as described in JIRA issue SDC-5039.

As a workaround, you can add the Hadoop proxy user in Pipeline configuration for Cluster Batch Execution Mode --> Cluster --> Launcher ENV as HADOOP_PROXY_USER (please see attached print screen). This way, the whole MapReduce job (cluster pipeline) runs under this ‘HADOOP_PROXY_USER’.

Did this topic help you find an answer to your question?

0 replies

Be the first to reply!

Reply