Skip to main content

HADOOPFS_13: Error While writing to HDFS: Java.io.FilenotFoundException  while processing _TMP_Tablename_0

 

While extracting SQL server table with 10million rows. Pipeline is restarting again and again after extracting few records in ADLS with this error. It goes on for hours.  Using JDBC query consumer to connect to sql server and ADLS gen 2 as sink.

 

please help to fix the issue.

Hi @sanchit Could you please share the pipeline? and also what is the SDC/SCH version you are using? Is it possible to share the sdc.log file?

From the attached screen-shot I can see path doesn't exist. So could you let us know is there any other pipeline reading the data from same directory? OR do you have any pipeline deleting the data from this directory?
I think if we enable DEBUG/TRACE logging on the destination we can get some more information about this issue. 

Hi @Ranjith P we are using SDC engine 4.4.1. We are not trying to read from ADLS. We are extracting the table from sql server in ADLS gen 2 using simple select * from table query in JDBC consumer query.

 

I can see it extract some records around 1 million successfully after that it fails with the error due to _tmp_* path not exist in ADLS. i have attached the screenshot.

Reply


Terms & Conditions