Unable to see the output in hive when the pipeline is in running state.


Userlevel 4
Badge

Issue:

Unable to see the output in hive by using select command when hive drift pipeline is running, Once the pipeline is stopped then the result is shown.


Resolution:

Adjust the value of Idle Timeout (in Hadoop FS).

More information about Idle Timeout:

The maximum time that an output file can remain idle. After no records are written to a file for this amount of time, the destination closes the file.


0 replies

Be the first to reply!

Reply