Issue:
Unable to see the output in hive by using select command when hive drift pipeline is running, Once the pipeline is stopped then the result is shown.
Resolution:
Adjust the value of Idle Timeout (in Hadoop FS).
More information about Idle Timeout:
The maximum time that an output file can remain idle. After no records are written to a file for this amount of time, the destination closes the file.