Question:
When writing to a File destination, Spark creates one output file for each partition. However, the goal is to create on a single output file
Answer:
Use the Repartition processor to change the number of partitions that are written to file systems.
Use Repartition by Number strategy and set the following parameter as below
Number of Partitions = 1
