Skip to main content

How to read Avro Files in File Origin using Spark 2.3 ?

  • December 21, 2021
  • 0 replies
  • 19 views

AkshayJadhav
StreamSets Employee
Forum|alt.badge.img

Product: StreamSets Transformer

Question: 

How to read Avro Files in Spark 2.3 that do not have an ".avro" extension?

Answer:

you must include this property the the Cluster's Extra Spark Config:

spark.hadoop.avro.mapred.ignore.inputs.without.extension = false

 

This topic has been closed for comments