Skip to main content

Hi,

Understand GCS stage in SDC does not support Parquet but is there any reason or workaround to read Parquet from GCS using Data Collector?

Thank you.

 

 

@anirbanch 

Parquet is a column oriented data storage format where as SDC is row-based micro batching processing engine hence it cannot read parquet.

You can use StreamSets Transformer engine to read parquet files and transform it.


Reply