Question

Parquet file Data collector

  • 24 January 2023
  • 2 replies
  • 75 views
S
Rank

Hi team

i using streamset control hub 3.x ,if possible to read and write parquet file in data collector ? 

2 replies

albertfc
Rank

I don’t think there is a native support, but you can try something like this: https://docs.streamsets.com/portal/platform-datacollector/latest/datacollector/UserGuide/Solutions/Parquet.html?hl=converting%2Cdata%2Cparquet%2Cdata%2Cformat

Hope it helps!

saleempothiwala
Rank
Userlevel 4
Badge

Data collector is row based where as parquet is columnar.

 

You can write the file and then use script to convert it to parquet. As shown above.

Reply


Terms & Conditions