Hi team
i using streamset control hub 3.x ,if possible to read and write parquet file in data collector ?
Question
Parquet file Data collector
- Roadie
I don’t think there is a native support, but you can try something like this: https://docs.streamsets.com/portal/platform-datacollector/latest/datacollector/UserGuide/Solutions/Parquet.html?hl=converting%2Cdata%2Cparquet%2Cdata%2Cformat
Hope it helps!
- Headliner
Data collector is row based where as parquet is columnar.
You can write the file and then use script to convert it to parquet. As shown above.
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.