Hi team
i using streamset control hub 3.x ,if possible to read and write parquet file in data collector ?
Page 1 / 1
I don’t think there is a native support, but you can try something like this: https://docs.streamsets.com/portal/platform-datacollector/latest/datacollector/UserGuide/Solutions/Parquet.html?hl=converting%2Cdata%2Cparquet%2Cdata%2Cformat
Hope it helps!
Data collector is row based where as parquet is columnar.
You can write the file and then use script to convert it to parquet. As shown above.
Reply
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.