i using streamset control hub 3.x ,if possible to read and write parquet file in data collector ?
Already have an account? Login
No account yet? Create an account
Enter your username or e-mail address. We'll send you an e-mail with instructions to reset your password.
Data collector is row based where as parquet is columnar.
You can write the file and then use script to convert it to parquet. As shown above.
I don’t think there is a native support, but you can try something like this: https://docs.streamsets.com/portal/platform-datacollector/latest/datacollector/UserGuide/Solutions/Parquet.html?hl=converting%2Cdata%2Cparquet%2Cdata%2Cformat
Hope it helps!