How easy would it be to create a custom Scala or PySpark stage, that can output an array of spark data frames (like it can receive for input) rather than just one?
+1- Headliner
- 486 replies
In this case the custom function will help you in fixing the issue.
Create the custom function scala and deploy the JAR in in streamsets transformer .In scala processor , import the UDF in it to use the function.
Reply
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.