After going through the configuration for Data bricks Delta lake destination ,I’m trying to partitioning table based on partition column but encountered with an "DELTA_LAKE_41" error message.
DELTA_LAKE_41 - Destination Table not found in partition columns
Userlevel 2
- StreamSets Employee
- 73 replies
-
14 October 2021
Hi!
If I’m not mistaken that should be solved in a newer version of the Databricks Enterprise Library.
Which version are you using?
Thanks
Userlevel 2
- StreamSets Employee
- 73 replies
-
14 October 2021
That is only supported in our databricks library from version 1.4 
- Roadie
- 7 replies
-
14 October 2021
Can you please elaborate Alex ,that means Partition Column is supported from databricks Library version 1.4 in Streamsets.
Userlevel 2
- StreamSets Employee
- 73 replies
-
14 October 2021
- Answer
Exactly, we added support for Databricks 7.x and 8.x clusters on that version, so partition columns are not supported before 1.4
Reply
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.