Solved

DELTA_LAKE_41 - Destination Table not found in partition columns

  • 14 October 2021
  • 5 replies
  • 59 views
R
Rank

After going through the configuration for Data bricks Delta lake destination ,I’m trying to partitioning table based on partition column but encountered with an "DELTA_LAKE_41" error message.

 

icon

Best answer by alex.sanchez 14 October 2021, 10:34

View original
Ravisaga!

5 replies

alex.sanchez
Rank
Userlevel 2
Badge

Hi!

If I’m not mistaken that should be solved in a newer version of the Databricks Enterprise Library.

 

Which version are you using?

Thanks

Alex - StreamSets Engineering Manager @ Collector
R
Rank

Hi Alex,

We are using Databricks Version - “ DBR - 8.3”

Ravisaga!
alex.sanchez
Rank
Userlevel 2
Badge

That is only supported in our databricks library from version 1.4 :smiley:

Alex - StreamSets Engineering Manager @ Collector
R
Rank

 

Can you please elaborate Alex ,that means Partition Column is supported from databricks Library version 1.4 in Streamsets.

Ravisaga!
alex.sanchez
Rank
Userlevel 2
Badge

Exactly, we added support for Databricks 7.x and 8.x clusters on that version, so partition columns are not supported before 1.4

Alex - StreamSets Engineering Manager @ Collector

Reply


Terms & Conditions