Skip to main content
Solved

DELTA_LAKE_41 - Destination Table not found in partition columns

  • October 14, 2021
  • 5 replies
  • 67 views
R
Roadie

After going through the configuration for Data bricks Delta lake destination ,I’m trying to partitioning table based on partition column but encountered with an "DELTA_LAKE_41" error message.

 

Best answer by alex.sanchez

Exactly, we added support for Databricks 7.x and 8.x clusters on that version, so partition columns are not supported before 1.4

Ravisaga!
alex.sanchez

5 replies

alex.sanchez
StreamSets Employee
Forum|alt.badge.img
  • alex.sanchezStreamSets Employee
  • StreamSets Employee
  • October 14, 2021

Hi!

If I’m not mistaken that should be solved in a newer version of the Databricks Enterprise Library.

 

Which version are you using?

Thanks

Alex - StreamSets Engineering Manager @ Collector
Dash
R
Roadie
  • RavisagaRoadie
  • Author
  • Roadie
  • October 14, 2021

Hi Alex,

We are using Databricks Version - “ DBR - 8.3”

Ravisaga!
alex.sanchez
StreamSets Employee
Forum|alt.badge.img
  • alex.sanchezStreamSets Employee
  • StreamSets Employee
  • October 14, 2021

That is only supported in our databricks library from version 1.4 :smiley:

Alex - StreamSets Engineering Manager @ Collector
R
Roadie
  • RavisagaRoadie
  • Author
  • Roadie
  • October 14, 2021

 

Can you please elaborate Alex ,that means Partition Column is supported from databricks Library version 1.4 in Streamsets.

Ravisaga!
alex.sanchez
StreamSets Employee
Forum|alt.badge.img
  • alex.sanchezStreamSets Employee
  • StreamSets Employee
  • Answer
  • October 14, 2021

Exactly, we added support for Databricks 7.x and 8.x clusters on that version, so partition columns are not supported before 1.4

Alex - StreamSets Engineering Manager @ Collector
Drew Kreiger
Dash
R
Terms & ConditionsAccessibility statement