Databricks Delta Layer - DELTA_LAKE_13 Validation Error

Hi,

I have created a pipeline in the StreamSets Data Collector which reads data from an Apache Kafka topic and inserts it into a Databricks Delta table.

Table Auto Creation has been disabled. I have created the table on the Databricks instance separately and the columns and data types in the table are correct.

But I am getting the following error while validating the pipeline. What could be the reason for the error?

Caused by: com.streamsets.pipeline.api.StageException: DELTA_LAKE_13 - Table 'gov_src_req_sts_v1' column '' unsupported type '' at com.streamsets.pipeline.stage.destination.definitions.JdbcTableDefStore.get(JdbcTableDefStore.java:157) at com.streamsets.pipeline.stage.destination.definitions.CacheTableDefStore$1.load(CacheTableDefStore.java:59) at com.streamsets.pipeline.stage.destination.definitions.CacheTableDefStore$1.load(CacheTableDefStore.java:53) at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3542) at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2323) at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2286) at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2201)