Question

How to process tables with zero records into target location from origin


I would like to process tables with zero records from JDBC Multitable Consumer to my target location,is it possible to do so please help me to find out solution to this query.


11 replies

Userlevel 5
Badge +2

Hi @Ravisaga!

 

What exactly do you want the platform to do when there are 0 records and what’s your target destination? Please elaborate. 

Hi Dash,

I have multiple tables with zero records I want to load these records from JDBC Multitable Consumer to Databricks.In the Databricks I would like to view the structure of table with zero records.Is it possible to do so is there any alternative solution.

Userlevel 5
Badge +2

Hi!

 

So the platform is not going to perform any kind of processing (create tables, etc. on Databricks) if there are no/0 records to process. Out of curiosity, what’s your use case? Why do you want structure / schema to be created even when there are no records?

Hi Dash,

My use case is to load tables into Databricks,I have successfully loaded tables with less than 100k records,while I am trying to process and load tables with zero records I end up with an error and no table created in Databricks.

 

Userlevel 5
Badge +2

What error are you getting?

Hi Dash,

I am not getting error while previewing but when I am running pipeline In real time summary it shows me number of errors as 2 and input is 1,with no table created in Databricks.

Userlevel 5
Badge +2

Can you please post the errors you’re seeing when you run the pipeline? Without all the details it’s really hard to help resolve the issue.

Hi Dash,

I am processing tables with no records into destination Databricks Delta lake.Once pipeline is finished I am encountering error count as 2.

 

 

Userlevel 5
Badge +2

That does not help. What do you see in the logs?

Hi Dash,

Please refer to below logs 

 

Userlevel 5
Badge +2

The screenshot of your screen is not that helpful. Can you access the logs and paste them here?

Reply