Skip to main content

I am using field type convertor to convert the column into time stamp and write into mysql but it only write 2023 year details how can i write 1970 year ,fix it by any other processor.

this is my configration 
1970 year data not writable in mysql
in mysql only 2023 data will be write 

 

Hello @lakshmi_narayanan_t - Thank you for reaching out to the StreamSets community. I have not been able to replicate the issue in house. I do see that same year is getting loaded in the mysql however my source is Mysql. Not sure about the source of the pipeline. 

 

This is the my setup information:

  • 8.0.32 MySQL Community Server - GP
  • mysql-connector-java-8.0.28.jar

What is the driver that you are using and source of the pipeline?

@AkshayJadhav  IN MY CASE csv to my sql ,the data i had is two dates years 2023,and 1970 but i only 2023 date are write in mysql another date is not written. 

@lakshmi_narayanan_t  Please add the following debug in the data collector UI » Administration » Logs » Logs Configs and append the following line then save.

log4j.logger.com.streamsets.pipeline.stage.destination.jdbc=DEBUG
log4j.logger.com.streamsets.pipeline.lib.jdbc=TRACE


This will show what query is being executed in the backend at the producer stage. You can process a single records, monitor the logs and find what values are being picked up by the stage.

 

 

@lakshmi_narayanan_t 

The date fields having the value 1970-01-01 won’t process to Oracle tables and this value is the default value coming from StreamSets if the value is NULL in source system .

One way is that to filter those rows and replace the value with original value coming from the source.

Please let me know if it helps.

 

Thanks & Regards

Bikram_

 

Reply


Terms & Conditions