Skip to main content

How to process JDBC TIMESTAMPs with microsecond or nanosecond resolution?

  • December 22, 2021
  • 0 replies
  • 482 views

AkshayJadhav
StreamSets Employee
Forum|alt.badge.img

When a table contains a TIMESTAMP column with resolution which is greater than milliseconds, it is handled a bit differently. In Data Collector - a field with a TIMESTAMP(9) datatype is stored internally as a Java Date which has millisecond precision, and Data Collector retains the microsecond and nanosecond portion of the TIMESTAMP value separately - stored as a Field attribute.

The Field and it's Attributes can be accessed in the Expression Language Stage; we can concatenate these two fields to reconstitute the original TIMESTAMP - as a String.

Example:

We have a JDBC column named 'timeStamp' the JDBC datatype is TIMESTAMP(9) so it's been split up with microseconds and nanoseconds in the Field attribute.

We use an EL to create a field with the String representation of the Java Date object (call it

`timeStampUpToMillis`:
$(time:extractStringFromDate(record:value('/timeStamp'), "yyyy-MM-dd 'T'HH:mm:ss.SSS")}

Next we append the micro and nanoseconds with the following EL

${str:concat(record:value('/timeStampUpToMillis'), record:fieldAttribute('/timeStamp', 'nanoSeconds')}

Or as a one-liner:

${str:concat(time:extractStringFromDate(record:value('/TS'), "yyyy-MM-dd'T'HH:mm:ss.SSS"), record:fieldAttribute('/TS', 'nanoSeconds'))}

 

This topic has been closed for replies.