Product: Streamsets Data Collector
Issue: When reading from Oracle and writing into Hive destination the primary key coulmn was defined as NUMBER(12,0)
When you have a pipeline with Field type convertor that performs a Decimal to Float conversion before writing to Hive.
Versions affected: All
Solution:
Due to Field Type Convertor the incoming PK column value was being converted to float
This was changing the data value and writing wrong values to the end point.
Basically if you have a field something line
SOURCE RECORDOBJECT_ID OBJECT_NAME1456098345.0 T13491348976.0 T24359803400.0 T3
The same records get written into the target as shown below when you have a Field Type Convertor performing a Decimal to Float values conversion
TARGET RECORDOBJECT_ID OBJECT_NAME1456098304 T13491348992 T24359803392 T3
If you focus on the Input and output records the OBJECT_ID which is primary key is getting changed because of the Field Type Convertor. If you set any conversion based on the Datatype ensure that the primary key columns are not affected because of that