@Dolphin 

The rename processor looks fine but I believe you might be using JDBC as producer as destination. 

If that is the case then column name should be same as per destination table.

In the logs , I can see JDBC related error. There might be some issue in JDBC .

Please refresh your browser and check if your changes are there or not then execute the pipeline to load data to destination.

If that is not the case , then please provide details where you are trying to load data into it, so i can assist you on the same.

Thanks & Regards

Bikram_

Hi Bikram, thanks for your input, it is helpful.

And I did further check, and found it should be related snowflake case-insensitive resolution, if I rename the source column name (regionid) to “REGION_ID”, the pipeline can load data properly, but if the target column name in snowflake is “region_ID”, the pipeline cannot successfully.

And I my field renamer processor, I have defined the target column in snowflake like “/"region_ID"”, exactly same name in snowflake "region_ID", but it failed.

I suppose it is a common situation to change the case of the column from source to target through StreamSets, can you please guide how to configure “Target Field Expression” in StreamSets, processor “Fielder Renamer”.

attachment is my config of “Fielder reanamer” and my snowflake target table exact column name, and sqlserver origin exact filed name.
As well as the failed running log. May I know where you saw it is jdbc related errors for destination? 

Thank you.

@Dolphin 

Please find attached the screen shot to rename column name as per the target fields. I hope this will help  you.

Hi Bikram,

I’ve tried your suggestion, make the target column as exactly as /region_ID, it still failed, but when I tried to preview for processor “Field Renamer”, it has no errors. But different from your screenshot, seems 1 input record has 10 output records, is it abnormal? See attached Screenshot below.

Can you please advise how to trouble shooting this problem? Thank you. By the way, May I know is your destination snowflake database?
 

another important point what I found which might lead to the failure is, every time I ran the pipeline, even the pipeline failed, but it automatically added one column name “REGION_ID” as the last column in my destination snowflake table, please see attached screenshot. That means, I just want rename origin column from “regionid” to “region_ID” in destination, the columns number should be kept as the same as origin table as 12, but now the columns number has been changed to 13 in destination snowflake table with one expected column added, can you please advise why?  

@Dolphin 

Source column name should be same as destination table name. You can removed unused columns using field remover processor.

No worry about the extra columns . 

If you can provide the standard input to me so i can provide you the complete pipeline with snowflake destination.

Hi Bikram, attachment is the fabricated but equivalent standard input for my test, you can refer to. thanks. 

@Dolphin 

Can you please try to rename the source column as per snow flake column names ,check if it helps in processing data from SQL to snowflake or not.

Attached the screen shot for the field rename.

Thanks & Regards

Bikram_

@Bikram I did the same as you suggested, but still failed. This time had no new column added.

How do we know the exact failed reason? 

@Dolphin 

Please make sure in snowflake destination , the below option are not enabled.

After that do rename the column name with field rename processor and it will fix the issue.

Please let me know if it helps.

Thanks & Regards

Bikram_

@Dolphin

Please make sure in snowflake destination , the below option are not enabled.

 

 

 

After that do rename the column name with field rename processor and it will fix the issue.

 

 

Please let me know if it helps.

 

Thanks & Regards

Bikram_

no it not works, still failed without anything loading

@Dolphin 

It works for me without any issues. 

I created a table using the “region_ID” column and loaded data into it .all went fine.

Kindly provide the error after above changes.

@Bikram can you show where and how to get the exact error? Paste the log to you, but seems no exact failed error there.

ca

@Dolphin 

Can you please try to preview the pipeline by enabling the write to destination option and i think it will give you error .

@Dolphin 

can you please provide me the test pipeline ,so i can check where its going wrong.

Please remove all sensitive information from your pipeline .

Thanks & Regards

Bikram_