Hi team,
I am using field renamer for mssql column name rename from origin mssql “regionid” to target snowflake “RegionID”.
Attchment 1 is my configure for Field Rename configuration. When I ran “previewer”, did not show any errors.
Attachment 2 is log.
Cannot figured out what is the exact reason for the failure from the log.
Can you suggest?
+1
The rename processor looks fine but I believe you might be using JDBC as producer as destination.
If that is the case then column name should be same as per destination table.
In the logs , I can see JDBC related error. There might be some issue in JDBC .
Please refresh your browser and check if your changes are there or not then execute the pipeline to load data to destination.
If that is not the case , then please provide details where you are trying to load data into it, so i can assist you on the same.
Thanks & Regards
Bikram_
The rename processor looks fine but I believe you might be using JDBC as producer as destination.
If that is the case then column name should be same as per destination table.
In the logs , I can see JDBC related error. There might be some issue in JDBC .
Please refresh your browser and check if your changes are there or not then execute the pipeline to load data to destination.
If that is not the case , then please provide details where you are trying to load data into it, so i can assist you on the same.
Thanks & Regards
Bikram_
Hi Bikram, thanks for your input, it is helpful.
And I did further check, and found it should be related snowflake case-insensitive resolution, if I rename the source column name (regionid) to “REGION_ID”, the pipeline can load data properly, but if the target column name in snowflake is “region_ID”, the pipeline cannot successfully.
And I my field renamer processor, I have defined the target column in snowflake like “/"region_ID"”, exactly same name in snowflake "region_ID", but it failed.
I suppose it is a common situation to change the case of the column from source to target through StreamSets, can you please guide how to configure “Target Field Expression” in StreamSets, processor “Fielder Renamer”.
attachment is my config of “Fielder reanamer” and my snowflake target table exact column name, and sqlserver origin exact filed name.
As well as the failed running log. May I know where you saw it is jdbc related errors for destination?
Thank you.
+1
Please find attached the screen shot to rename column name as per the target fields. I hope this will help you.
Please find attached the screen shot to rename column name as per the target fields. I hope this will help you.
Hi Bikram,
I’ve tried your suggestion, make the target column as exactly as /region_ID, it still failed, but when I tried to preview for processor “Field Renamer”, it has no errors. But different from your screenshot, seems 1 input record has 10 output records, is it abnormal? See attached Screenshot below.
Can you please advise how to trouble shooting this problem? Thank you. By the way, May I know is your destination snowflake database?
another important point what I found which might lead to the failure is, every time I ran the pipeline, even the pipeline failed, but it automatically added one column name “REGION_ID” as the last column in my destination snowflake table, please see attached screenshot. That means, I just want rename origin column from “regionid” to “region_ID” in destination, the columns number should be kept as the same as origin table as 12, but now the columns number has been changed to 13 in destination snowflake table with one expected column added, can you please advise why?
+1
Source column name should be same as destination table name. You can removed unused columns using field remover processor.
No worry about the extra columns .
If you can provide the standard input to me so i can provide you the complete pipeline with snowflake destination.
Hi Bikram, attachment is the fabricated but equivalent standard input for my test, you can refer to. thanks.
+1
Can you please try to rename the source column as per snow flake column names ,check if it helps in processing data from SQL to snowflake or not.
Attached the screen shot for the field rename.
Thanks & Regards
Bikram_
Can you please try to rename the source column as per snow flake column names ,check if it helps in processing data from SQL to snowflake or not.
Attached the screen shot for the field rename.
Thanks & Regards
Bikram_
How do we know the exact failed reason?
+1
Please make sure in snowflake destination , the below option are not enabled.
After that do rename the column name with field rename processor and it will fix the issue.
Please let me know if it helps.
Thanks & Regards
Bikram_
Please make sure in snowflake destination , the below option are not enabled.
After that do rename the column name with field rename processor and it will fix the issue.
Please let me know if it helps.
Thanks & Regards
Bikram_
no it not works, still failed without anything loading
+1
It works for me without any issues.
I created a table using the “region_ID” column and loaded data into it .all went fine.
Kindly provide the error after above changes.
It works for me without any issues.
I created a table using the “region_ID” column and loaded data into it .all went fine.
Kindly provide the error after above changes.
ca
+1
Can you please try to preview the pipeline by enabling the write to destination option and i think it will give you error .
+1
can you please provide me the test pipeline ,so i can check where its going wrong.
Please remove all sensitive information from your pipeline .
Thanks & Regards
Bikram_
Enter your username or e-mail address. We'll send you an e-mail with instructions to reset your password.
