How to handle German Characters(Non-English characters) coming from API response?

@Priyabala 

can you please select the character set to ISO-8859-1 or ISO-8859-2 and let me know if it helps.

Please provide me sample input and expected output ,so i can give a try and will update you on it.

HI @Bikram 

I tried changing the charset to ISO-8859-1 or ISO-8859-2 but i am getting the same issue as before.

sample i/p: DENWKÖ0422

sample o/p: DENWK%C3%960422

@Priyabala 

Can you please try encoding after the character set.

e.g , Keep the character set as UTF-8 and after that use jython/groovy and try to encode the field with the below.

record.value['f1'].encode("ISO-8859-1")

record.value['f1'].encode("ISO-8859-2")

@Priyabala 

By using postman , posted a message with german character (Ö)  into HTTP API  and when i retrieved data from HTTP URL with character set (UTF-8) , it gave me the correct result .

No issues in german characters .

@Bikram In postman i am able to retrieve the data. when I am passing German char as an input parameter.

I am facing this issue only with stream sets while joining.

So included groovy evaluator before joining with API. But the Code is throwing errors. Can you look into this?

import java.util.*
records = sdc.records
for (record in records) {
    try {
               
       tmp=record.value["jobDetails"][0]["jobLocationIdentifier"]['value'];
       enctmp=encode(tmp,"ISO-8859-2");
        record.value["jobDetails"][0]["jobLocationIdentifier"]=enctmp;
       
        // Write a record to the processor output
        sdc.output.write(record)
    } catch (e) {
        // Write a record to the error pipeline
        sdc.log.error(e.toString(), e)
        sdc.error.write(record, e.toString())
    }
}

#jobLocationIdentifier----field having German Char

@Priyabala 

Please find below the corrected code . I am not sure if the encoding will fix the issue . I will have a look into it if it didn’t help .

for record in sdc.records:
  try:
    
    
    tmp=record.value["jobDetails"][0]["jobLocationIdentifier"]['value'];
    enctmp= tmp.encode("ISO-8859-2");
    record.value["jobDetails"][0]["jobLocationIdentifier"]=enctmp;
   
    sdc.output.write(record)

  except Exception as e:
    # Send record to error
    sdc.error.write(record, str(e))

HI @Bikram,

The code is throwing lots of issues and we are not sure weather it will resolve our issue of handling German characters.

Is there any other way through we can handle it?

@Priyabala 

can you please try it jython processors. I verified it and code was not throwing any error but for the conversion of German characters i need to dig into it to handle the issue.

Attached the output and pipeline for your reference.

@Priyabala 

Can you please add below details in HTTP origin and give a try.

Content-type:"application/json"; charset=utf-8 in your HTTP client origin.

Bikram wrote:

@Priyabala

Can you please add below details in HTTP origin and give a try.

 

Content-type:"application/json"; charset=utf-8 in your HTTP client origin.

 

 

Did it work?

Did it work?