Question

How to handle German Characters(Non-English characters) coming from API response?

  • 30 September 2022
  • 12 replies
  • 771 views
Priyabala
Rank

Receiving German characters from one of the attribute from API response and while joining the response with another API, we are facing issues because of the non-english characters.

Can anyone suggest how we can handle this in pipeline?

12 replies

Bikram
Rank
Userlevel 5
Badge +1

@Priyabala 

 

can you please select the character set to ISO-8859-1 or ISO-8859-2 and let me know if it helps.

 

Please provide me sample input and expected output ,so i can give a try and will update you on it.

Priyabala
Rank

HI @Bikram 

I tried changing the charset to ISO-8859-1 or ISO-8859-2 but i am getting the same issue as before.

sample i/p: DENWKÖ0422

sample o/p: DENWK%C3%960422

Bikram
Rank
Userlevel 5
Badge +1

@Priyabala 

Can you please try encoding after the character set.

 

e.g , Keep the character set as UTF-8 and after that use jython/groovy and try to encode the field with the below.

record.value['f1'].encode("ISO-8859-1")

record.value['f1'].encode("ISO-8859-2")

 

 

 

 

Bikram
Rank
Userlevel 5
Badge +1

@Priyabala 

 

By using postman , posted a message with german character (Ö)  into HTTP API  and when i retrieved data from HTTP URL with character set (UTF-8) , it gave me the correct result .

 

No issues in german characters .

Priyabala
Rank

@Bikram In postman i am able to retrieve the data. when I am passing German char as an input parameter.

I am facing this issue only with stream sets while joining.

So included groovy evaluator before joining with API. But the Code is throwing errors. Can you look into this?

import java.util.*
records = sdc.records
for (record in records) {
    try {
               
       tmp=record.value["jobDetails"][0]["jobLocationIdentifier"]['value'];
       enctmp=encode(tmp,"ISO-8859-2");
        record.value["jobDetails"][0]["jobLocationIdentifier"]=enctmp;
       
        // Write a record to the processor output
        sdc.output.write(record)
    } catch (e) {
        // Write a record to the error pipeline
        sdc.log.error(e.toString(), e)
        sdc.error.write(record, e.toString())
    }
}

#jobLocationIdentifier----field having German Char

Bikram
Rank
Userlevel 5
Badge +1

@Priyabala 

 

Please find below the corrected code . I am not sure if the encoding will fix the issue . I will have a look into it if it didn’t help .

 

 

for record in sdc.records:
  try:
    
    
    tmp=record.value["jobDetails"][0]["jobLocationIdentifier"]['value'];
    enctmp= tmp.encode("ISO-8859-2");
    record.value["jobDetails"][0]["jobLocationIdentifier"]=enctmp;
   
    sdc.output.write(record)

  except Exception as e:
    # Send record to error
    sdc.error.write(record, str(e))

Priyabala
Rank

HI @Bikram,

The code is throwing lots of issues and we are not sure weather it will resolve our issue of handling German characters.

Is there any other way through we can handle it?

Bikram
Rank
Userlevel 5
Badge +1

@Priyabala 

 

can you please try it jython processors. I verified it and code was not throwing any error but for the conversion of German characters i need to dig into it to handle the issue.

Bikram
Rank
Userlevel 5
Badge +1

Attached the output and pipeline for your reference.

2 Attachments
Bikram
Rank
Userlevel 5
Badge +1

@Priyabala 

Can you please add below details in HTTP origin and give a try.

 

Content-type:"application/json"; charset=utf-8 in your HTTP client origin.

 

 

Ritantara
Rank

@Priyabala

Can you please add below details in HTTP origin and give a try.

 

Content-type:"application/json"; charset=utf-8 in your HTTP client origin.

 

 

Did it work?

Ritantara
Rank

Did it work?

Reply


Terms & Conditions