Skip to main content
Question

Read file from ADLS Origin and Update same file in ADLS destination stage


I have a ADLS origin stage where I have a data file. The data file needs to be updated with a different delimiter value in it. After updating the values in the file, I need to overwrite it back to the same file. I am using jython evaluator to do the updates. However, I am getting a new file created in ADLS instead of writing to the existing file.

Here is my Jython code that I am using 
 

 try:

    text_attribute = record.value["text"]
    unicode_text = unicode(text_attribute,'utf-8')
    modified_text = unicode_text.replace(u'|', u'~')
    record.value["text"]=modified_text
    sdc.output.write(record)



  except Exception as e:
    sdc.error.write(record, str(e))

My Pipeline :
ADLS Origin Stage -→ Jython Evaluator -→ ADLS Destination Stage

Reply