I have a ADLS origin stage where I have a data file. The data file needs to be updated with a different delimiter value in it. After updating the values in the file, I need to overwrite it back to the same file. I am using jython evaluator to do the updates. However, I am getting a new file created in ADLS instead of writing to the existing file.
Here is my Jython code that I am using
try:
text_attribute = record.valuel"text"]
unicode_text = unicode(text_attribute,'utf-8')
modified_text = unicode_text.replace(u'|', u'~')
record.valuel"text"]=modified_text
sdc.output.write(record)
except Exception as e:
sdc.error.write(record, str(e))
My Pipeline :
ADLS Origin Stage -→ Jython Evaluator -→ ADLS Destination Stage