Issue:
- Is there any way to remove the magic marker (¡) from the JSON file?
- When Error Records - Write to File is enabled, SDC is generating error file in the specified folder, but adding junk character in the first record.
Solution:
- The files which are generated are not in JSON data format, its SDC Record data format (which is StreamSets proprietary binary format).
- Create a pipeline that reads this ERROR record files(SDC Record data format) and place it in a different folder as a JSON file.
Note: To capture the error message in the JSON file, use Expression Evaluator and capture the error from the SDC record header and add it is a field.
