How to convert JSON data to CSV by using SDC pipeline?

  • 10 January 2022
  • 2 replies
  • 800 views

Userlevel 4
Badge

Overview:

  • This document provides instructions to convert JSON data to CSV by using the SDC pipeline.

Configuration:

JSON to CSV converter pipeline looks as below way:

Directory Origin:

  • This stage reads the employee JSON file from the input directory and provides the output as an array of JSON.
  • The output of this stage looks as below way:

Field Pivoter:

  • This stage converts the array of JSON to individual records.
  • Here is the output of the stage:

Field Order:

  • This stage converts the record root field 'MAP' to List-Map. So, that it is compatible with CSV records.
  • If this stage is not used then you will see the below error message in destination:
DELIMITED_GENERATOR_00 - Record 'employee.json::0_1' root field should be List or List-Map instead of 'MAP'

LocalFs Destination:

  • This stage places the data as CSV in the destination location.

Input JSON file:

 

Output CSV file:

 


2 replies

How to do vice-versa operation CSV to JSON data transformation via streamsets? any one can help on this?

Userlevel 5
Badge +1

@Sankar Arunachalam 

 

Kindly try json generator and it will convert csv data to  json format.

 

 

Reply