Skip to main content

Convert multiple CSV records into a single JSON and make an HTTP bulk API POST call


swayam
Discovered Fame
Forum|alt.badge.img
  • Discovered Fame
  • 20 replies

Hi,

The above pipeline reads the CSV files uploaded into an S3 bucket for new contact file uploads. Based on the new contacts available as a CSV, it converts them into a JSON. But the critical part here is that it merges each batches (each batch size is 1000) into a single JSON of multiple records and finally make a single HTTP POST call to one of our internal system to carry out a CONTACT update call.

5 replies

  • Fan
  • 4 replies
  • September 22, 2022

Hi Swayam,

Can you share the stages configuration.


swayam
Discovered Fame
Forum|alt.badge.img
  • Author
  • Discovered Fame
  • 20 replies
  • September 22, 2022

Send me your email address and I'll share the pipeline.


swayam
Discovered Fame
Forum|alt.badge.img
  • Author
  • Discovered Fame
  • 20 replies
  • September 22, 2022

Can you import the attached pipeline and try? Just unzip and import the pipeline json.

@Rohith 


  • Fan
  • 4 replies
  • September 22, 2022

Thanks Swayam 


  • Fan
  • 4 replies
  • September 23, 2022

Hi Swayam,

Thanks for sharing the pipeline.

I have a requirement where my input json will look like this.

[

{

“name”:”xyz”,

“age”:”21”

},

{

“name”:”xyz”,

“age”:”21”

},

.

.

.

]

 

And I want the output to be in the below json format

{
    "data": {
        "columns": {
            "name": "character",
            "age": "numeric"
        },
        "records": [

{

“name”:”xyz”,

“age”:”21”

},

{

“name”:”xyz”,

“age”:”21”

},

.

.

.

]

    },
    "options": {
        "purge": false
    }
}

Can you help me with this? Is it possible using a single a pipeline?

Thanks in Advance


Reply