Solved

aggregate the records into a single array of objects

N
Rank

I have some json objects that looks as follows 

and i want them to be all merged in an array like :
data: [

{

nstaskid: “...”,

awb: “….”

},

{

nstaskid: “...”,

awb: “….”

},

]

 

i tried to use javascript evalutor , loop over the records and push them in an array and write this record array , but it didn’t work.

Can you please help me with that .

icon

Best answer by Bikram 1 August 2022, 10:35

View original

15 replies

Bikram
Rank
Userlevel 5
Badge +1

@N_ali 

please find attached pipeline for your use case and do let. me know if it helps .

1 Attachment
N
Rank

Hello , Thanks for your reply 

i imported the pipeline but it gave me these errors;

 

it says that my current version is not compatible with this version , is there a way to fix that ?

 

Bikram
Rank
Userlevel 5
Badge +1

@N_ali 

 

You need to change the data collector engine and try to validate after that.

 

Bikram
Rank
Userlevel 5
Badge +1

@N_ali 

My data collector version is 4.4.0 

Bikram
Rank
Userlevel 5
Badge +1

@N_ali 

 

please try this.

 

Inputs :

{
  "Name": "abc",
  "age": "12"
}
{
  "Name": "xyz",
  "age": "21"
}
{
  "Name": "abd",
  "age": "99"
}
{
  "Name": "ABC",
  "age": "32"
}

 

Jython Evaluator :

 

my_j_list=[]

def jsonList(record):
  listItems = {
    "Name" : record.value['Name'],
    "age" : record.value['age']
   }
  return listItems
    
for record in sdc.records:
  try:
    my_j_list.append(jsonList(record))
    record.value = my_j_list
    

  except Exception as e:
    # Send record to error
    sdc.error.write(record, str(e))
sdc.output.write(record)

 

 

output :

 

In the attachment

1 Attachment
N
Rank

it gave me an error in the last line says that record is not defined.

NameError: name 'record' is not defined in <script> at line number 18

 

N
Rank

i fixed it , thanks alot , it’s working now

Bikram
Rank
Userlevel 5
Badge +1

@N_ali 

 

Nice to know that the solution helps and issue you were facing its fixed.

Thanks a lot and let me know in case of any issues/concerns.

tamilselg

it gave me an error in the last line says that record is not defined.

NameError: name 'record' is not defined in <script> at line number 18

 

@N_ali Hi, how did you fix it?

N
Rank

@tamilselg Hii , it was an indentation problem in python.

tamilselg

@tamilselg Hii , it was an indentation problem in python.

Hi @N_ali , thanks for your response. I can’t seem to find the issue still. I am seeing below error

Script error while processing batch: javax.script.ScriptException: NameError: name 'record' is not defined in <script>

 

Here is my code 

my_j_list = []


def jsonList(record):
    listItems = {
        "item1": record.value["item1"],
        "item2": record.value["item2"],
        "item3": record.value["item3"],
        "item4": record.value["item4"],
        "item5": record.value["item5"],
    }
    return listItems


for record in sdc.records:
    try:
        my_j_list.append(jsonList(record))
        record.value["data"] = my_j_list

    except Exception as e:
        # Send record to error
        sdc.error.write(record, str(e))

sdc.output.write(record)

 

I tried fixing the indentation , but doesn’t seem to work. Your help would be much appreciated. Thanks.

N
Rank

Hi @tamilselg ,
i copied the code and pasted in a jython processor but it gave me no syntax errors and worked fine.
can you please provide me with a sample of the input data and the error referred to which line of code ?

 

Thanks!

tamilselg

Hi @tamilselg ,
i copied the code and pasted in a jython processor but it gave me no syntax errors and worked fine.
can you please provide me with a sample of the input data and the error referred to which line of code ?

 

Thanks!

 

@N_ali This is how my input looks like, it is failing at the last line which is sdc.output.write(record). 

 

Thanks.

N
Rank

Hi @tamilselg ,
So the problem was that we are trying to access variable “record” outside the for loop.

That’s why it gave the error because this variable now is out of scope after the for loop.

 

we can make a slight change: we want the list to be written one time when all of the records are processed thus we can do the following (line 18,19) in the for loop after formatting the input object and now we can remove the last line that was causing the error.

my_j_list = []

def jsonList(record):
  listItems = {
    "item1": record.value["item1"],
    "item2": record.value["item2"],
    "item3": record.value["item3"],
    "item4": record.value["item4"],
    "item5": record.value["item5"]
  }
  return listItems

for idx , record in enumerate(sdc.records):
  try:
    my_j_list.append(jsonList(record))
    record.value = my_j_list

    if(idx == len(sdc.records) - 1):
      sdc.output.write(record)

  except Exception as e:
    sdc.error.write(record, str(e))

Hope this helps.

 

tamilselg

Hi @tamilselg ,
So the problem was that we are trying to access variable “record” outside the for loop.

 In the last line -  that’s why it gave the error because this variable now is out of scope after the for loop.

 

we can make a slight change: we want the list to be written one time when all of the records are processed thus we can do the following (line 18,19) in the for loop after formatting the input object and now we can remove the last line that was causing the error.

my_j_list = []

def jsonList(record):
  listItems = {
    "item1": record.value["item1"],
    "item2": record.value["item2"],
    "item3": record.value["item3"],
    "item4": record.value["item4"],
    "item5": record.value["item5"]
  }
  return listItems

for idx , record in enumerate(sdc.records):
  try:
    my_j_list.append(jsonList(record))
    record.value = my_j_list

    if(idx == len(sdc.records) - 1):
      sdc.output.write(record)

  except Exception as e:
    sdc.error.write(record, str(e))

Hope this helps.

 

@N_ali Thanks very much for the response, it works fine now. 

Reply


Terms & Conditions