Hello, I have a response from a HTTP client stage that has been pivoted. Some of the fields are lists and these return errors when trying to load into Snowflake.
Ideally I’m trying to get the lists (array) inserted as JSON in variant columns for flattening later.
I can use a JSON parser however I need to do that for every field and there are some cases where the fields are dynamic.
Best answer by Pete
Thanks Alex for your response.
I could not find an effective way to convert a List to a List-Map especially in a dynamic way.
My solution uses a Javascript Evaluator to loop though the keys of an object i’m targeting and where an array is found it embeds that array into a Map field. I get a static nested level for the array which is annoying but it caters for dynamic JSON and loads into Snowflake as I needed.
I hope this helps others or if someone has a better way that would be great to know :)
Thank you
Hi
List are not currently supports, but ListMaps are, so you might be able to convert to that and be able to ingest it.
Thanks Alex for your response.
I could not find an effective way to convert a List to a List-Map especially in a dynamic way.
My solution uses a Javascript Evaluator to loop though the keys of an object i’m targeting and where an array is found it embeds that array into a Map field. I get a static nested level for the array which is annoying but it caters for dynamic JSON and loads into Snowflake as I needed.
I hope this helps others or if someone has a better way that would be great to know :)
Thank you
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.
