How to select a subset of properties in a list of objects

Application ID

4A47197B-AE30-FA84-FF56-0071F4010900

Expected Behavior

I would like to pull a list of objects from an external API and then be able to create a new list with a subset of the properties in the initial JSON.

When I get the JSON, it looks like this:

 [
{
	"contacts": [
		{
			"date_updated": "2020-12-18T13:38:11.809000+00:00",
			"date_created": "2020-08-26T11:05:04.198000+00:00",
			"integration_links": [
				{
					"name": "Bio",
					"url": "http://XXX"
				},
				{
					"name": "Video",
					"url": "http://video.XXX"
				},
				{
					"name": "Send Document via PandaDoc",
					"url": "XXX"
				}
			],
			"phones": [
				{
					"country": "GB",
					"phone": "+44XXX",
					"phone_formatted": "+44XXX",
					"type": "mobile"
				}
			],
			"custom.ccf_F7atO93CXpppO74qw6bZRot4J6Y4KOHXK7zVwOzkI7U": "2020-12-17",
			"title": "",
			"display_name": "XXX",
			"created_by": "user_8ym1NFSM77aDdlFCOMjdhkPgUaX4hqfckmdkx22cWPa",
			"custom.ccf_JQfEM4g49NVpBMOpetyBenCRXZJMTW9P5s3fANgOYlE": [
				"//Thu...17"
			],
			"custom.ccf_IXfh1a89OXSkbcjAIbMeG4MXcSsoTBWJHu1HskI4Odl": "2020-12-17",
			"emails": [
				{
					"type": "office",
					"email": "XXX"
				}
			],
			"custom.ccf_dmJMdFxvfjz3EomR7MZPB9ogp5uOElqEAc8rix9aQRN": [
				"Math"
			],
			"custom.ccf_joLD4t0eCqLhzqKLtKQBbm8WLS0DkQMrKpbGVrBAdGA": [
				"MAX2"
			],
			"urls": [],
			"organization_id": "orga_7lUFr4azrjwtX3Ls2aZM14Yt8n369T25XNNlk6J9Y6C",
			"updated_by": "user_8ym1NFSM77aDdlFCOMjdhkPgUaX4hqfckmdkx22cWPa",
			"name": "XXX",
			"id": "cont_9vfqGY0zUNqh0fZIcfb9QRMrewaqDlPWX8RDMzHL1XZ",
			"custom.ccf_B0nzghIi86909vNLANoDEqSBY8cf3AMQcHWQq8EVBim": [
				"XXX"
			],
			"lead_id": "lead_pfhaTocTsau7upz1fHI4fC7EmyIURGrHH3U9FlLPmSo",
			"custom.ccf_HhFzm0bLKAhy9OXFzCD8ojjFcJbUqFF3VhNmIxQg91N": [
				"Grade 10"
			]
		}
	]
},
{
	"contacts": [
		{
			"date_updated": "2020-12-20T14:00:47.071000+00:00",
			"date_created": "2020-11-26T01:49:23.171000+00:00",
			"integration_links": [
				{
					"name": "Bio",
					"url": "http://bio.brilliantgrades.com/XXX"
				},
				{
					"name": "Video",
					"url": "http://video.brilliantgrades.com/XXX"
				},
				{
					"name": "Send Document via PandaDoc",
					"url": "XXX"
				}
			],
			"phones": [
				{
					"country": "US",
					"phone": "+XXX",
					"phone_formatted": "+1 XXX",
					"type": "mobile"
				}
			],
			"custom.ccf_F7atO93CXpppO74qw6bZRot4J6Y4KOHXK7zVwOzkI7U": "2020-12-20",
			"title": "",
			"display_name": "XXX",
			"created_by": "user_8ym1NFSM77aDdlFCOMjdhkPgUaX4hqfckmdkx22cWPa",
			"custom.ccf_JQfEM4g49NVpBMOpetyBenCRXZJMTW9P5s3fANgOYlE": [
				"/Sun...14"
			],
			"custom.ccf_IXfh1a89OXSkbcjAIbMeG4MXcSsoTBWJHu1HskI4Odl": "2020-11-29",
			"emails": [
				{
					"type": "office",
					"email": "XXX"
				}
			],
			"custom.ccf_dmJMdFxvfjz3EomR7MZPB9ogp5uOElqEAc8rix9aQRN": [
				"ACT's"
			],
			"custom.ccf_joLD4t0eCqLhzqKLtKQBbm8WLS0DkQMrKpbGVrBAdGA": [
				"MAX1"
			],
			"urls": [],
			"organization_id": "orga_7lUFr4azrjwtX3Ls2aZM14Yt8n369T25XNNlk6J9Y6C",
			"updated_by": "user_8ym1NFSM77aDdlFCOMjdhkPgUaX4hqfckmdkx22cWPa",
			"name": "XXX",
			"id": "cont_1dFTP6VlPY5Ym3s3fLlFDdlO368q6EhrSkUztHESanY",
			"custom.ccf_B0nzghIi86909vNLANoDEqSBY8cf3AMQcHWQq8EVBim": [
				"XXX"
			],
			"lead_id": "lead_hRaS99I1NCTg2UAevXQ4JxXtByBYxokMk9KEnVG8ijL",
			"custom.ccf_HhFzm0bLKAhy9OXFzCD8ojjFcJbUqFF3VhNmIxQg91N": [
				"Grade 9"
			]
		}
	]
}  ]

On a high level, I am looking to collect some of these properties which are located on different JSON paths and put them into 3 different tables in Backendless.

For now, I am trying to create a list of contacts with fewer properties.

I then want to add the contacts with the selected properties into a table.

As you can see, my data contains objects of arrays…of objects of arrays. I am trying to figure out how to specify JSON fields I need in an array using Codeless, a bit like using JSONPath.

Here’s what I have so far:

You can disregard the fact that I have return:enrolledStudents as I have been playing with that while debugging. When this is set to enrolledStudents1, it gives an empty array.

I am missing something and I am frustrated with myself. Is there a rule of thumb I can follow? eg. I know that when I want to get a list that is nested under a root object, I must use:

However, when there is a list of objects which contain arrays, how do I predictably capture the properties of those arrays that are 2 levels down?

Hello, @Andreas_Marinopoulos.

I would advise you to insert Print blocks to see intermediate results.
Also the first Set Enrolled Students will be presented as an array of arrays, as a result the next map will not work as you need. You need to concatenate arrays or make a selection of the first element.

Regards, Nikita.

Can you kindly let me know how to concatenate arrays in Codeless?

@Andreas_Marinopoulos, here is an example of how you can do it (2 arrays are concatenated, one with letters, the other with numbers):
image

This does not achieve what I need. Is there a way to use JSON dot notation in Codeless or another way to access a specific property or list of properties in an array of arrays?

Hi @Andreas_Marinopoulos,

Using the Map items in list is the best approach to iterate over a collection and pick and choose the properties you need to keep or leave out.

You cannot use the dot notation in codeless, because each block operates strictly on the object/array it represents.

As for the question below, I am having a hard time wrapping my head around the question. Could you possibly illustrate the question with an example showing what specifically you need to extract given a sample JSON document?

Thank you @mark-piller, yes for sure.

In the question you quoted above, I am citing a JSON format that contains a list of contacts. Each contact has a lot of properties and some of the properties are themselves arrays of objects or arrays of arrays.

What I want to do at this step is get the properties in my ‘map items block’ here:

My final objective is to use bulk create to fill this table, then keep the table updated with the latest information coming from the external API:

But it feels really hard to do, since for example the email and phone number are still one level down in the array hierarchy.

The enrolledStudent1 variable is an overkill. You can simply move the Create Object block to the return connector:

Have you verified if enrolledStudents has all the data you expected? You can use the print block to log it and then see the data in the Real-Time Logging window (accessible through the Business Logic screen).

Regards,
Mark

So I am not sure if I need to remove the “contacts” root object or not. The enrolledStudents variable, as I now have it, is intented to remove the root object “contacts” and gives an array that looks like this:

[
[
    {
        "date_updated": "2020-12-21T01:28:26.167000+00:00",
        "date_created": "2020-12-05T00:55:56.558000+00:00",
        "integration_links": [
            {
                "name": "Bio",
                "url": "http://bio.brilliantgrades.com/XX"
            },
            {
                "name": "Video",
                "url": "http://video.brilliantgrades.com/XX"
            },
            {
                "name": "Send Document via PandaDoc",
                "url": "https://app.pandadoc.com/integrations/closeio/new?contact=XX"
            }
        ],
        "phones": [
            {
                "country": "US",
                "phone": "+XX",
                "phone_formatted": "+1XX",
                "type": "mobile"
            }
        ],
        "custom.ccf_F7atO93CXpppO74qw6bZRot4J6Y4KOHXK7zVwOzkI7U": "2020-12-12",
        "display_name": "XXX",
        "title": "",
        "created_by": "user_i9ptLeXflvh3ct16sRiOfZeplwhKsQgNFEZ0lAdGwRW",
        "custom.ccf_JQfEM4g49NVpBMOpetyBenCRXZJMTW9P5s3fANgOYlE": [
            "/Sat....9"
        ],
        "custom.ccf_IXfh1a89OXSkbcjAIbMeG4MXcSsoTBWJHu1HskI4Odl": "2020-12-21",
        "emails": [
            {
                "type": "office",
                "email": "XX@yahoo.com"
            }
        ],
        "custom.ccf_dmJMdFxvfjz3EomR7MZPB9ogp5uOElqEAc8rix9aQRN": [
            "English"
        ],
        "custom.ccf_joLD4t0eCqLhzqKLtKQBbm8WLS0DkQMrKpbGVrBAdGA": [
            "MAX1"
        ],
        "urls": [],
        "organization_id": "orga_7lUFr4azrjwtX3Ls2aZM14Yt8n369T25XNNlk6J9Y6C",
        "updated_by": "user_8ym1NFSM77aDdlFCOMjdhkPgUaX4hqfckmdkx22cWPa",
        "name": "XX",
        "id": "cont_cCgEYOQoF7eeKXWaaTnVoOaRXcKQwlHpshuMQR2BAm3",
        "custom.ccf_B0nzghIi86909vNLANoDEqSBY8cf3AMQcHWQq8EVBim": [
            "XX XX"
        ],
        "lead_id": "lead_qVRqKEwNlIVXjmtQPvoRFAT2ce0adQtYh9lvGWiRv4B",
        "custom.ccf_HhFzm0bLKAhy9OXFzCD8ojjFcJbUqFF3VhNmIxQg91N": [
            "Grade 9"
        ]
    },
[
[
    {
        "date_updated": "2020-12-21T01:28:26.167000+00:00",
        "date_created": "2020-12-05T00:55:56.558000+00:00",
        "integration_links": [
            {
                "name": "Bio",
                "url": "http://bio.brilliantgrades.com/XX"
            },
            {
                "name": "Video",
                "url": "http://video.brilliantgrades.com/XX"
            },
            {
                "name": "Send Document via PandaDoc",
                "url": "https://app.pandadoc.com/integrations/closeio/new?contact=XX"
            }
        ],
        "phones": [
            {
                "country": "US",
                "phone": "+XX",
                "phone_formatted": "+1XX",
                "type": "mobile"
            }
        ],
        "custom.ccf_F7atO93CXpppO74qw6bZRot4J6Y4KOHXK7zVwOzkI7U": "2020-12-12",
        "display_name": "XXX",
        "title": "",
        "created_by": "user_i9ptLeXflvh3ct16sRiOfZeplwhKsQgNFEZ0lAdGwRW",
        "custom.ccf_JQfEM4g49NVpBMOpetyBenCRXZJMTW9P5s3fANgOYlE": [
            "/Sat....9"
        ],
        "custom.ccf_IXfh1a89OXSkbcjAIbMeG4MXcSsoTBWJHu1HskI4Odl": "2020-12-21",
        "emails": [
            {
                "type": "office",
                "email": "XX@yahoo.com"
            }
        ],
        "custom.ccf_dmJMdFxvfjz3EomR7MZPB9ogp5uOElqEAc8rix9aQRN": [
            "English"
        ],
        "custom.ccf_joLD4t0eCqLhzqKLtKQBbm8WLS0DkQMrKpbGVrBAdGA": [
            "MAX1"
        ],
        "urls": [],
        "organization_id": "orga_7lUFr4azrjwtX3Ls2aZM14Yt8n369T25XNNlk6J9Y6C",
        "updated_by": "user_8ym1NFSM77aDdlFCOMjdhkPgUaX4hqfckmdkx22cWPa",
        "name": "XX",
        "id": "cont_cCgEYOQoF7eeKXWaaTnVoOaRXcKQwlHpshuMQR2BAm3",
        "custom.ccf_B0nzghIi86909vNLANoDEqSBY8cf3AMQcHWQq8EVBim": [
            "XX XX"
        ],
        "lead_id": "lead_qVRqKEwNlIVXjmtQPvoRFAT2ce0adQtYh9lvGWiRv4B",
        "custom.ccf_HhFzm0bLKAhy9OXFzCD8ojjFcJbUqFF3VhNmIxQg91N": [
            "Grade 9"
        ]
    }
]
]

Then the enrolledStudents1 is currently indented to give the same array but with fewer properties (so that I can include them in the table. But instead, it is printing:

]
{},
{}

]

19:47:43 | SERVER_CODE | INFO | [18891] [5EBEAC13-0D54-6F95-FFB3-9D18F9D90600] [INVOKE SERVICE] services.getParents.getStudents

19:47:44 | SERVER_CODE | INFO | [18891] [ { date_updated: undefined, date_created: undefined, phones: undefined, display_name: undefined, emails: undefined, tutor: undefined }, { date_updated: undefined, date_created: undefined, phones: undefined, display_name: undefined, emails: undefined, tutor: undefined } ]
19:47:44 | SERVER_CODE | INFO | [18891] Processing finished in 983.713ms

So I am doing something wrong.

I put the JSON document from your previous message into here with an intent to see it more graphically but it says it is not a valid JSON document. Could you please copy and paste the entire document just so we have a good solid starting point?

Sure, I think one of the brackets was missing in the previous copy/paste, here goes:

[
[
    {
        "date_updated": "2020-12-21T15:55:48.925000+00:00",
        "date_created": "2020-11-17T07:23:32.908000+00:00",
        "integration_links": [
            {
                "name": "Bio",
                "url": "http://bio.brilliantgrades.com/XX"
            },
            {
                "name": "Video",
                "url": "http://video.brilliantgrades.com/XX"
            },
            {
                "name": "Send Document via PandaDoc",
                "url": "https://app.pandadoc.com/integrations/closeio/new?contact=XX"
            }
        ],
        "phones": [
            {
                "country": "GB",
                "phone": "+XX",
                "phone_formatted": "+44 XX XX",
                "type": "mobile"
            }
        ],
        "custom.ccf_F7atO93CXpppO74qw6bZRot4J6Y4KOHXK7zVwOzkI7U": "2020-12-21",
        "title": "",
        "display_name": "XX",
        "created_by": "user_8ym1NFSM77aDdlFCOMjdhkPgUaX4hqfckmdkx22cWPa",
        "custom.ccf_JQfEM4g49NVpBMOpetyBenCRXZJMTW9P5s3fANgOYlE": [
            "///Mon....18"
        ],
        "custom.ccf_IXfh1a89OXSkbcjAIbMeG4MXcSsoTBWJHu1HskI4Odl": "2020-12-21",
        "emails": [
            {
                "type": "office",
                "email": "XX@gmail.com"
            }
        ],
        "custom.ccf_dmJMdFxvfjz3EomR7MZPB9ogp5uOElqEAc8rix9aQRN": [
            "English"
        ],
        "custom.ccf_joLD4t0eCqLhzqKLtKQBbm8WLS0DkQMrKpbGVrBAdGA": [
            "MAX1"
        ],
        "urls": [],
        "organization_id": "orga_7lUFr4azrjwtX3Ls2aZM14Yt8n369T25XNNlk6J9Y6C",
        "name": "XX",
        "updated_by": "user_8ym1NFSM77aDdlFCOMjdhkPgUaX4hqfckmdkx22cWPa",
        "id": "cont_eDpHrpJC3haAR0VXuPJKnxRhq9x7pEB0ogR0E4FWHBQ",
        "custom.ccf_B0nzghIi86909vNLANoDEqSBY8cf3AMQcHWQq8EVBim": [
            "XX XX"
        ],
        "lead_id": "lead_prGL3BmFRqjj8sQ44rO37alzNkVTmqMhr6D32RZffX2",
        "custom.ccf_HhFzm0bLKAhy9OXFzCD8ojjFcJbUqFF3VhNmIxQg91N": [
            "Grade 9"
        ]
    }
],
[
    {
        "date_updated": "2020-12-19T19:21:35.346000+00:00",
        "date_created": "2020-09-30T15:42:59.134000+00:00",
        "integration_links": [
            {
                "name": "Bio",
                "url": "http://bio.brilliantgrades.com/XX"
            },
            {
                "name": "Video",
                "url": "http://video.brilliantgrades.com/XX"
            },
            {
                "name": "Send Document via PandaDoc",
                "url": "https://app.pandadoc.com/integrations/closeio/new?contact=XX"
            }
        ],
        "phones": [
            {
                "country": "GB",
                "phone": "+XX",
                "phone_formatted": "+44 XX XX",
                "type": "office"
            }
        ],
        "custom.ccf_F7atO93CXpppO74qw6bZRot4J6Y4KOHXK7zVwOzkI7U": "2020-12-19",
        "title": "",
        "display_name": "XX",
        "created_by": "user_AVslHTZd0fMLpN0QYiNLDJPxuHFBTjfiB6IkWx48XBx",
        "custom.ccf_JQfEM4g49NVpBMOpetyBenCRXZJMTW9P5s3fANgOYlE": [
            "/Sat...11"
        ],
        "custom.ccf_IXfh1a89OXSkbcjAIbMeG4MXcSsoTBWJHu1HskI4Odl": "2020-12-19",
        "emails": [
            {
                "type": "office",
                "email": "XX@yahoo.co.uk"
            }
        ],
        "custom.ccf_dmJMdFxvfjz3EomR7MZPB9ogp5uOElqEAc8rix9aQRN": [
            "Biology"
        ],
        "custom.ccf_joLD4t0eCqLhzqKLtKQBbm8WLS0DkQMrKpbGVrBAdGA": [
            "MAX1"
        ],
        "urls": [],
        "organization_id": "orga_7lUFr4azrjwtX3Ls2aZM14Yt8n369T25XNNlk6J9Y6C",
        "name": "XX",
        "updated_by": "user_8ym1NFSM77aDdlFCOMjdhkPgUaX4hqfckmdkx22cWPa",
        "id": "cont_xBlcHXAy6DBiQq5PMTyxHl6AYur7qRbIA4jLdaBCe4e",
        "custom.ccf_B0nzghIi86909vNLANoDEqSBY8cf3AMQcHWQq8EVBim": [
            "XX XX"
        ],
        "lead_id": "lead_t7t7kb0FnLXJCxoeI7N7R6PaMOzKerZSuXCsFrRII0v",
        "custom.ccf_HhFzm0bLKAhy9OXFzCD8ojjFcJbUqFF3VhNmIxQg91N": [
            "Grade 8"
        ]
    },
    {
        "date_updated": "2020-12-19T09:02:36.526000+00:00",
        "date_created": "2020-07-15T11:36:12.561000+00:00",
        "integration_links": [
            {
                "name": "Bio",
                "url": "http://bio.brilliantgrades.com/XX"
            },
            {
                "name": "Video",
                "url": "http://video.brilliantgrades.com/XX"
            },
            {
                "name": "Send Document via PandaDoc",
                "url": "https://app.pandadoc.com/integrations/closeio/new?contact=XXX"
            }
        ],
        "phones": [
            {
                "country": "GB",
                "phone": "+XXX",
                "phone_formatted": "+44 XXX XX",
                "type": "mobile"
            }
        ],
        "custom.ccf_F7atO93CXpppO74qw6bZRot4J6Y4KOHXK7zVwOzkI7U": "2020-12-19",
        "title": "",
        "display_name": "XX",
        "created_by": "user_8ym1NFSM77aDdlFCOMjdhkPgUaX4hqfckmdkx22cWPa",
        "custom.ccf_JQfEM4g49NVpBMOpetyBenCRXZJMTW9P5s3fANgOYlE": [
            "/Sat...12"
        ],
        "custom.ccf_IXfh1a89OXSkbcjAIbMeG4MXcSsoTBWJHu1HskI4Odl": "2020-11-16",
        "emails": [
            {
                "type": "direct",
                "email": "XX@yahoo.co.uk"
            }
        ],
        "custom.ccf_dmJMdFxvfjz3EomR7MZPB9ogp5uOElqEAc8rix9aQRN": [
            "Physics"
        ],
        "custom.ccf_joLD4t0eCqLhzqKLtKQBbm8WLS0DkQMrKpbGVrBAdGA": [
            "MAX1"
        ],
        "urls": [],
        "organization_id": "orga_7lUFr4azrjwtX3Ls2aZM14Yt8n369T25XNNlk6J9Y6C",
        "name": "XX",
        "updated_by": "user_8ym1NFSM77aDdlFCOMjdhkPgUaX4hqfckmdkx22cWPa",
        "id": "cont_3yecbm25Y4BgOaRDV169Cp0hZ3zfELLo2ETjgXBkBMV",
        "custom.ccf_B0nzghIi86909vNLANoDEqSBY8cf3AMQcHWQq8EVBim": [
            "XX XX"
        ],
        "lead_id": "lead_t7t7kb0FnLXJCxoeI7N7R6PaMOzKerZSuXCsFrRII0v",
        "custom.ccf_HhFzm0bLKAhy9OXFzCD8ojjFcJbUqFF3VhNmIxQg91N": [
            "Grade 8"
        ]
    }
] ]

Thank you, that’s better. So we have an array of arrays. Each enclosed array contains one or more objects of the following structure:

date_updated : String
date_created : String
integration_links : array of objects with `name` and `url` props
phones : array of objects with `country`, `phone`, etc props
title : String
display_name : String
urls : an array
name : String
updated_by : String
id : String
bunch of `custom.xxxx` props

If I understood you correctly, you need to extract all these objects from the individual 2nd level arrays into one collection. Is that correct?

1 Like

Yes that is correct

So your algorithm should be this:

  1. create a list that will accumulate the results
  2. iterate over the first array/list, for each element X in the array:
  3. iterate over objects in X using the map items in list block. For each element in X:
  4. extract the properties you need.
  5. all elements in the result of (3) need to be added to (1)

ok I can try this. For 1, I just create an empty list. For 2-4 do you have any tips on what I am doing wrong above?

What I didn’t see you doing is iterating over the top-level array.

hmm I am missing something as I am not sure what to do differently. should I use the map items block for that? I will give it a try

I’d do something along these lines/blocks:

2 Likes

Thank you for showing me jsonformatter.org, it is helpful. It is interesting that when I do this:

I go from this:
image

to this:

image

This means tha I manage to do (1) above, ie. iterate over the first array/list, for each element X in the array, but the variable erases prior elements X and keeps the last one. At least I can hop down a step in the array structure, which is promising.

Then I want to make it iterate over each element X but keeping each element, as you suggest, so that the screenshot above looks like:

array [1]
0 {23}
1 {21}

So then I try a small step forward:

but I get this error:

{
    "code": 0,
    "message": "Cannot read property 'concat' of undefined",
    "errorData": {}
}

If I go further and do this


I get the same error.

I also tried

and got

{
    "code": 0,
    "message": "j.map is not a function",
    "errorData": {}
}

I also tried

and got

200 - OK
Body: []

Any thoughts?

I find it more difficult to follow what’s going on in the logic you describe than sitting down and implementing it myself :wink: however, this is where support stops, and consulting begins. I recommend starting using the print block so you can inspect what each variable has at any point of time and based on that add additional logic. When you add the print block, you can see it results in the Real-Time Logging Window.

Regards,
Mark