topic Re: Merge JSON outputs > 500 objects? in API / CLI Discussion

Merge JSON outputs > 500 objects?

DanielS — Wed, 21 Jul 2021 19:24:18 GMT

Hi guys, literally tearing my hair out here (don't have much to start off with). But working with PHP to pull data from the webapi using CURL. This all works great as long as there is under 500 objects, soon as I go over the 500 object mark I'm unable to merge the arrays successfully to output a unified data view.

Any ideas on the best way to do this, I believe the issue is that we've got multi arrays { [objects] [to] [from] [total] } I tried deleting the to/from/total and merging but was unsuccessful.

Any help would be much appreciated 🙂

Re: Merge JSON outputs > 500 objects?

PhoneBoy — Wed, 21 Jul 2021 20:25:37 GMT

Curious why are you trying to create a single JSON output.
And how precisely is it failing when you try and do so.

Re: Merge JSON outputs > 500 objects?

DanielS — Wed, 21 Jul 2021 22:07:30 GMT

Hi phoneboy I have a list of IP addresses that I need to compare the database against, this would be very easy in 1.8API with the filter command.

My thoughts were to save me constantly pulling the objects, too store them in 1 big json array for processing, this is just one example and may need to be done on regular occasions hence building a tool for it.

I can that the second load of objects are being successfully pulled from the mds via a var_dump of the functions return, but array_merge just isn't working. I've tried json_decode before merge them json_encode after and the other way around but it's not wanting to go in and array size stays at 500, or it wipes out the array completely (that could have been a typo) as I was trying lots of things.

Re: Merge JSON outputs > 500 objects?

PhoneBoy — Wed, 21 Jul 2021 22:22:07 GMT

Actually, that's a very good reason for creating a merged JSON file 🙂
I believe you can use jq to do this like so:

jq -s '.[0] * .[1]' 1.json 2.json

Re: Merge JSON outputs > 500 objects?

DanielS — Thu, 22 Jul 2021 08:52:42 GMT

anyway you can think of how to do this in PHP? This is my current loop, excuse the messiness of the code.

do { if(!isset($mergedArrays)) { $mergedArrays = array(); $mergedArrays = json_decode(runAPI_Command($ip,$domainsid,$domain,$actions,$currentPosition),true); $totalObjects = $mergedArrays["total"]; $currentPosition = $mergedArrays["to"]; } else { $output = json_decode(runAPI_Command($ip,$domainsid,$domain,$actions,$currentPosition),true); $currentPosition = $output["to"]; $mergedArrays = array_merge($output["objects"],$output["to"],$output["from"],$output["total"]); } $currentPosition = $mergedArrays["to"]; $size = sizeof($mergedArrays["objects"]); echo "<br>Total Objects:$totalObjects <br>"; echo "Current Position:$currentPosition<br>"; echo "Size of Array:$size"; } while ($currentPosition < $totalObjects);

Re: Merge JSON outputs > 500 objects?

PhoneBoy — Thu, 22 Jul 2021 22:36:44 GMT

Unfortunately, I'm not as familiar with PHP.
That said, there is a JQ module for PHP, maybe that will help here?