topic Re: Getting all Hosts(ip addresses) under a big group that has other groups inside it in Firewall and Security Management

Getting all Hosts(ip addresses) under a big group that has other groups inside it

Network_Engine2 — Mon, 25 Feb 2019 13:54:20 GMT

Hi,

I am using R80.10, and need some help regarding getting ip addresses under a big group.

Lets say i have a big group that has few groups under it, and then those groups have other groups inside them, and so on and so on until i reach the objects (servers,computers, etc).

i want to get to a situation that i have all ip addresses that are under the big group.

Does anyone know/ or did do such thing in the past?

Maybe some way in the API or SQL query through posgresql?

Thank you

Re: Getting all Hosts(ip addresses) under a big group that has other groups inside it

PhoneBoy — Tue, 26 Feb 2019 03:18:58 GMT

First of all this is not a single API call but a series of API calls in iteration.

It would actually be recursive.

At a high level, you would query the relevant group for:

1. All the host objects

2. All the network objects (e.g. objects that represent subnets)—how do you want to handle these?

3. All the groups, which would be iterated and queried as was the original group.

You could probably do this with a bash script and jq.

If you take this approach, I would make sure you create a single read-only session and use that for all your queries.

Re: Getting all Hosts(ip addresses) under a big group that has other groups inside it

Maarten_Sjouw — Tue, 26 Feb 2019 14:41:27 GMT

Yeah I agree what happened to this button that was part of the Group Object definition:

Re: Getting all Hosts(ip addresses) under a big group that has other groups inside it

Network_Engine2 — Wed, 27 Feb 2019 08:36:40 GMT

That sounds like a great idea.

i know it sounds a bit too much, but i'm very rusty on scripting and these kinda methods, is there any chance you'd give a little low level example of how it would look?

Thank you so much

Re: Getting all Hosts(ip addresses) under a big group that has other groups inside it

PhoneBoy — Wed, 27 Feb 2019 17:05:26 GMT

Actually, it turns out there's a single API command that should give you every IP covered by a group (including its various sub objects).

mgmt_cli --session-id $SID --format json show group name group_name show-as-ranges true

This will break down all the objects into a range of IP addresses (with start/end ranges).

Not sure how far down the rabbit hole this goes (i.e. nesting levels), but it does appear to go down a couple levels at least.

I tested as follows:

Created a group called test group as follows:
- A few host objects
- A group containing the following:
  - A group containing some network objects (including some IPv6 objects)
  - A single network (not host) object
  - A group containing a single host object
Ran the above command

The resulting output gave me:

The ranges of all IPv4 addresses covered by the above
No IPv6 ranges were returned
An "other" section that included the IPv6 objects

Re: Getting all Hosts(ip addresses) under a big group that has other groups inside it

Network_Engine2 — Thu, 28 Feb 2019 13:27:07 GMT

Thank you so much for the reply.

but the show-as-ranges true doesn't seem to work because the parameter isn't right..?

and i don't really understand what the session id part means .

Maybe you can explain about what the command does exactly? i know what the show group name does but thats it

Re: Getting all Hosts(ip addresses) under a big group that has other groups inside it

PhoneBoy — Thu, 28 Feb 2019 23:06:34 GMT

Keep in mind the CLI is just a wrapper for API calls.

In general, the flow is something like this:

Login, which gives you a session ID
Perform tasks
If edits/additions were made, then you must publish (or discard) those changes
Logout, which clears the session

Other than the login, every call needs a session ID passed.

Or, if you're using the CLI from the Security Management server, you can avoid all that by using a -r true to the command instead

To break down my command: mgmt_cli --session-id $SID --format json show group name group_name show-as-ranges true

--session-id $SID says to use the session ID specified in the $SID shell variable (I had set this previously)
--format json says to give JSON output
show group name group_name show-as-ranges true shows the group group_name with the output in ranges (show-as-ranges true).

Unfortunately, that feature (show-as-ranges true) was added in API version 1.3, which corresponds to R80.20, not R80.10.

I should have checked that before providing you a solution, my apologies.

In any case, it's easy enough to get the members of a group using:

mgmt_cli -r true --format json --version 1.1 show group name test-group details-level full

That will output (in JSON) the list of all objects in a group.

All groups are referred to by UID.

To parse that and get all the IPv4 and IPv6 addresses in said group:

mgmt_cli -r true --format json --version 1.1 show group name test-group details-level full | jq -r '.members[] | select(.type=="host") | [."ipv4-address", ."ipv6-address"] | @csv'

Note this will be in the format IPv4 Address,IPv6 Address (an object can have both)

To get all the networks, we have to parse IPv4 and IPv6 networks and netmasks (output in this case is CSV format):

mgmt_cli -r true --format json --version 1.1 show group name test-group details-level full | jq -r '.members[] | select(.type=="network") | [.subnet4, ."mask-length4", .subnet6, ."mask-length6"] | @csv'

To get all the uids of the groups contained in my group (the API does not return names, only UIDs in this case):

mgmt_cli -r true --format json --version 1.1 show group name test-group details-level full | jq -r '.members[] | select(.type=="group") | .uid'

Now you can repeat the above using the group(s) identified except you use "uid whatever" instead of "name test-group".

Hopefully that's enough to help you get started.

Re: Getting all Hosts(ip addresses) under a big group that has other groups inside it

Network_Engine2 — Sun, 03 Mar 2019 14:11:37 GMT

So now i need to do a script that involves these commands recursively? can you give me a little example of how it would look?

Re: Getting all Hosts(ip addresses) under a big group that has other groups inside it

PhoneBoy — Mon, 04 Mar 2019 09:31:01 GMT

That's the basic idea, yes.

I would put the relevant commands into a function that calls itself.

I would also do this with a single session versus trying to do this with a bunch of -r true commands (it will improve efficiency).

There are several scripting examples in Developers (Code Hub)‌ you could borrow from.

They don't necessarily cover this exact use case, but they do show how you can work with the API.

For example CLI API Example for exporting, importing, and deleting different objects using CSV files (v 00.33.00 and later)‌

Re: Getting all Hosts(ip addresses) under a big group that has other groups inside it

Network_Engine2 — Mon, 04 Mar 2019 11:48:01 GMT

Ok, Thank you very much for everything.