Extract Policy Target Details across an MDS

[Alex_Tooze]

Hi,

I have an MDS with multiple domains.

I want to be able to extract a list of Domains, Packages and their installation targets for use within a script.

How would I do this from the MDS CLI?

On R77.30, we have a number of scripts which cycle through the CMAs to provide an MDS -wide view of certain things. This used MDSENV to define the relevant environment prior to interrogating files and command output for that CMA (Domain). On our current test platform, we can run the MDSENV, but it doesn't appear to set the environment for the subsequent mgmt_cli commands.

Does anybody have an example of some scripting which does something similar?

Thanks,

Alex

[Tomer_Sole]

Hi,

First you need to login to the specific domain to get its data. In the "login" call, make sure that you specify the domain name or IP, or "MDS" for the System Domain. See Login command specification here: Check Point - Management API reference 

Then, inside, you can get all the policy packages and their target gateways using the "show packages" command. See this command specification here: Check Point - Management API reference 

mdsenv is still a valid command, for some internal executions, but the R80 API simplifies it and does not need the mdsenv outcome. It works on all domains no matter which "mdsenv" you ran it with. 

[Iain_King]

Yes, scripts have been written for this kind of thing for years..

for i in $MDSDIR/customers/*; do mdsenv $i; mgmt_cli ... ; done

you can also use cpmiquerybin attr "" network_objects "type=... ..  to pull lists of objects and do batch processing.

you can also use the mdsquerydb (edit the $MDSDIR/conf/queries.conf file) and add aliases for the queries like:

CMA "" Hostlist attr network_objects "type='host .. etc

How to use the 'cpmiquerybin' command to list objects and their attributes 

mdsquerydb / cpmiquerybin [Archive] - CPUG: The Check Point User Group 

Now, for the cheat of the day.. if you are managing hundreds of CMA's with global objects.. and you have many global objects.. and it takes forever to do mdsdb processing (like batch object name changes etc).. there is a dirty trick that can speed it up dramatically.. move the entire MDSDIR/conf/mdsdb (or $MDSDIR) to a linux ramdisk.

Stop the mds, create a ramdisk (there's plenty of online guides for creating linux ramdisks) mount it.. move the $MDSDIR/conf/mdsdb (or $MDSDIR) directory to it.. then symlink it. Restart the mds.. do your processing and you will find that what might normally take hours can be done in minutes.

When it's done and working.. stop the mds services, remove the sym link and move the files back.. ace fun!

This is good to test in lab.. 

Iain

[Tomer_Sole]

https://community.checkpoint.com/people/iain.a36f712c-3b0c-435a-988a-9b5ddd1dc3f0‌ these probably will not work for R80 and above though. We simplified. 

[Iain_King]

Oh right, I only just noticed this is in the R80.10 management place. Sorry!

It's completely changed yeah, the docs for it are here.. 

Multi-Domain Security Management R80.10 (Part of Check Point Infinity) 

There are some examples from the doco below, but scripts can probably be written easily enough to do something along the lines of:

mdsquerydb CMAs | xargs mgmt_cli add host name host1 ip-address 1.2.3.4 ... servers.multi-domain-server servers.name 

or something like 

CMAs=`mdsquerydb CMAs`; MDSs=`mdsquerydb DomainManagementServers`; for i in $CMAs; do mgmt_cli add host name host1 ip-address 1.2.3.4 ... servers.name $CMAs servers.multi-domain-server $MDSs

Where a syntax example is below, it'l likely take some playing around to re-write your admin scripts from < R77.30 to R80.10:

That sounds like a fun job! 

//To create a new Domain Server:

1. Open a terminal emulation program (such as PuTTY).
2. Open an SSH connection to the Multi-Domain Server.
3. Log in with the superuser credentials.
4. Enter expert mode.
5. Run this command:

   mgmt_cli add domain name <domain_name> servers.ip address "<ipv4>" servers.name "<server_name>" servers.multi-domain-server "<mdm_name>"

   For Example:

   mgmt_cli add domain name "domain1" servers.ip-address "192.0.2.1" servers.name "domain1_ManagementServer_1" servers.multi-domain-server "primary_mdm"

   The Domain Server is created. Log in to 192.0.2.1 to configure the settings.

   //

Iain King / https://www.linkedin.com/in/iain-king-77075a1b/ 

[Alex_Tooze]

Thanks for the suggestions above - I've made some progress with this, and now able to loop through Domains & policies to carry out a full web export of all policies.

A couple more specific things I'd like to do:

1) Search through a security policy to find rules matching certain criteria (e.g. any/any/allow)

2) Identify the defined installation targets for a given policy

Is there an easy way to get this?

[Alex_Tooze]

Found a bit of information regarding finding the installation targets, but I'm having trouble getting the result I need. I want to point the query from the MDS towards a specific domain - how do I do this? The following command should do the job, but how do I aim at the specific domain. Bear in mind that I want to run this from the MDS to loop across all domains.

mgmt_cli show-package name Standard --root true --format json
{
  "code" : "generic_err_object_not_found",
  "message" : "Requested object [Standard] not found"
}

[Alex_Tooze]

Even including the management server in the call doesn't seem to work:

 mgmt_cli show-packages -u <UID> -p <PW> -m <IP> --format json
{
  "packages" : [ ],
  "total" : 0
}

[Tomer_Sole]

If this is a Multi-Domain environment then you also need to provide the relevant domain name or IP address which holds policy packages. The default domain, MDS, does not have policy packages.

[Alex_Tooze]

Thanks Tomer - this is what I did in the 2nd example above, using the -m argument with the IP of the domain management server, but the result was the same (I have hidden the credentials & IPs):

mgmt_cli show-packages -u <UID> -p <PW> -m <IP> --format json
{
  "packages" : [ ],
  "total" : 0
}

[Tomer_Sole]

OK. So apparently in Multi-Domain environments the -m switch must be the MDS IP, and the -d needs to be provided with the domain name or IP address. 

If you use the mgmt_cli command already inside the MGMT server, then you don't need to provide the -m parameter. This parameter is mostly for remote calls.

The fact that providing an IP of a server which is not the MDS is a bug and will be fixed in a later update of the API engine.

tl;dr you should have used this:

mgmt_cli show-packages -u <UID> -p <PW> -d <IP> --format json

or when calling from a remote host:

mgmt_cli show-packages -u <UID> -p <PW> -m <MDS IP>-d <IP> --format json

[Tomer_Sole]

Alex, have you succeeded with this?

[Brian_Deutmeyer · Accepted solution]

Hi Alex-

I've written automation to pull all policy packages (so our automation site stays up to date) and to install policy via the cli.  All the following should work in an MDS environment, you just need to fill in the variables.  You can also log into the API via read only if you are looking to just extract data (this way you don't need to discard your session).

To list policy packages for a domain:

mgmt_cli login -r true domain "$DOMAIN" > $DOMAIN-id-$$.txt

mgmt_cli show packages -s $DOMAIN-id-$$.txt --format json | $CPDIR/jq/jq -r '.packages[] .name'

mgmt_cli discard -s $DOMAIN-id-$$.txt
mgmt_cli logout -s $DOMAIN-id-$$.txt

To get the installation targets for a policy package:

mgmt_cli login -r true domain "$DOMAIN" > $DOMAIN-id.txt.$$

mgmt_cli show package name "$POLICY" -s $DOMAIN-id.txt.$$ --format json | $CPDIR/jq/jq -r '."installation-targets"[] .name'

mgmt_cli discard -s $DOMAIN-id.txt.$$

mgmt_cli logout -s $DOMAIN-id.txt.$$

As far as sifting through policy looking for any/any/allow, I haven't done that, so I'm not sure.

I hope this information helps!

[Iain_King]

Ace!

[Alex_Tooze]

Thanks Brian, I now have this working in a very similar way to your suggestion above. I am not currently using any session management, however. This reduces it to a single command:

mgmt_cli show-package name ${POLICY_NAME} -d $CMA --root true --format json

In terms of filtering the policy, there is a very handy 'filter' argument to 'show-access-rulebase'. In theory, I should be able to simply put my requirements into this filter in the same way that I would in SmartDashboard. Unfortunately, there are currently problems with this, and it doesn't work (I've raised an S/R for this). I have therefore had to find another way to do this. I'm nearly there with it, but it's a bit clunky.

We carry out a number of pre-install checks, and it is quite challenging translating these to R80, with the implementation of multiple layers etc.

[Tomer_Sole]

elaborate on the part of the filter that doesn't work please?

[Brian_Deutmeyer]

Please be careful without using session management.  You can reach the maximum number of allowed WEB_API slots and you won't be able to log in (I've been there).  I strongly encourage using read-only or session management to discard your active session.

[Tomer_Sole]

With R80.10 GA, the logout call should clear the session. So unless the script crashed or unless you ran it without logout during development, your session should be cleared.

Brian, the max sessions lockout is kind of the "API initiation ceremony". This means that you checked your stuff during development enough times to reach this problem

Either way this is the max sessions fixer: "You have reached the maximum number of active sessions" error in SmartConsole 

‌

[Brian_Deutmeyer]

[Alex_Tooze]

All the commands I am running (so far) are 'show', therefore shouldn't be creating sessions (I don't see any sessions building up on the console). I can see this being a real consideration when running updates, though, so thanks for the heads-up.

[Tomer_Sole]

login creates a session while logout clears a session if everything was published or discard prior to logout.

[Robert_Decker]

There is no need to call the "discard" commad if there are no changes, just "logout" command.

Robert.

[Iain_King]

I have some old code which does this by parsing the $MDSDIR/customers/*.. conf/ .. rulebases_5_0.fws etc files and looking for matches (of any / any etc). It ran in cron, I won't post it here because it's not supported and some people will get their knickers in a twist over that. PM me and i'll send it across (it's in perl). 

There is for sure a better way of doing that nowadays.. probably using mgmt_cli (but I'll need to have a closer look at the new rule handling) to figure out a good way to do it. For sure there's a better way of handling it now in R80.10.

[Alex_Tooze]

Thanks Ian,

This is pretty well exactly what we do in R77, but the question is how to do the equivalent in R80.

Regards,

Alex

[Tomer_Sole]

Hi Alex, if you can share your steps with the API we might be able to assist you, it seems you are close to getting what you were looking for...