topic Re: Search multiple CMA in API / CLI Discussion

Search multiple CMA

Harald_Hansen — Mon, 12 Mar 2018 14:12:41 GMT

In R77.30 one had the possibility to search all CMAs for object usage. I cannot find this feature in R80.10 MDS. There is no mention of the Cross-Domain Management Server Search function in the CP_R80.10_Multi-DomainSecurityManagement_AdminGuide.

This was especially useful to find global object use, to limit the number of policies one had to push after updating a single object.

Any suggestions for a replacement?

Best regards,

Harald

Re: Search multiple CMA

Robert_Decker — Mon, 12 Mar 2018 16:31:17 GMT

Hi Harald,

This task can be accomplished by using several Management API commands in a bash script.

If I'll have some time, maybe I'll post this code during a week.

Robert.

Re: Search multiple CMA

Robert_Decker — Tue, 20 Mar 2018 10:01:38 GMT

Hi Harald,

I'm very short in time nowdays and cannot find time to come to this, but if you are fluent with bash scripting -

Please take a look at this post - https://community.checkpoint.com/docs/DOC-2788 from @Ivan Moore, he has done a great work related to MDS automation.

It involves bash scripting along with API calls and GAIA calls to accomplish useful MDS tasks.

For a given domain, he loops over policy packages in that domain and retrieves some useful information.

You can combine this code with my post here - https://community.checkpoint.com/thread/7116-r8010-where-used-object and Ivan's post here - https://community.checkpoint.com/thread/7143-is-there-a-equivalent-command-to-a-global-policy-assignment-with-install.

As I promised, I'll find time to post this script myself, but if it is urgent for you, you may proceed meanwhile.

Robert.

Re: Search multiple CMA

Robert_Decker — Wed, 04 Sep 2019 16:15:55 GMT

Hi,

Please find attached an example script that uses R80.10 management API along with bash scripting to find usages of object X over all CMAs in MDM.

#!/bin/sh JQ=${CPDIR}/jq/jq OBJECT_NAME=$1 DOMAINS_FILE="domains.json" PACKAGES_FILE="packages.json" PACKAGE_FILE="package.json" echo 'Getting a list of domains...' mgmt_cli -r true -d MDS show domains limit 500 --format json > $DOMAINS_FILE if [ $? -eq 1 ]; then echo "Error getting list of domains. Aborting!" exit 1 fi DOMAINS_NAMES=($($JQ -r ".objects[] | .name" $DOMAINS_FILE)) echo 'Searching for object '"$OBJECT_NAME"' in all domains...' FOUND=0 OBJECT_UID="" for DOMAIN in ${DOMAINS_NAMES[@]} do echo 'Searching in domain '"$DOMAIN"'...' mgmt_cli -r true -d "$DOMAIN" show objects offset 0 limit 1 in.1 name in.2 "$OBJECT_NAME" --format json > $OBJECT_NAME.json if [ $? -ne 1 ]; then OBJECT_COUNT=$($JQ -r ".total" $OBJECT_NAME.json) if [ $OBJECT_COUNT -ne 0 ]; then FOUND=1 OBJECT_UID=$($JQ -r ".objects[0].uid" $OBJECT_NAME.json) echo 'Found in domain '"$DOMAIN"'!!!' break fi fi done if [ $FOUND -ne 1 ]; then echo 'Object '"$OBJECT_NAME"' does not exist. Aborting!' exit 1 fi echo 'Searching for object '"$OBJECT_NAME"' usages in all policy packages in all domains...' for DOMAIN in ${DOMAINS_NAMES[@]} do echo 'Searching in domain '"$DOMAIN"'...' mgmt_cli -r true -d "$DOMAIN" show packages limit 500 --format json > $PACKAGES_FILE if [ $? -ne 1 ]; then PACKAGES_NAMES=($($JQ -r ".packages[] | .name" $PACKAGES_FILE)) for PACKAGE in ${PACKAGES_NAMES[@]} do echo 'Searching in package '"$PACKAGE"'...' mgmt_cli -r true -d "$DOMAIN" show-package name $PACKAGE --format json > $PACKAGE_FILE if [ $? -ne 1 ]; then ACCESS_LAYERS=($($JQ '.["access-layers"][] | .name' -r $PACKAGE_FILE)) for LAYER in ${ACCESS_LAYERS[@]} do mgmt_cli -r true -d "$DOMAIN" show access-rulebase package "$PACKAGE" name "$LAYER" offset 0 limit 1 filter $OBJECT_UID --format json > $OBJECT_NAME.json if [ $? -ne 1 ]; then OBJECT_COUNT=$($JQ -r ".total" $OBJECT_NAME.json) if [ $OBJECT_COUNT -ne 0 ]; then echo 'The requested object is used in policy package '"$PACKAGE" break fi fi done fi done fi done echo 'Done!'

You should provide an object name as a parameter for this script.

This is only an example, and you may change it as you need.

Hope this helps.

Robert.

Re: Search multiple CMA

Harald_Hansen — Mon, 23 Apr 2018 09:33:38 GMT

Thank you. I have not been able to test it before now, it worked as a charm.

Re: Search multiple CMA

Harald_Hansen — Mon, 23 Apr 2018 10:49:10 GMT

Btw, you need to check JQ for $? -ne 0 in all uses, for instance in line 61. The API doesn't handle duplicate names very gracefully.

I was also stumped that the filter option returns group memberships on global rules. These rules are not always relevant, I'm going to check if rule-number contains a dot "." and one needs to expand the limit to more than 1.

Re: Search multiple CMA

Sven_Glock — Wed, 09 May 2018 12:35:57 GMT

Hi Robert,

thanks for your work!

I am missing the cross domain search, too.

So I was very happy to find Haralds initiative here.

The first test of your script led to some errors:

Searching in domain xyz...
Searching in package A...
parse error: Invalid literal at line 120, column 10

...

Searching in package D...
parse error: Unmatched '}' at line 123, column 3

I am not a developer. Is this something that needs to be improved in the code or is there a problem with my database?

My first search was about an global object.

This causes the following result:

{
"code" : "generic_err_object_field_not_unique",
"message" : "Requested object name [Network] is not unique."
}

Are global objects not supported?

Thanks in advance.

Sven

Re: Search multiple CMA

Robert_Decker — Wed, 09 May 2018 13:08:36 GMT

Hi Sven,

The code above is only an example.

The code does not search for global objects. To fix this, please add the following after line 17 -

DOMAINS_NAMES+=('Global')‍

This will also search in Global domain's database.

The parsing errors are due to the data in the responses from your database. For some reason, JQ tool fails to parse the data. The data is saved inside temporary files as noted at the top of the script (I suspect in package.json file). You may look at this data and verify the data inconsistency at the specified lines.

The "Requested object name [Network] is not unique" error is probably due to the fact that I use access-layer names instead of names+UIDs. If you have a global policy assigned in any of the domains, there may be two layers with a name 'Network'. To make this work, the code needs to save layer UID along with the layer name. A fix for this requires a bit more coding. I'll try to find some time and fix it later on.

Robert.

Re: Search multiple CMA

Sven_Glock — Wed, 09 May 2018 14:30:40 GMT

...looking forward to see your next post!

Re: Search multiple CMA

Sven_Glock — Fri, 11 May 2018 12:15:11 GMT

I strongly recommend to use session ID to log in so that you can logout at the end of your job.

Otherwise there will stay plenty of dead sessions in your management!

#!/bin/sh

JQ=${CPDIR}/jq/jq

sid=$(mgmt_cli login -r true --format json | jq -r '.sid')

<here will be your code>


# Logout CMA
mgmt_cli logout --session-id "$sid"
‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Re: Search multiple CMA

Robert_Decker — Fri, 11 May 2018 15:58:05 GMT

yes, you are correct!

robert.

Re: Search multiple CMA

Olavi_Lentso — Fri, 18 May 2018 08:50:26 GMT

Still, does the cross cma search return in R80.XX?

The API is nice to have, but it is way too slow compared to R77.30 mdscmd runcrossdomainquery.

Re: Search multiple CMA

Andreas_Lorenze — Mon, 14 Jan 2019 10:36:55 GMT

Hi Robert,

after migrating to R80 for production we realized the Cross Domain search is missing. To be honest this is mandantory and logic you need it for multiple domains. And we need all options. You need to check occurence of Global objects and only local occurence. Use in rule in object etc. So primarily we were using 2/3 of the options cross domain search offering.
We pay a lot of money for this product and I am not willing to waste my time playing a script kiddy, just because someone at Checkpoint has missed to migrate this feature (can't believe it anyway).
So PLEASE make this re-appearing SHORTLY.
Thx in advance

Re: Search multiple CMA

Amiad_Stern — Tue, 15 Jan 2019 22:24:12 GMT

Hi Andreas, Robert just suggested a replacement as was asked by Harald. I will make sure our PJM will get your feedback.

Regards,

Amiad.

Re: Search multiple CMA

JoseOriundo — Thu, 05 Dec 2019 16:08:05 GMT

Hi Robert,

We are new to R80.20 and need some help with your script.

Is there a way to search for IP addresses rather than object names?

Also, your script searches for the object name in all CMAs, but in addition to that it also searches you for each policy and this could take some time if you are looking for multiple objects.

I would like to upgrade our script and use an updated version designed for R80 versions.

Attached is the simple script we are currently using for the time being.

Also, do you see a problem with us using this simple script in R80.20?

Jose Oriundo