Search multiple CMA

[Harald_Hansen]

In R77.30 one had the possibility to search all CMAs for object usage. I cannot find this feature in R80.10 MDS. There is no mention of the Cross-Domain Management Server Search function in the CP_R80.10_Multi-DomainSecurityManagement_AdminGuide.

This was especially useful to find global object use, to limit the number of policies one had to push after updating a single object.

Any suggestions for a replacement?

Best regards,

Harald

[Robert_Decker]

Hi Harald,

This task can be accomplished by using several Management API commands in a bash script.

If I'll have some time, maybe I'll post this code during a week.

Robert.

[Robert_Decker]

Hi Harald,

I'm very short in time nowdays and cannot find time to come to this, but if you are fluent with bash scripting -

Please take a look at this post - https://community.checkpoint.com/docs/DOC-2788 from @Ivan Moore, he has done a great work related to MDS automation.

It involves bash scripting along with API calls and GAIA calls to accomplish useful MDS tasks.

For a given domain, he loops over policy packages in that domain and retrieves some useful information.

You can combine this code with my post here - https://community.checkpoint.com/thread/7116-r8010-where-used-object and Ivan's post here - https://community.checkpoint.com/thread/7143-is-there-a-equivalent-command-to-a-global-policy-assign....

As I promised, I'll find time to post this script myself, but if it is urgent for you, you may proceed meanwhile.

Robert.

[Robert_Decker · Accepted solution]

Hi,

Please find attached an example script that uses R80.10 management API along with bash scripting to find usages of object X over all CMAs in MDM.

#!/bin/sh

JQ=${CPDIR}/jq/jq

OBJECT_NAME=$1
DOMAINS_FILE="domains.json"
PACKAGES_FILE="packages.json"
PACKAGE_FILE="package.json"

echo 'Getting a list of domains...'
mgmt_cli -r true -d MDS show domains limit 500 --format json > $DOMAINS_FILE
if [ $? -eq 1 ]; then
  echo "Error getting list of domains. Aborting!"
  exit 1
fi

DOMAINS_NAMES=($($JQ -r ".objects[] | .name" $DOMAINS_FILE))

echo 'Searching for object '"$OBJECT_NAME"' in all domains...'
FOUND=0
OBJECT_UID=""

for DOMAIN in ${DOMAINS_NAMES[@]}
do
  echo 'Searching in domain '"$DOMAIN"'...'
  mgmt_cli -r true -d "$DOMAIN" show objects offset 0 limit 1 in.1 name in.2 "$OBJECT_NAME" --format json > $OBJECT_NAME.json
  if [ $? -ne 1 ]; then
    OBJECT_COUNT=$($JQ -r ".total" $OBJECT_NAME.json)
    if [ $OBJECT_COUNT -ne 0 ]; then
      FOUND=1
      OBJECT_UID=$($JQ -r ".objects[0].uid" $OBJECT_NAME.json)
      echo 'Found in domain '"$DOMAIN"'!!!'
      break
    fi
  fi
done

if [ $FOUND -ne 1 ]; then
  echo 'Object '"$OBJECT_NAME"' does not exist. Aborting!'
  exit 1
fi

echo 'Searching for object '"$OBJECT_NAME"' usages in all policy packages in all domains...'
for DOMAIN in ${DOMAINS_NAMES[@]}
do
  echo 'Searching in domain '"$DOMAIN"'...'
  mgmt_cli -r true -d "$DOMAIN" show packages limit 500 --format json > $PACKAGES_FILE
  if [ $? -ne 1 ]; then
    PACKAGES_NAMES=($($JQ -r ".packages[] | .name" $PACKAGES_FILE))
    for PACKAGE in ${PACKAGES_NAMES[@]}
    do
      echo 'Searching in package '"$PACKAGE"'...'
      mgmt_cli -r true -d "$DOMAIN" show-package name $PACKAGE --format json > $PACKAGE_FILE
      if [ $? -ne 1 ]; then
        ACCESS_LAYERS=($($JQ '.["access-layers"][] | .name' -r $PACKAGE_FILE))
        for LAYER in ${ACCESS_LAYERS[@]}
        do
          mgmt_cli -r true -d "$DOMAIN" show access-rulebase package "$PACKAGE" name "$LAYER" offset 0 limit 1 filter $OBJECT_UID --format json > $OBJECT_NAME.json
          if [ $? -ne 1 ]; then
            OBJECT_COUNT=$($JQ -r ".total" $OBJECT_NAME.json)
            if [ $OBJECT_COUNT -ne 0 ]; then
              echo 'The requested object is used in policy package '"$PACKAGE"
              break
            fi
          fi
        done
      fi
    done
  fi
done

echo 'Done!'

You should provide an object name as a parameter for this script.

This is only an example, and you may change it as you need.

Hope this helps.

Robert.

[Harald_Hansen]

Thank you. I have not been able to test it before now, it worked as a charm. 

[Harald_Hansen]

Btw, you need to check JQ for $? -ne 0 in all uses, for instance in line 61. The API doesn't handle duplicate names very gracefully.

I was also stumped that the filter option returns group memberships on global rules. These rules are not always relevant, I'm going to check if rule-number contains a dot "." and one needs to expand the limit to more than 1.

[Andreas_Lorenze]

Hi Robert,

after migrating to R80 for production we realized the Cross Domain search is missing. To be honest this is mandantory and logic you need it for multiple domains. And we need all options. You need to check occurence of Global objects and only local occurence. Use in rule in object etc. So primarily we were using 2/3 of the options cross domain search offering.
We pay a lot of money for this product and I am not willing to waste my time playing a script kiddy, just because someone at Checkpoint has missed to migrate this feature (can't believe it anyway).
So PLEASE make this re-appearing SHORTLY.
Thx in advance

[Amiad_Stern]

Hi Andreas, Robert just suggested a replacement as was asked by Harald. I will make sure our PJM will get your feedback.

Regards,

Amiad.

[JoseOriundo]

Hi Robert,

We are new to R80.20 and need some help with your script.

Is there a way to search for IP addresses rather than object names?

Also, your script searches for the object name in all CMAs, but in addition to that it also searches you for each policy and this could take some time if you are looking for multiple objects.

I would like to upgrade our script and use an updated version designed for R80 versions.

Attached is the simple script we are currently using for the time being.

Also, do you see a problem with us using this simple script in R80.20?

Jose Oriundo

[Sven_Glock]

Hi Robert,

thanks for your work!

I am missing the cross domain search, too.

So I was very happy to find Haralds initiative here.

The first test of your script led to some errors:

Searching in domain xyz... Searching in package A... parse error: Invalid literal at line 120, column 10  ...
Searching in package D... parse error: Unmatched '}' at line 123, column 3

I am not a developer. Is this something that needs to be improved in the code or is there a problem with my database?

My first search was about an global object.

This causes the following result:

{
"code" : "generic_err_object_field_not_unique",
"message" : "Requested object name [Network] is not unique."
}

Are global objects not supported?

Thanks in advance.

Sven

[Robert_Decker]

Hi Sven,

The code above is only an example.

The code does not search for global objects. To fix this, please add the following after line 17 - 

DOMAINS_NAMES+=('Global')‍

This will also search in Global domain's database.

The parsing errors are due to the data in the responses from your database. For some reason, JQ tool fails to parse the data. The data is saved inside temporary files as noted at the top of the script (I suspect in package.json file). You may look at this data and verify the data inconsistency at the specified lines.

The "Requested object name [Network] is not unique" error is probably due to the fact that I use access-layer names instead of names+UIDs. If you have a global policy assigned in any of the domains, there may be two layers with a name 'Network'. To make this work, the code needs to save layer UID along with the layer name. A fix for this requires a bit more coding. I'll try to find some time and fix it later on.

Robert.

[Sven_Glock]

...looking forward to see your next post!

[Sven_Glock]

I strongly recommend to use session ID to log in so that you can logout at the end of your job.

Otherwise there will stay plenty of dead sessions in your management!

#!/bin/sh

JQ=${CPDIR}/jq/jq

sid=$(mgmt_cli login -r true --format json | jq -r '.sid')

<here will be your code>


# Logout CMA
mgmt_cli logout --session-id "$sid"
‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

[Robert_Decker]

yes, you are correct!

robert.

[Olavi_Lentso]

Still, does the cross cma search return in R80.XX?

The API is nice to have, but it is way too slow compared to R77.30 mdscmd runcrossdomainquery.