topic Re: HIT Counts R80.40 in Firewall and Security Management

HIT Counts R80.40

Malik1 — Mon, 14 Nov 2022 14:56:12 GMT

Hi Experts,

I have a client who wants to remove unused rules , one way of doing it is by identify rules with zero hits. Here I have a query.

1. lets say I have set the hit count for 3 months. that means SMS will store the hit count data for 3 months for each rule in the policy. If I had hits on a specific rule in Jan and for the next 3 months that rule was unused. does the hit count reset to zero?

2. is there any other way to identify/ delete unused rule. ?

Regards,

Re: HIT Counts R80.40

Chris_Atkinson — Mon, 14 Nov 2022 15:21:02 GMT

Log analysis is another way depending on your tooling.

Refer also:

https://sc1.checkpoint.com/documents/r80.40/webadminguides/en/cp_r80.40_securitymanagement_adminguide/topics-secmg/analyzing-the-rule-base-hit-count.htm

Re: HIT Counts R80.40

G_W_Albrecht — Mon, 14 Nov 2022 15:47:16 GMT

Connect to command line on Security Gateway / each cluster member.
Log in to Clish / Expert mode.
Run the cpstat blades command.

Re: HIT Counts R80.40

Duane_Toler — Mon, 14 Nov 2022 18:37:52 GMT

I did this for a customer, and used the API command on the management server:

FROM_DATE=$(date -d "90 days ago" +"%Y-%m-%d") TO_DATE=$(date +"%Y-%m-%d") JSON_FILE="firewall_rules.last_90_days.json" CSV_FILE="firewall_rules.last_90_days.csv" export MGMT_CLI_FORMAT=json TOTAL_RULES=$(mgmt_cli -r true show-access-rulebase name Network package Standard limit 1 details-level uid | jq '.total') mgmt_cli -r true show-access-rulebase name Network package Standard show-hits true hits-settings.from-date ${FROM_DATE} hits-settings.to-date ${TO_DATE} use-object-dictionary true limit ${TOTAL_RULES} > ${JSON_FILE} echo '"Rule Number","Source","Destination","VPN","Service","Rule Action","Install On"' |tee ${CSV_FILE} for rule in $(jq -r '.rulebase[] | select(.type=="access-rule"), select(.type=="access-section").rulebase[] | select(.enabled) | select(.hits.value == 0)."rule-number"|@text' ${JSON_FILE}) do jq --arg rule $rule '( [ ."objects-dictionary"[] | { key:.uid, value:.name } ] | from_entries ) as $objs | .rulebase[]| select(.type=="access-rule"),select(.type=="access-section").rulebase[] | select((."rule-number"|@text)==$rule) | [ ( [ ."rule-number"|@text ]|@csv ), ( [ $objs[."source"[]] ]|@csv), ( [$objs[."destination"[]] ]|@csv), ( [ $objs[."vpn"[]] ]|@csv ), ( [ $objs[."service"[]] ]|@csv ), ( [ $objs[."action"] ]|@csv ), ( [ $objs[."install-on"[] ] ]|@csv ) ] |@csv ' < ${JSON_FILE} done |\ sed -e 's@\\"\\"@@g' -e 's@\\"@"@g' -e 's/^""/"/g' -e 's/""$/"/g' |tee -a ${CSV_FILE} echo

The CSV file has a list of the rules with zero hits. Use however you wish. Adjust numbers in the script however you want.

# cat firewall_rules.last_90_days.csv

"number","source","destination","vpn","service","action","install on"

"1","obj1,obj2","obj1,obj2","Any","Any","Accept","gateway"

"28","obj3","obj4","Any","http","Accept","gateway2"

Re: HIT Counts R80.40

Malik1 — Wed, 16 Nov 2022 11:59:37 GMT

Hi Duane,

im new to API , i did a copy/paste of the syntax shared on the smart console command cli and every time it gets crashed.

Can I run this via expert mode. is there any syntax that I have to add.

Please guide.

Re: HIT Counts R80.40

G_W_Albrecht — Wed, 16 Nov 2022 12:25:12 GMT

This is a .sh script that should run in expert mode - you have to rename the policy package to your policy package name...

Re: HIT Counts R80.40

PhoneBoy — Wed, 16 Nov 2022 12:26:09 GMT

Scripts like that won’t work in the SmartConsole CLI, it has to be done in expert mode.

Re: HIT Counts R80.40

G_W_Albrecht — Wed, 16 Nov 2022 12:29:41 GMT

Tried cpstat blades yet ?

Re: HIT Counts R80.40

Malik1 — Wed, 16 Nov 2022 14:21:34 GMT

This gives the rules with top hits . what I need is the rules with zero hits.

Re: HIT Counts R80.40

Malik1 — Wed, 16 Nov 2022 14:25:50 GMT

Hello,

Can u help me or guide how to run such scripts

Re: HIT Counts R80.40

G_W_Albrecht — Wed, 16 Nov 2022 14:38:46 GMT

Find some possibilities to see it here: sk85780: How to use the 'connstat' utility

Re: HIT Counts R80.40

PhoneBoy — Wed, 16 Nov 2022 15:08:36 GMT

It is generally expected that, when using expert mode (including for running scripts), that the administrator has some basic Linux/Unix knowledge.

Versus copy/pasting a script, you might try a pre-built script that largely accomplishes the same thing: https://community.checkpoint.com/t5/API-CLI-Discussion/Disable-Delete-Rules-with-a-Zero-Hit-Count-MDS-or-SMS/m-p/40005#M2562
After downloading the script to the management station, use chmod 755 to make the script executable (e.g. chmod 755 cleanup-zero-hits.sh).

Re: HIT Counts R80.40

Malik1 — Thu, 17 Nov 2022 07:54:29 GMT

unfortunately this is only for Windows OS,

Re: HIT Counts R80.40

Pavan9096 — Thu, 14 Nov 2024 11:54:32 GMT

In the below command, you are using the 'Network' name, does it change firewall to firewall , customer to customer?
Is it static or configurable?

TOTAL_RULES=$(mgmt_cli -r true show-access-rulebase name Network package Standard limit 1 details-level uid | jq '.total')

Re: HIT Counts R80.40

Duane_Toler — Thu, 14 Nov 2024 13:58:39 GMT

This is what your policy layer name is. You can make it variable. Likewise for the policy package name. Your policy package may not be called Standard, so this can be variable, too.

POLICY_LAYER="Whatever_yours_is_called"

POLICY_PACKAGE="Not So Standard"

TOTAL_RULES=$(mgmt_cli -r true show-access-rulebase name "${POLICY_LAYER}" package "${POLICY_PACKAGE}" limit 1 details-level uid | jq '.total')

In the mgmt_cli command, be sure it's double-quoted in case your policy layer has spaces in its name.

With the variable policy layer name, you can also use that to search the Application layer, if your app control rules are in a separate policy. Likewise, if you have any nested policy layers, just change the name of that variable to get only the rules of that nested policy.

The main security policy is now "just another layer", albeit the top-level ordered layer.

Hope that helps!

Re: HIT Counts R80.40

Pavan9096 — Mon, 18 Nov 2024 10:04:36 GMT

@Duane_Toler
what command/API would list POLICY_LAYER and POLICY_PACKAGE

Re: HIT Counts R80.40

Pavan9096 — Mon, 18 Nov 2024 10:09:23 GMT

I'm writing a script that should run on any of the firewalls irrespective of package/policy names,
I want to collect the hit count(last/first hit) of all security-rule from all checkpoint firewalls in my network.
so I would run the first API from which I will get the policy-layer and package name and will pass it to the below command :

mgmt_cli -r true show access-rulebase name {POLICY_LAYER} details-level {POLICY_PACKAGE} use-object-dictionary false show-hits true --format json > rulebase_output.json

Re: HIT Counts R80.40

Duane_Toler — Mon, 18 Nov 2024 16:09:28 GMT

One option is to set those variables on your own manually. Keep in mind a few things:

1) You can't pull hit counts for a given rule on a given gateway; the hit counts are aggregated from all gateways on which the rule is installed.

2) You don't pull rules from a given gateway; they're stored on the management server and applied to the gateway(s) based on the policy targets and the Install On column.

3) You can't get the first-hit date of a rule nor the last-hit date; these are aggregates from a window of time you specify in the show-access-rulebase command.

4) These API commands are ran against the management server (or management domain for MDS), not a gateway

To get all policy packages on the management server, you use the "show-packages" API command.

To pull all policy layers, you use the "show-access-layers" API command.

Use the show-access-rulebase API to pull the entire rulebase of the policy layer and with an optional package argument.

If you want to craft your own sequences, you should review the management API command reference for your management server's version (and Jumbo HFA level!): https://sc1.checkpoint.com/documents/latest/APIs/#introduction~v1.9.1%20

You can do all of this at the command line in Bash (and I've done it), but you'll need to have plenty of shell scripting skills to do so. Depending on the size of your top level policy layer and sub-layers, pulling all the rules and hit counts may be an intense operation and should be done wisely. If you have 5,000 rules, you may not want to pull all 5,000 rules in one operation (and I think the API server may not allow that much in one task). You can page through the rules in chunks, but you will have to write your own page-management loop to account for the various page chunks. Again, tread cautiously, wisely, and judiciously depending on your configuration.

Very lightly tested, and not at all verified, but this might get you started:

# Set an array for the packages declare -a policy_packages # List of policy packages (hopefully no spaces in your package names) policy_packages=$(mgmt_cli -r true show-packages |jq -r '.packages[].name') # Loop through the policy packages for package in ${policy_packages[@]}; do echo "Policy package: ${package}" # Get list of policy layers, replace spaces in policy layer names with 0xFF so Bash can tolerate for layer in $(mgmt_cli -r true show-access-layers|jq -r '."access-layers"[].name'|sed -e 's/ /\xff/g'); do # Replace 0xFF with spaces again policy_layer=$(echo $layer|sed -e 's/\xff/ /g') echo " Policy layer: $policy_layer" # Fetch rulebase of policy layer in the package and get total rule count mgmt_cli -r true show-access-rulebase name "${policy_layer}" package ${package} limit 1 |jq -r '.total' # From here you can page through the rules in chunks to get the hit-counts in the layer done done

Re: HIT Counts R80.40

Pavan9096 — Mon, 25 Nov 2024 10:26:35 GMT

So, just to clarify, I need to run this APIs on the management server, not the gateway. If I happen to run it on the gateway, would it return the last hit results for only the local gateway or all gateways? Or will it not return anything or error out?

"mgmt_cli -r true show packages details-level full -f json" --- >> i will run this and will collect the Layer name and package name and will it in following API.
"mgmt_cli -r true show-access-rulebase name "{POLICY_LAYER}" package "{POLICY_PACKAGE}" show-hits true"

Re: HIT Counts R80.40

G_W_Albrecht — Mon, 25 Nov 2024 11:09:58 GMT

So what ? You need a Win VM at least as all the Dashboard parts only fully work on Win. Concerning you post, you can use sk111162: How to reset the Hit Count to make sure no older data is counted.

Also see sk135212: How to export the rulebase Hit Count and sk79300: How to correlate a rule from SmartConsole to its corresponding entry in kernel table to see the rule's Hit Count