HIT Counts R80.40

[Malik1]

Hi Experts, 

I have a client who wants to remove unused rules , one way of doing it is by identify rules with zero hits. Here I have a query.

1. lets say I have set the hit count for 3 months. that means SMS will store the hit count data for 3 months for each rule in the policy. If I had hits on a specific rule in Jan and for the next 3 months that rule was unused. does the hit count reset to zero? 

2. is there any other way to identify/ delete unused rule. ?

Regards,

SM

[Chris_Atkinson]

Log analysis is another way depending on your tooling.

Refer also:

https://sc1.checkpoint.com/documents/r80.40/webadminguides/en/cp_r80.40_securitymanagement_adminguid...

[G_W_Albrecht]

• Connect to command line on Security Gateway / each cluster member.

• Log in to Clish / Expert mode.

• Run the cpstat blades command.

[Duane_Toler]

I did this for a customer, and used the API command on the management server:

FROM_DATE=$(date -d "90 days ago" +"%Y-%m-%d")

TO_DATE=$(date +"%Y-%m-%d")

JSON_FILE="firewall_rules.last_90_days.json"

CSV_FILE="firewall_rules.last_90_days.csv"

export MGMT_CLI_FORMAT=json



TOTAL_RULES=$(mgmt_cli -r true show-access-rulebase name Network package Standard limit 1 details-level uid | jq '.total')

mgmt_cli -r true show-access-rulebase name Network package Standard show-hits true hits-settings.from-date ${FROM_DATE} hits-settings.to-date ${TO_DATE} use-object-dictionary true limit ${TOTAL_RULES}  > ${JSON_FILE}



echo '"Rule Number","Source","Destination","VPN","Service","Rule Action","Install On"' |tee ${CSV_FILE}



for rule in $(jq -r '.rulebase[] |

  select(.type=="access-rule"), select(.type=="access-section").rulebase[] |

  select(.enabled) |

  select(.hits.value == 0)."rule-number"|@text' ${JSON_FILE})

do



jq --arg rule $rule '( [ ."objects-dictionary"[] | { key:.uid, value:.name  } ] | from_entries ) as $objs |

  .rulebase[]| select(.type=="access-rule"),select(.type=="access-section").rulebase[] |

  select((."rule-number"|@text)==$rule) |

   [ ( [ ."rule-number"|@text ]|@csv ),

     ( [ $objs[."source"[]] ]|@csv),

     ( [$objs[."destination"[]] ]|@csv),

     ( [ $objs[."vpn"[]] ]|@csv ),

     ( [ $objs[."service"[]] ]|@csv ),

     ( [ $objs[."action"] ]|@csv ),

     ( [ $objs[."install-on"[] ] ]|@csv )

   ] |@csv

' < ${JSON_FILE}

done |\

sed -e 's@\\"\\"@@g' -e 's@\\"@"@g' -e 's/^""/"/g' -e 's/""$/"/g' |tee -a ${CSV_FILE}

echo

The CSV file has a list of the rules with zero hits.  Use however you wish.  Adjust numbers in the script however you want.

# cat firewall_rules.last_90_days.csv

"number","source","destination","vpn","service","action","install on"

"1","obj1,obj2","obj1,obj2","Any","Any","Accept","gateway"

"28","obj3","obj4","Any","http","Accept","gateway2"

[Malik1]

Hi Duane, 

im new to API , i did a copy/paste of the syntax shared on the smart console command cli  and every time it gets crashed.

Can I run this via expert mode. is there any syntax that I have to add.

Please guide.

[G_W_Albrecht]

This is a .sh script that should run in expert mode - you have to rename the policy package to your policy package name...

[Malik1]

Hello, 

Can u help me or guide how to run such scripts 

[PhoneBoy]

It is generally expected that, when using expert mode (including for running scripts), that the administrator has some basic Linux/Unix knowledge.

Versus copy/pasting a script, you might try a pre-built script that largely accomplishes the same thing: https://community.checkpoint.com/t5/API-CLI-Discussion/Disable-Delete-Rules-with-a-Zero-Hit-Count-MD...
After downloading the script to the management station, use chmod 755 to make the script executable (e.g. chmod 755 cleanup-zero-hits.sh).

[PhoneBoy]

Scripts like that won’t work in the SmartConsole CLI, it has to be done in expert mode.

[G_W_Albrecht]

Tried cpstat blades yet ?

[Malik1]

This gives the rules with top hits . what I need is the rules with zero hits.

[G_W_Albrecht]

Find some possibilities to see it here: sk85780: How to use the 'connstat' utility

[Malik1]

unfortunately this is only for Windows OS,

[G_W_Albrecht]

So what ? You need a Win VM at least as all the Dashboard parts only fully work on Win. Concerning you post, you can use sk111162: How to reset the Hit Count to make sure no older data is counted.

Also see sk135212: How to export the rulebase Hit Count and sk79300: How to correlate a rule from SmartConsole to its corresponding entry in kernel table to see...

[Pavan9096]

In the below command, you are using the 'Network' name, does it change firewall to firewall , customer to customer?
Is it static or configurable?

TOTAL_RULES=$(mgmt_cli -r true show-access-rulebase name Network package Standard limit 1 details-level uid | jq '.total')

[Duane_Toler]

This is what your policy layer name is.  You can make it variable.  Likewise for the policy package name.  Your policy package may not be called Standard, so this can be variable, too.

POLICY_LAYER="Whatever_yours_is_called"

POLICY_PACKAGE="Not So Standard"

TOTAL_RULES=$(mgmt_cli -r true show-access-rulebase name "${POLICY_LAYER}" package "${POLICY_PACKAGE}" limit 1 details-level uid | jq '.total')

In the mgmt_cli command, be sure it's double-quoted in case your policy layer has spaces in its name.

With the variable policy layer name, you can also use that to search the Application layer, if your app control rules are in a separate policy.  Likewise, if you have any nested policy layers, just change the name of that variable to get only the rules of that nested policy.

The main security policy is now "just another layer", albeit the top-level ordered layer.

Hope that helps!

[Pavan9096]

@Duane_Toler 
what command/API would list  POLICY_LAYER and POLICY_PACKAGE

[Pavan9096]

I'm writing a script that should run on any of the firewalls irrespective of package/policy names, 
I want to collect the hit count(last/first hit) of all security-rule from all checkpoint firewalls in my network.
so I would run the first API from which I will get the policy-layer and package name and will pass it to the below command :

mgmt_cli -r true show access-rulebase name {POLICY_LAYER} details-level {POLICY_PACKAGE} use-object-dictionary false show-hits true --format json > rulebase_output.json

[Duane_Toler]

One option is to set those variables on your own manually.  Keep in mind a few things:

1) You can't pull hit counts for a given rule on a given gateway; the hit counts are aggregated from all gateways on which the rule is installed.

2) You don't pull rules from a given gateway; they're stored on the management server and applied to the gateway(s) based on the policy targets and the Install On column.

3) You can't get the first-hit date of a rule nor the last-hit date; these are aggregates from a window of time you specify in the show-access-rulebase command.

4) These API commands are ran against the management server (or management domain for MDS), not a gateway

To get all policy packages on the management server, you use the "show-packages" API command.

To pull all policy layers, you use the "show-access-layers" API command.

Use the show-access-rulebase API to pull the entire rulebase of the policy layer and with an optional package argument.

If you want to craft your own sequences, you should review the management API command reference for your management server's version (and Jumbo HFA level!): https://sc1.checkpoint.com/documents/latest/APIs/#introduction~v1.9.1%20

You can do all of this at the command line in Bash (and I've done it), but you'll need to have plenty of shell scripting skills to do so.  Depending on the size of your top level policy layer and sub-layers, pulling all the rules and hit counts may be an intense operation and should be done wisely.  If you have 5,000 rules, you may not want to pull all 5,000 rules in one operation (and I think the API server may not allow that much in one task).  You can page through the rules in chunks, but you will have to write your own page-management loop to account for the various page chunks.  Again, tread cautiously, wisely, and judiciously depending on your configuration.

Very lightly tested, and not at all verified, but this might get you started:

# Set an array for the packages
declare -a policy_packages

# List of policy packages (hopefully no spaces in your package names)
policy_packages=$(mgmt_cli -r true show-packages |jq -r '.packages[].name')

# Loop through the policy packages
for package in ${policy_packages[@]}; do
  echo "Policy package: ${package}"
  # Get list of policy layers, replace spaces in policy layer names with 0xFF so Bash can tolerate
  for layer in $(mgmt_cli -r true show-access-layers|jq -r '."access-layers"[].name'|sed -e 's/ /\xff/g'); do
    # Replace 0xFF with spaces again
    policy_layer=$(echo $layer|sed -e 's/\xff/ /g')
    echo "  Policy layer: $policy_layer"

    # Fetch rulebase of policy layer in the package and get total rule count
    mgmt_cli -r true show-access-rulebase name "${policy_layer}" package ${package} limit 1 |jq -r '.total'
        # From here you can page through the rules in chunks to get the hit-counts in the layer
    done
done

[Pavan9096]

So, just to clarify, I need to run this APIs on the management server, not the gateway. If I happen to run it on the gateway, would it return the last hit results for only the local gateway or all gateways? Or will it not return anything or error out?

"mgmt_cli -r true show packages details-level full -f json" --- >> i will run this and will collect the Layer name and package name and will it in following API.
"mgmt_cli -r true show-access-rulebase name "{POLICY_LAYER}" package "{POLICY_PACKAGE}" show-hits true"

[the_rock]

Thats correct @Pavan9096 , that command you run on management server. By the way, below is what you can run on the gateway itself.

Andy

https://sc1.checkpoint.com/documents/latest/GaiaAPIs/#introduction~v1.8%20

[PhoneBoy]

The only thing the Gaia API exposes policy-wise is the Dynamic Layer features in R82.
Hit counts are a function of the Access Policy installed, and depending on the policy package, may include hits from multiple gateways.
The Management APIs are where Hit Count information is exposed.

[Duane_Toler]

You can run this on the management server if you're running it local with "-r true".  You can run it from a remote host which is an authorized Trusted Client, including a host where you have SmartConsole installed, but you will have to use a login session instead. 

The first command will show you the policy packages, not the policy access layers.  You need to use the "show-access-layers" API to get those. You will need to either parse the JSON output to get the fields you want, or read the output manually (unless you already  know your policy layers, then you can write those commands directly).

If you intend to paste those command lines directly, be sure you are referencing the variables correctly as Bash variables (with the $ notation).  They won't work as you have them written here; you are missing the $ notation.  As PhoneBoy mentioned earlier, you will need to have your own basic scripting knowledge and skill in order to craft your own scripts, which is outside the scope of this forum and thread.