topic Re: Export of rules with zero hits in dashboard in Firewall and Security Management

Export of rules with zero hits in dashboard

the_rock — Thu, 27 May 2021 12:47:03 GMT

Hi everyone,

I saw some posts about this before, but there was never a confirmation if this was ever available. I am trying to export rules in excel format for a customer that requested list of rules with zero hits, but does not seem its possible. I exported all the rules and can filter for example for any disabled rules, but I dont see column anywhere in excel file for hits, though hits column is enabled in dashboard.

Any idea if this is possible in R81 at all? It is cloud mgmt, but I dont think that makes any difference.

Also, another thing I noticed, though this could be pure cosmetic is that all 100 some NAT rules show zero hits, which also makes no sense, since we know bunch of them are getting hit for sure.

Thanks in advance!

Re: Export of rules with zero hits in dashboard

Bob_Zimmerman — Thu, 27 May 2021 14:45:52 GMT

The level of possibility depends on how much effort you're willing to expend. 😉

It's relatively easy to handle sectionless rules (that is, rules above any section header) and rules within sections. It's a lot harder to handle inline layers, as you have to run a separate API call to get their contents. Try this:

mgmt_cli -r true \ --format json \ show access-rulebase \ uid "<UUID>" \ show-hits true \ use-object-dictionary false \ | jq -c '.rulebase[]|if .rulebase then {section:.name,rule:.rulebase[]|{name:.name,hits:.hits.value}} else {name:.name,hits:.hits.value} end'

The 'if .rulebase then ... else ... end' structure gives you separate output for items which have a rulebase (read: rule sections) and objects which don't (sectionless rules). For one of the access layers on my development box, it returns this:

{"name":"Sectionless","hits":0} {"name":"Bad browsing","hits":0} {"section":"WebApp-1","rule":{"name":"Internet access in","hits":0}} {"section":"WebApp-1","rule":{"name":"Web to App","hits":0}} {"section":"WebApp-1","rule":{"name":"App to DB","hits":0}} {"section":"WebApp-1","rule":{"name":"Admin access","hits":0}} {"section":"Some Other Web App","rule":{"name":"Internet access in","hits":0}} {"section":"Some Other Web App","rule":{"name":"Web to App","hits":0}} {"section":"Some Other Web App","rule":{"name":"App to DB","hits":0}} {"section":"Some Other Web App","rule":{"name":"Admin access","hits":0}} {"section":"Access to Public Services","rule":{"name":null,"hits":0}} {"section":"Access to Public Services","rule":{"name":null,"hits":0}} {"section":"Access to Public Services","rule":{"name":null,"hits":0}} {"section":"Access to Public Services","rule":{"name":null,"hits":0}} {"section":"Cleanup","rule":{"name":"Cleanup rule","hits":0}}

You can then grep for "hits":0 and get the rules you're interested in. The jq filter should be relatively easy to expand to cover whatever fields you want.

Re: Export of rules with zero hits in dashboard

the_rock — Thu, 27 May 2021 16:18:10 GMT

I hear ya, but I wanted to do this via dashboard, not cli, as its cloud server, so there is no ssh available to us, only from backend.

Re: Export of rules with zero hits in dashboard

Bob_Zimmerman — Thu, 27 May 2021 19:42:46 GMT

You should still be able to make API calls. mgmt_cli even has a way to log in to a remote management server (though the '-r true' won't work; you'll need to provide credentials).

Re: Export of rules with zero hits in dashboard

PhoneBoy — Fri, 28 May 2021 07:22:11 GMT

This was in SmartDashboard back in the day but had not been added to R8x SmartConsole.
I don’t know what the precise plan is to add it back.
Current workaround is to use the API and there are several examples in the community on how to do this.

Re: Export of rules with zero hits in dashboard

the_rock — Mon, 31 May 2021 19:18:03 GMT

I tried running below, but no luck, command did work, but I cant see anything that shows hits values at all...

mgmt_cli show access-rulebase offset 0 limit 20 name "Network" details-level "standard" use-object-dictionary true show-hits true hits-settings.from-date "2021-04-30" hits-settings.to-date "2021-05-30"

Re: Export of rules with zero hits in dashboard

the_rock — Mon, 31 May 2021 19:20:44 GMT

Tried that, but no luck...TAC said they will run web visualization script on backend to see if they can parse it that way, but sounds like a lot of work though. I really wanted to give customer excel spreadsheet with zero hits rules via dashboard, but definitely does not appear it is possible.

Re: Export of rules with zero hits in dashboard

Bob_Zimmerman — Mon, 31 May 2021 21:25:28 GMT

Without the 'hits-settings' part, I get this on R80.40 jumbo 114:

uid: "b406b732-2437-4848-9741-6eae1f5bf112" name: "Network" rulebase: - uid: "f8206d78-7c5f-4995-b519-865e2bf0730c" name: "Sectionless" type: "access-rule" domain: uid: "41e821a0-3720-11e3-aa6e-0800200c9fde" name: "SMC User" domain-type: "domain" rule-number: 1 track: type: "29e53e3d-23bf-48fe-b6b1-d59bd88036f9" per-session: false per-connection: false accounting: false enable-firewall-session: false alert: "none" source: - "97aeb369-9aea-11d5-bd16-0090272ccb30" source-negate: false destination: - "97aeb369-9aea-11d5-bd16-0090272ccb30" destination-negate: false service: - "97aeb405-9aea-11d5-bd16-0090272ccb30" service-negate: false vpn: - "97aeb369-9aea-11d5-bd16-0090272ccb30" action: "6c488338-8eec-4103-ad21-cd461ac2c472" action-settings: enable-identity-captive-portal: false content: - "97aeb369-9aea-11d5-bd16-0090272ccb30" content-negate: false content-direction: "any" time: - "97aeb369-9aea-11d5-bd16-0090272ccb30" hits: percentage: "0%" level: "zero" value: 0 custom-fields: field-1: "" field-2: "" field-3: "" meta-info: ...

Do you get the hits section if you leave the 'hits-settings' off?

Re: Export of rules with zero hits in dashboard

Tomer_Noy — Mon, 07 Jun 2021 22:36:48 GMT

Following feedback from the field (including this post 😀), we're adding hitcount information to the csv export of the rulebase. This will of course be accessible without API scripting or expert privileges on the machine.

This enhancement will be included in the upcoming R81.10 and we are also looking into porting it back to earlier versions via JHF.

Here is a snippet of how it will look:

Re: Export of rules with zero hits in dashboard

_Val_ — Tue, 08 Jun 2021 06:30:28 GMT

Thanks @Tomer_Noy , we appreciate your prompt action here.

Re: Export of rules with zero hits in dashboard

Tomer_Noy — Tue, 08 Jun 2021 07:47:48 GMT

Glad to assist 😀

Credit for fast implementation goes to @Alon_Alapi and @Youssef_Obeidal.

Re: Export of rules with zero hits in dashboard

the_rock — Tue, 08 Jun 2021 13:10:26 GMT

Good to hear!

Re: Export of rules with zero hits in dashboard

Ganesan — Fri, 18 Jun 2021 03:28:46 GMT

Any possibility to get in SmartDashboard the hitcount extracting via csv? Any other option without API ?

Re: Export of rules with zero hits in dashboard

PhoneBoy — Fri, 18 Jun 2021 04:02:42 GMT

Answered in an earlier post in this thread: https://community.checkpoint.com/t5/Management/Export-of-rules-with-zero-hits-in-dashboard/m-p/120552/highlight/true#M26931

Re: Export of rules with zero hits in dashboard

Hugo_vd_Kooij — Wed, 08 Sep 2021 14:41:40 GMT

I expanded this to a sript that will get all policies and create a hitcounter CSV file.

#!/bin/bash

# Show Hitcounters for all policies

NOW=`/bin/date +%Y%m%d`
PACKAGES=`mgmt_cli -r true --port 443 show packages --format json | jq '.packages[] | .name' | sed 's/\"//g'`

for POLICY in $PACKAGES
do
        echo "Hitcounters for $POLICY"
        mgmt_cli -r true --port 443 show access-rulebase name "$POLICY Security" show-hits true --format json limit 50000 \
                | jq  '.rulebase[] | .rulebase[] | [."rule-number", .name, .hits.value]' --compact-output \
                | sed 's/\[//g'| \
                sed 's/\]//g' > HitCount-$POLICY-Security-$NOW.csv
done

The assumption is that you have "default" policy names with " Security" added to the name of the policy package.

Re: Export of rules with zero hits in dashboard

Bob_Zimmerman — Wed, 08 Sep 2021 15:30:21 GMT

Have you tested this? Pretty sure limit only goes up to 500.

Re: Export of rules with zero hits in dashboard

the_rock — Thu, 09 Sep 2021 00:33:30 GMT

Sorry to respond to this few months later, but just for my own reference, will this work say if management is R81.10 and gateways are, for example, on R80.xx versions, or everything needs to be on R81.10 code?

Re: Export of rules with zero hits in dashboard

Tomer_Noy — Thu, 09 Sep 2021 05:47:05 GMT

Yes, it will work when just the Management is R81.10.

The feature is implemented on the Management side, so gateway upgrades are not needed to get it. It relies on the same data that you see in the SmartConsole UI.

Re: Export of rules with zero hits in dashboard

Tomer_Noy — Thu, 09 Sep 2021 06:25:34 GMT

BTW, you can also get this new feature on R81 with the latest JHF on the Management server + latest SmartConsole build.

The team ported it back to that version as well.

Of course, upgrading to R81.10 is still a great option 😀

Re: Export of rules with zero hits in dashboard

the_rock — Thu, 09 Sep 2021 11:20:37 GMT

So I tried it on R81 mgmt managing R81 gateways (all jumbo 36) and latest console build 553 and when I export the rules, I see in the csv file there is hits column, but no numbers there at all, though in policy I see bunch of hits on every rule. Any idea?