Create a Post
cancel
Showing results for 
Search instead for 
Did you mean: 
the_rock
Authority
Authority

Export of rules with zero hits in dashboard

Jump to solution

 Hi everyone,

 

I saw some posts about this before, but there was never a confirmation if this was ever available. I am trying to export rules in excel format for a customer that requested list of rules with zero hits, but does not seem its possible. I exported all the rules and can filter for example for any disabled rules, but I dont see column anywhere in excel file for hits, though hits column is enabled in dashboard.

 

Any idea if this is possible in R81 at all? It is cloud mgmt, but I dont think that makes any difference. 

Also, another thing I noticed, though this could be pure cosmetic is that all 100 some NAT rules show zero hits, which also makes no sense, since we know bunch of them are getting hit for sure.

 

Thanks in advance!

0 Kudos
2 Solutions

Accepted Solutions
Tomer_Noy
Employee
Employee

Following feedback from the field (including this post 😀), we're adding hitcount information to the csv export of the rulebase. This will of course be accessible without API scripting or expert privileges on the machine.

This enhancement will be included in the upcoming R81.10 and we are also looking into porting it back to earlier versions via JHF.

Here is a snippet of how it will look:

Tomer_Noy_0-1623105315090.png

 

View solution in original post

Tomer_Noy
Employee
Employee
22 Replies
Bob_Zimmerman
Advisor

The level of possibility depends on how much effort you're willing to expend. 😉

It's relatively easy to handle sectionless rules (that is, rules above any section header) and rules within sections. It's a lot harder to handle inline layers, as you have to run a separate API call to get their contents. Try this:

 

mgmt_cli -r true \
--format json \
show access-rulebase \
uid "<UUID>" \
show-hits true \
use-object-dictionary false \
| jq -c '.rulebase[]|if .rulebase then {section:.name,rule:.rulebase[]|{name:.name,hits:.hits.value}} else {name:.name,hits:.hits.value} end'

 

The 'if .rulebase then ... else ... end' structure gives you separate output for items which have a rulebase (read: rule sections) and objects which don't (sectionless rules). For one of the access layers on my development box, it returns this:

 

{"name":"Sectionless","hits":0}
{"name":"Bad browsing","hits":0}
{"section":"WebApp-1","rule":{"name":"Internet access in","hits":0}}
{"section":"WebApp-1","rule":{"name":"Web to App","hits":0}}
{"section":"WebApp-1","rule":{"name":"App to DB","hits":0}}
{"section":"WebApp-1","rule":{"name":"Admin access","hits":0}}
{"section":"Some Other Web App","rule":{"name":"Internet access in","hits":0}}
{"section":"Some Other Web App","rule":{"name":"Web to App","hits":0}}
{"section":"Some Other Web App","rule":{"name":"App to DB","hits":0}}
{"section":"Some Other Web App","rule":{"name":"Admin access","hits":0}}
{"section":"Access to Public Services","rule":{"name":null,"hits":0}}
{"section":"Access to Public Services","rule":{"name":null,"hits":0}}
{"section":"Access to Public Services","rule":{"name":null,"hits":0}}
{"section":"Access to Public Services","rule":{"name":null,"hits":0}}
{"section":"Cleanup","rule":{"name":"Cleanup rule","hits":0}}

 

You can then grep for "hits":0 and get the rules you're interested in. The jq filter should be relatively easy to expand to cover whatever fields you want.

the_rock
Authority
Authority

I hear ya, but I wanted to do this via dashboard, not cli, as its cloud server, so there is no ssh available to us, only from backend.

0 Kudos
Bob_Zimmerman
Advisor

You should still be able to make API calls. mgmt_cli even has a way to log in to a remote management server (though the '-r true' won't work; you'll need to provide credentials).

0 Kudos
PhoneBoy
Admin
Admin

This was in SmartDashboard back in the day but had not been added to R8x SmartConsole.
I don’t know what the precise plan is to add it back.
Current workaround is to use the API and there are several examples in the community on how to do this.

0 Kudos
the_rock
Authority
Authority

I tried running below, but no luck, command did work, but I cant see anything that shows hits values at all...

 

mgmt_cli show access-rulebase offset 0 limit 20 name "Network" details-level "standard" use-object-dictionary true show-hits true hits-settings.from-date "2021-04-30" hits-settings.to-date "2021-05-30"

0 Kudos
Bob_Zimmerman
Advisor

Without the 'hits-settings' part, I get this on R80.40 jumbo 114:

uid: "b406b732-2437-4848-9741-6eae1f5bf112"
name: "Network"
rulebase: 
- uid: "f8206d78-7c5f-4995-b519-865e2bf0730c"
  name: "Sectionless"
  type: "access-rule"
  domain: 
    uid: "41e821a0-3720-11e3-aa6e-0800200c9fde"
    name: "SMC User"
    domain-type: "domain"
  rule-number: 1
  track: 
    type: "29e53e3d-23bf-48fe-b6b1-d59bd88036f9"
    per-session: false
    per-connection: false
    accounting: false
    enable-firewall-session: false
    alert: "none"
  source: 
  - "97aeb369-9aea-11d5-bd16-0090272ccb30"
  source-negate: false
  destination: 
  - "97aeb369-9aea-11d5-bd16-0090272ccb30"
  destination-negate: false
  service: 
  - "97aeb405-9aea-11d5-bd16-0090272ccb30"
  service-negate: false
  vpn: 
  - "97aeb369-9aea-11d5-bd16-0090272ccb30"
  action: "6c488338-8eec-4103-ad21-cd461ac2c472"
  action-settings: 
    enable-identity-captive-portal: false
  content: 
  - "97aeb369-9aea-11d5-bd16-0090272ccb30"
  content-negate: false
  content-direction: "any"
  time: 
  - "97aeb369-9aea-11d5-bd16-0090272ccb30"
  hits: 
    percentage: "0%"
    level: "zero"
    value: 0
  custom-fields: 
    field-1: ""
    field-2: ""
    field-3: ""
  meta-info: 
...

Do you get the hits section if you leave the 'hits-settings' off?

0 Kudos
the_rock
Authority
Authority

Tried that, but no luck...TAC said they will run web visualization script on backend to see if they can parse it that way, but sounds like a lot of work though. I really wanted to give customer excel spreadsheet with zero hits rules via dashboard, but definitely does not appear it is possible.

0 Kudos
Hugo_vd_Kooij
Advisor

I expanded this to a sript that will get all policies and create a hitcounter CSV file.

 

#!/bin/bash

# Show Hitcounters for all policies

NOW=`/bin/date +%Y%m%d`
PACKAGES=`mgmt_cli -r true --port 443 show packages --format json | jq '.packages[] | .name' | sed 's/\"//g'`

for POLICY in $PACKAGES
do
        echo "Hitcounters for $POLICY"
        mgmt_cli -r true --port 443 show access-rulebase name "$POLICY Security" show-hits true --format json limit 50000 \
                | jq  '.rulebase[] | .rulebase[] | [."rule-number", .name, .hits.value]' --compact-output \
                | sed 's/\[//g'| \
                sed 's/\]//g' > HitCount-$POLICY-Security-$NOW.csv
done

The assumption is that you have "default" policy names with " Security" added to the name of the policy package.

0 Kudos
Bob_Zimmerman
Advisor

Have you tested this? Pretty sure limit only goes up to 500.

0 Kudos
Tomer_Noy
Employee
Employee

Following feedback from the field (including this post 😀), we're adding hitcount information to the csv export of the rulebase. This will of course be accessible without API scripting or expert privileges on the machine.

This enhancement will be included in the upcoming R81.10 and we are also looking into porting it back to earlier versions via JHF.

Here is a snippet of how it will look:

Tomer_Noy_0-1623105315090.png

 

View solution in original post

_Val_
Admin
Admin

Thanks @Tomer_Noy , we appreciate your prompt action here.

0 Kudos
Tomer_Noy
Employee
Employee

Glad to assist 😀

Credit for fast implementation goes to @Alon_Alapi and @Youssef_Obeidal.

the_rock
Authority
Authority

Good to hear! 

0 Kudos
Ganesan
Explorer

Any possibility to get in SmartDashboard the hitcount extracting via csv? Any other option without API ?

 

 

0 Kudos
PhoneBoy
Admin
Admin
0 Kudos
the_rock
Authority
Authority

Sorry to respond to this few months later, but just for my own reference, will this work say if management is R81.10 and gateways are, for example, on R80.xx versions, or everything needs to be on R81.10 code?

0 Kudos
Tomer_Noy
Employee
Employee

Yes, it will work when just the Management is R81.10.

The feature is implemented on the Management side, so gateway upgrades are not needed to get it. It relies on the same data that you see in the SmartConsole UI.

0 Kudos
Tomer_Noy
Employee
Employee

BTW, you can also get this new feature on R81 with the latest JHF on the Management server + latest SmartConsole build.

The team ported it back to that version as well.

Of course, upgrading to R81.10 is still a great option 😀

0 Kudos
the_rock
Authority
Authority

So I tried it on R81 mgmt managing R81 gateways (all jumbo 36) and latest console build 553 and when I export the rules, I see in the csv file there is hits column, but no numbers there at all, though in policy I see bunch of hits on every rule. Any idea?

0 Kudos
Tomer_Noy
Employee
Employee
the_rock
Authority
Authority

K, fair enough, le me try that and I will update the forum : )

0 Kudos
the_rock
Authority
Authority

I like that a lot Tomer. Tried it in 2 setups and when I export the rules, what I really love about it is that hits column gives EXACT number of hits. So say if dashboard shows 5k as number of hits, then csv file would show say 5785 number, which is great. One more question if you dont mind...cant recall now. Does number of hits get reset only if you do reboot of the firewalls or does it happen in any other instance?

 

Thanks again!

0 Kudos