Firewall - Automating Firewall Rule Cleanup Based on Usage

[HeikoAnkenbrand]

The following one-liner identifies all rules with a hit count of 0 entries. Depending on your environment, you may need to adjust the policy name to ensure the command runs against the correct configuration.

You can store all rules with a hit count of 0 in a file (e.g. delete_rules.txt). This file can then be processed by a script to automatically remove the listed rules.

mgmt_cli -r true show access-rulebase offset 0 limit 20 name "Network" details-level "standard" show-hits "true" use-object-dictionary true  --format json | jq '.rulebase[]| select(.hits.value==0) | {number:.["rule-number"], name:.name, uid:.uid}' > delete_rules.txt

You can use the file delete_rules.txt, which contains the list of rules with a hit count of 0, as input for an automated script. The script should read each line of the file (each line representing a rule), and then construct the appropriate delete command for that rule.

For security reasons,
I’m not sharing the one-liner for direct deletion, as it could easily cause serious damage to the policy.

This program writes the delete commands securely to an echo output; you can then remove the “echo” command in the one-liner and all rules will be deleted on the SMS 😉

jq -r '.uid' delete_rules.txt | while read uid; do
    echo mgmt_cli -r true delete access-rule uid "$uid" -s id.txt
done​

The id.txt file contains the status of the deletion action.

[Bob_Zimmerman]

Important notes: the one-liner above to find rules with no hits does not consider rules inside a section. It also does not consider the age of the rule. A brand new rule (or newly updated rule) which hasn't yet been tested would show up in the list of rules to be removed.

Here is a script which will get all of the rules including ones in sections:

[Expert@DallasSC]# mgmt_cli -f json -r true show access-rulebase name "Network" details-level full show-hits true use-object-dictionary true limit 500 offset 0 | jq -c '.rulebase[]|if has("rulebase") then .rulebase[] else . end|{uuid:.uid,hits:.hits.value,ruleNumber:."rule-number",ruleName:.name,lastModified:."meta-info"."last-modify-time".posix}'
{"uuid":"7f2e8b2e-a444-4e78-ae30-2c353fcb1811","hits":5601804,"ruleNumber":1,"ruleName":"Sectionless","lastModified":1749511597766}
{"uuid":"cfaeefa5-a96a-4ed8-8306-72e72d79f23d","hits":35332,"ruleNumber":2,"ruleName":"Management access","lastModified":1749511600777}
{"uuid":"5dae513d-934d-4a47-b8af-1b5d6b201eb4","hits":0,"ruleNumber":3,"ruleName":"Bad browsing","lastModified":1749511601446}
{"uuid":"c5b09864-5c5c-4775-b781-2287c56496b4","hits":0,"ruleNumber":4,"ruleName":null,"lastModified":1749511602850}
{"uuid":"7da03051-74b7-4d2f-92ff-7af45d8a625b","hits":0,"ruleNumber":5,"ruleName":null,"lastModified":1749511603707}
{"uuid":"21c0044c-f1a2-4fa7-bfb8-b19c0ca1dab1","hits":0,"ruleNumber":6,"ruleName":null,"lastModified":1749511604665}
{"uuid":"7f54a07b-ec25-4ce3-a427-73dfa6affc54","hits":0,"ruleNumber":7,"ruleName":null,"lastModified":1749511605528}
{"uuid":"26e693d5-360e-4ff6-a8f1-b3a33fd96a0f","hits":0,"ruleNumber":8,"ruleName":null,"lastModified":1749511606414}
{"uuid":"5d7e798b-3df8-4c5b-bc49-dc8c0cde40aa","hits":0,"ruleNumber":9,"ruleName":null,"lastModified":1749511607238}
{"uuid":"f873fabe-9d6d-41a5-8d63-0bb445ef141f","hits":0,"ruleNumber":10,"ruleName":null,"lastModified":1749511608131}
{"uuid":"81f1116a-b8b2-43cc-b0ad-802b584bdca1","hits":0,"ruleNumber":11,"ruleName":null,"lastModified":1749511609027}
{"uuid":"b7d2f494-631a-4a77-8480-d70297346aa5","hits":0,"ruleNumber":12,"ruleName":null,"lastModified":1749511609889}
{"uuid":"e80c2af2-30d7-4077-a332-274218019762","hits":0,"ruleNumber":13,"ruleName":null,"lastModified":1749511610664}
{"uuid":"267b45b0-efe5-47fc-83ec-b6b70335191a","hits":0,"ruleNumber":14,"ruleName":null,"lastModified":1749511612564}
{"uuid":"4c7aa052-0077-4cdc-8444-71c886232ea5","hits":0,"ruleNumber":15,"ruleName":null,"lastModified":1749511613791}
{"uuid":"04c0c300-4a22-47a1-9738-6bac10cf73f1","hits":0,"ruleNumber":16,"ruleName":null,"lastModified":1749511615241}
{"uuid":"a2a5e904-b467-4eb0-82f1-862891c670d5","hits":0,"ruleNumber":17,"ruleName":null,"lastModified":1749511616156}
{"uuid":"8e6adc3c-d8ff-472d-8daa-ad7055319ed4","hits":0,"ruleNumber":18,"ruleName":null,"lastModified":1749511617077}
{"uuid":"95b4133d-bb45-4d2a-9778-ae824819f033","hits":0,"ruleNumber":19,"ruleName":null,"lastModified":1749511617998}
{"uuid":"817d4149-bbb7-4dc0-8ba6-c11b8b9aad02","hits":0,"ruleNumber":20,"ruleName":null,"lastModified":1749511618889}
{"uuid":"7fa553a0-f199-48b8-83eb-b62799e5193d","hits":0,"ruleNumber":21,"ruleName":null,"lastModified":1749511619723}
{"uuid":"4b48e412-21e4-46a3-b8e4-9cda96664e56","hits":0,"ruleNumber":22,"ruleName":null,"lastModified":1749511620605}
{"uuid":"d4d3f951-09c1-42b2-a38c-d3390fde56ab","hits":0,"ruleNumber":23,"ruleName":null,"lastModified":1749511621484}
{"uuid":"35055d5e-86da-443c-8719-2f85a4271d43","hits":0,"ruleNumber":24,"ruleName":null,"lastModified":1749511622364}
{"uuid":"59571bb0-b18f-4869-b19f-ebb3c8f51721","hits":0,"ruleNumber":25,"ruleName":null,"lastModified":1749511623204}
{"uuid":"5a3d81d2-85bf-457d-a546-65c0b97e8602","hits":0,"ruleNumber":26,"ruleName":null,"lastModified":1749511624010}
{"uuid":"c3ea60ca-99e1-4d46-a48f-2f664c8eda68","hits":0,"ruleNumber":27,"ruleName":null,"lastModified":1749511625185}
{"uuid":"d6ce82b3-9c3b-4819-98b8-69146ee21f8f","hits":0,"ruleNumber":28,"ruleName":null,"lastModified":1749511626879}
{"uuid":"91bec9e7-6df4-4d0f-8563-eb5f600c14fb","hits":9393231,"ruleNumber":29,"ruleName":null,"lastModified":1749511628953}

Here's a version to get rules older than 370 days and with 0 hits:

mgmt_cli -f json -r true show access-rulebase name "Network" details-level full show-hits true use-object-dictionary true limit 500 offset 0 | jq -c '.rulebase[]|if has("rulebase") then .rulebase[] else . end|{uuid:.uid,hits:.hits.value,ruleNumber:."rule-number",ruleName:.name,lastModified:."meta-info"."last-modify-time".posix}' | jq -c "select(.lastModified < $(date --date='-370 days' +'%s000'))|select(.hits == 0)"

In both cases, if you have more than 500 rules, you'll have to run it multiple times adjusting the offset value to get all of the rules. You will probably also need to adjust the access layer name ("Network" above).

[HeikoAnkenbrand]

@Bob_Zimmerman 
Thanks for the good tip with the date of the old rules!

Technically, your statement is completely correct.

I had to smile a bit at the suggestion to run it multiple times if there are more than 500 entries.  If someone really needs to delete more than 500 unused firewall rules, they should probably reconsider their security policy in the first place 😉

[Bob_Zimmerman]

Most of my policies are 3000+ rules with ~30% unused rules scattered throughout (stuff which got decommissioned without telling the firewall team, unnecessarily bidirectional rules, etc.). 😉 Cleanup is a headache.

[the_rock]

3000+ rules? Why so many? : - )

Andy

[Bob_Zimmerman]

1. Interior firewalls (tons of application-to-application rules)
2. Which are a decade or more old (lots of time to build up old cruft)
3. With a change control process which makes adding easy and removing difficult

My biggest access layer is a bit over 13,000 rules.

It's a long story, but now that we have dealt with other, higher-priority issues (like getting rid of R67, upgrading all the firewalls to R81.20 on a reasonably current jumbo, and so on), we can spend some time cleaning stuff up. I've deleted over 60,000 unused objects so far this year without even touching duplicate objects (one CMA has NINE different objects for 10.0.0.0/8, all used in different places after automated migrations from other vendors).

This is why I'm so pleased with CDT. It has made upgrades and updates simple and repeatable, freeing time to spend on cleanup.

And why I like Where Used > Replace so much. Now, when things are decommissioned, we replace them with None. Rules trend towards an empty source or destination, then they get disabled, and we have a process which cleans up rules which have been disabled for a while.

It's also why I put so much effort into my onEachFirewall script and my cluster member config diff script. The cluster members had so many little (and not-so-little) differences, like a route existing on one member but not another. The diff script warns us about problems like that before a failover leads to a partial outage. The onEachFirewall script lets me quickly do things like check whether all of the firewalls can actually reach their NTP servers, or dump all of the interfaces across the environment to see if any are accidentally at 100M.

[the_rock]

13000 rules?! My only comment = no comment :=)

[Daniel_Kuhl1]

SmartOptimize offered by Check Point PS can help with rulebase cleanups. But I guess you guys are aware of it. 😊

[the_rock]

Its very good tool!

[Bob_Zimmerman]

I'm aware of SmartOptimize, and I regret every time we have paid for PS to run it for us. Every single time, it has reduced the rule count, but absolutely wrecked the maintainability of the policy. As an example, one started with 40 rules each allowing clients to one or two web services with a ticket covering that access. SmartOptimize turned those into one rule allowing clients to 40+ web services with a name which no longer makes sense, and an incomprehensible jumble of tickets in the comments. The problem was only noticed months later when auditors selected a rule which had been modified like that and asked us to prove everything in it had gone through the proper change control process.

[genisis__]

We looked at this, however from what I could see majority of what it does could be scripted.  A major thing that we where hoping that it did (and it does not), is to interrogate active rules to determine what is actually in use.
From what we can see, AlgoSec or Tuffin are going to be a better option for the cost.

[Tomer_Noy]

Worth mentioning that looking at active (hit) rules to see what in the rule is actually being used (hitcount per object) is something that Policy Insights does very well. It's a new Management blade that is built directly into the Management and SmartConsole.

It's definitely worth evaluating it if you are considering one of the partners just for the use-case of cleaning up or tightening your rulebase.

[the_rock]

That is SUPERB. Just ran it in my lab and it showed exact rules with 0 hits...amazing.

Andy

[Expert@CP-MANAGEMENT:0]# more delete_rules.txt
{
"number": 1,
"name": "ica mgmt tool rule",
"uid": "be01a1e1-e4ff-4463-84f3-1c7865978735"
}
{
"number": 2,
"name": "Network feed block",
"uid": "e5a96a3d-8296-4eb5-87f6-6813026ee843"
}
{
"number": 12,
"name": "stealth rule",
"uid": "1061f0d8-516f-4af3-b388-74ba735f5e44"
}
[Expert@CP-MANAGEMENT:0]#

[genisis__]

One thing I've mentioned in Checkmates (I think), is the ability to not only identify the zero hit rules but to also export these (together with the UID) into html format that could be presented to client.  
You know this anyway as I've provided you my feeble attempt at this 🙂

[JozkoMrkvicka]

Back in good old times (before R8x), there was SmartDashboard feature build in by default where you were able to filter all 0 hits or disabled rules directly in rulebase 🙂 Someone just decided that such a great feature is not going to be integrated into SmartConsole, even in latest R82 version.

There is also Show Package Tool which can export rulebase with hit count data once -c parameter is set.

Or just use Export option directly in SmartConsole which exports hit count by default:

[the_rock]

It still bugs me why that feature was removed...it was so useful.

[Tomer_Noy]

The feature is coming back 😁

It's in the latest R82 JHF (not yet recommended take).

@Tal_Ben_Bassat wrote a great highlight post recently that included this:
https://community.checkpoint.com/t5/Management/This-Month-s-Spotlight-4-Features-You-Should-Start-Us... 

[the_rock]

Wait a second...what caught my eye in your screenshot is why layer shows automatic...isnt this ONLY available if you enable playblocks? Because if I try this in my R82 labs with latest jumbo, it never comes up 🙂

Andy

[Tomer_Noy]

It's not dependent on Playblocks. It's just an example in the screenshot.

You do need the latest JHF - R82 take 36 or above. Note that the recommended take is still 34, so should be recommended soon.

You might also need an updated SmartConsole.

[the_rock]

I have latest smart console and everything is on R82 jumbo 36 in the lab, but that option never comes up when I search for hits. Maybe something else has to be enabled?

Andy

[Youssef_Obeidal]

Hi,

Can you share the SmartConsole build?
And via SSH, share the output of: cpinfo -y all
You can contact me directly via email - youssefob@checkpoint.com

[the_rock]

smart console build, 1056, latest one

[Expert@CP-MANAGEMENT:0]# cpinfo -yall

This is Check Point CPinfo Build 914000250 for GAIA
[CPshared]
HOTFIX_R82_JUMBO_HF_MAIN Take: 36
[IDA]
No hotfixes..
[CPFC]
No hotfixes..
[MGMT]
HOTFIX_R82_JUMBO_HF_MAIN Take: 36
[FW1]
HOTFIX_WEBCONSOLE_AUTOUPDATE
HOTFIX_INEXT_NANO_EGG_AUTOUPDATE
HOTFIX_NGM_DOCTOR_AUTOUPDATE
HOTFIX_R82_JUMBO_HF_MAIN Take: 36
HOTFIX_VCE_R81_20_AUTOUPDATE
HOTFIX_PUBLIC_CLOUD_CA_BUNDLE_AUTOUPDATE
HOTFIX_GOT_MGMT_AUTOUPDATE
HOTFIX_GOT_TPCONF_MGMT_AUTOUPDATE

FW1 build number:
This is Check Point Security Management Server R82 - Build 006
This is Check Point's software version R82 - Build 006
[SecurePlatform]
HOTFIX_GAIA_API_AUTOUPDATE
HOTFIX_R82_JUMBO_HF_MAIN Take: 36
[CPinfo]
No hotfixes..
[AutoUpdater]
HOTFIX_INFRA_CONFIG_AUTOUPDATE
[DIAG]
No hotfixes..
[Reporting Module]
HOTFIX_LOGEXPORTER_AUTOUPDATE
HOTFIX_R82_JUMBO_HF_MAIN Take: 36
[CPuepm]
HOTFIX_R82_JUMBO_HF_MAIN Take: 36
[VSEC]
HOTFIX_CLOUDGUARD_CONTROLLER_R82_AUTOUPDATE
[CPDepCon]
HOTFIX_R82_JUMBO_HF_MAIN Take: 36
[CPRepMan]
No hotfixes..
[SmartLog]
HOTFIX_R82_JUMBO_HF_MAIN Take: 36
[R7540CMP]
No hotfixes..
[R76CMP]
No hotfixes..
[SFWR77CMP]
No hotfixes..
[SFWR80CMP]
No hotfixes..
[SFWR81CMP]
HOTFIX_R82_JHF_COMP Take: 36
[SFWR82CMP]
HOTFIX_R82_JHF_COMP Take: 36
[R77CMP]
No hotfixes..
[R8040CMP]
No hotfixes..
[R8120CMP]
HOTFIX_R82_JHF_COMP Take: 36
[core_uploader]
HOTFIX_CHARON_HF
[CPUpdates]
BUNDLE_LOGEXPORTER_AUTOUPDATE Take: 44
BUNDLE_GWS_ONBOARD_AUTOUPDATE Take: 50
BUNDLE_AFW_AUTOUPDATE Take: 74
BUNDLE_CME_AUTOUPDATE Take: 310
BUNDLE_INEXT_NANO_EGG_AUTOUPDATE Take: 23
BUNDLE_ESOD_CSHELL_AUTOUPDATE Take: 20
BUNDLE_R82_JUMBO_HF_MAIN_SC Take: 15
BUNDLE_GAIA_API_AUTOUPDATE Take: 9
BUNDLE_INFRA_AUTOUPDATE Take: 72
BUNDLE_DEP_INSTALLER_AUTOUPDATE Take: 31
BUNDLE_NGM_DOCTOR_AUTOUPDATE Take: 28
BUNDLE_CLOUDGUARD_CONTROLLER_R82_AUTOUPDATE Take: 21
BUNDLE_R82_JUMBO_HF_MAIN Take: 36
BUNDLE_DC_CONTENT_AUTOUPDATE Take: 20
BUNDLE_WEBCONSOLE_AUTOUPDATE Take: 149
BUNDLE_VCE_R81_20_AUTOUPDATE Take: 42
BUNDLE_TUNNEL_AUTOUPDATE Take: 127
BUNDLE_DANA_AUTOUPDATE Take: 212
BUNDLE_INFRA_CONFIG_AUTOUPDATE Take: 10
BUNDLE_INFINITY_CRA_AUTOUPDATE Take: 37
BUNDLE_PUBLIC_CLOUD_CA_BUNDLE_AUTOUPDATE Take: 21
BUNDLE_HCP_AUTOUPDATE Take: 85
BUNDLE_GOT_MGMT_AUTOUPDATE Take: 136
BUNDLE_DC_INFRA_AUTOUPDATE Take: 30
BUNDLE_GOT_TPCONF_MGMT_AUTOUPDATE Take: 39
BUNDLE_MINMUS_AUTOUPDATE Take: 23
BUNDLE_KERBIN_AUTOUPDATE Take: 47
BUNDLE_CPVIEWEXPORTER_AUTOUPDATE Take: 75
BUNDLE_QUID_AUTOUPDATE Take: 48
BUNDLE_CPSDC_AUTOUPDATE Take: 34
BUNDLE_CPOTLPAGENT_AUTOUPDATE Take: 115
BUNDLE_CPOTELCOL_AUTOUPDATE Take: 192
BUNDLE_CORE_FILE_UPLOADER_AUTOUPDATE Take: 23
[CPotelcol]
HOTFIX_OTLP_GA
[CPotlpAgent]
HOTFIX_OTLP_GA
[cpsdc_wrapper]
HOTFIX_CPSDC_AUTOUPDATE
[CPquid]
HOTFIX_QUID_AUTOUPDATE
[MGMTAPI]
No hotfixes..
[CPviewExporter]
HOTFIX_OTLP_GA
[diff_report_wrapper]
HOTFIX_MINMUS_AUTOUPDATE
HOTFIX_KERBIN_AUTOUPDATE
[itp_wrapper]
HOTFIX_GOT_MGMT_AUTOUPDATE
[hcp_wrapper]
HOTFIX_HCP_AUTOUPDATE
[icra_wrapper]
HOTFIX_INFINITY_CRA_AUTOUPDATE
[sho_wrapper]
HOTFIX_DANA_AUTOUPDATE
[infinity_onprem_wrapper]
HOTFIX_TUNNEL_AUTOUPDATE
[CME]
HOTFIX_CME_AUTOUPDATE
[CPDepInst]
No hotfixes..
[afw_wrapper]
HOTFIX_AFW_AUTOUPDATE
[gws_onboard_wrapper]
HOTFIX_GWS_ONBOARD_AUTOUPDATE

[Expert@CP-MANAGEMENT:0]#

[Youssef_Obeidal]

Hi,

Thank you for your feedback. We found out that the SmartConsole release didn't include the hitcount search support. and will be fixed in the upcoming days.

I will update once the new release is available.

Sorry for the inconvenience 

Meanwhile, you can still use the MGMT API new capability, for example:

mgmt_cli show access-rulebase offset 0 limit 20 name "Network" details-level "standard" use-object-dictionary true filter "hits:ZERO" hits-settings.from-date "2014-01-01" hits-settings.to-date "2014-12-31T23:59" hits-settings.target "corporate-gw"  --format json

[HeikoAnkenbrand · Accepted solution]

@Bob_Zimmerman, @the_rock, @PhoneBoy 

I quickly created the SmartConsole extension during the last two days of my vacation.
When I have a bit more time, I’ll build a nicer version and make it available in the tools section.

Toolbox article:
SmartConsole Extension - Easy deletion of unused rules

Since deleting unused rules always annoyed me, I created a SmarteConsole plugin.
It does the following:

• Shows all rules in SmartConsole that have not been used (hit counter = 0).

• Lets you select any of the unused rules you want to delete.

• Generates the CLI commands for the Management Server to delete those rules.

I deliberately did not automate the deletion directly in SmartConsole, to avoid accidental rule removal if the tool is misused.

Install Smart Console Extension:
https://www.checkpoint.tips/ex/delete.json

Overview of the selection window:

Output of the delete commands (you only need to copy and paste them into an SSH session on the SMS):

[the_rock]

Its BRILLIANT!

[Daniel_Kuhl1]

Awesome extension.

[Bob_Zimmerman]

Note the deletion commands need to be modified in many cases. The layer is currently hard-coded to "Network", and there's no domain for MDSs. Neither is a huge deal, just something to be aware of.

[HeikoAnkenbrand]

@Bob_Zimmerman, @the_rock, @_Val_ 

In the new version 1.2, I have added the layer, so it should now work with any layer.
Furthermore, the list of objects to be deleted is now automatically loaded when the page is opened.

[the_rock]

Great!

[the_rock]

I saw what Bob posted about layers...any way Heiko this can be referenced for ALL layers, including both inline and ordered ones?

Best,

Andy