This website uses Cookies. Click Accept to agree to our website's cookie use as described in our Privacy Policy. Click Preferences to customize your cookie settings.
Accept
Reject
Preferences
ABOUT CHECKMATES CYBER SECURITY COMMUNITY & FAQ
Create a Post
Help
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

Are you a member of CheckMates?

×
Sign in with your Check Point UserCenter/PartnerMap account to access more great content and get a chance to win some Apple AirPods! If you don't have an account, create one now for free!
flachance
Advisor
‎2022-10-14 08:23 AM
Jump to solution

High CPU

Hi, we're experiencing some high cpu usage and to be honest I'm not sure what to make of the cpview results. What is PM tier 1? Is it related to IPS?

Cpu5.JPGCpu4.JPGCpu3.JPGCpu2.JPGCpu1.JPG

0 Kudos
1 Solution

Accepted Solutions
PhoneBoy
Admin
Admin
‎2022-10-14 11:08 AM
In response to flachance
Jump to solution

Yes, using fw ctl fast_accel.
Refer to: https://supportcenter.checkpoint.com/supportcenter/portal?eventSubmit_doGoviewsolutiondetails=&solut... 

View solution in original post

0 Kudos
9 Replies
flachance
Advisor
‎2022-10-14 08:58 AM
Jump to solution

I tried turning off ips but it didn't help

0 Kudos
PhoneBoy
Admin
Admin
‎2022-10-14 09:17 AM
Jump to solution

Pattern Matcher is used with any blade that is not Firewall and VPN.
That would include but isn't necessarily caused by IPS.
Please provide the output of the Super 7 Commands: https://community.checkpoint.com/t5/Scripts/S7PAC-Super-Seven-Performance-Assessment-Commands/m-p/40... 

0 Kudos
flachance
Advisor
‎2022-10-14 09:38 AM
In response to PhoneBoy
Jump to solution

+-----------------------------------------------------------------------------+
| Super Seven Performance Assessment Commands v0.5 (Thanks to Timothy Hall) |
+-----------------------------------------------------------------------------+
| Inspecting your environment: OK(B |
| This is a firewall....(continuing) |
| |
| Referred pagenumbers are to be found in the following book: |
| Max Power: Check Point Firewall Performance Optimization - Second Edition |
| |
| Available at http://www.maxpowerfirewalls.com/ |
| |
+-----------------------------------------------------------------------------+
| Command #1: fwaccel stat |
| |
| Check for : Accelerator Status must be enabled (R77.xx/R80.10 versions) |
| Status must be enabled (R80.20 and higher) |
| Accept Templates must be enabled |
| Message "disabled" from (low rule number) = bad |
| |
| Chapter 9: SecureXL throughput acceleration |
| Page 278 |
+-----------------------------------------------------------------------------+
| Output: |
+---------------------------------------------------------------------------------+
|Id|Name |Status |Interfaces |Features |
+---------------------------------------------------------------------------------+
|0 |SND |enabled |eth1,eth2,eth3,eth8,eth9 |Acceleration,Cryptography |
| | | | |Crypto: Tunnel,UDPEncap,MD5, |
| | | | |SHA1,NULL,3DES,DES,AES-128, |
| | | | |AES-256,ESP,LinkSelection, |
| | | | |DynamicVPN,NatTraversal, |
| | | | |AES-XCBC,SHA256,SHA384 |
+---------------------------------------------------------------------------------+

Accept Templates : disabled by Firewall
Layer CIRB incoming disables template offloads from rule #3
Throughput acceleration still enabled.
Layer RZ Inbound disables template offloads from rule #1
Throughput acceleration still enabled.
Layer OnPrem2AzureInfrastructure disables template offloads from rule #2
Throughput acceleration still enabled.
Layer Mtl2OnPremRZ disables template offloads from rule #1
Throughput acceleration still enabled.
Layer Azure2OnPrem RZ disables template offloads from rule #1
Throughput acceleration still enabled.
Layer RZ-OnPrem&Azure disables template offloads from rule #8
Throughput acceleration still enabled.
Drop Templates : enabled
NAT Templates : disabled by Firewall
Layer CIRB incoming disables template offloads from rule #3
Throughput acceleration still enabled.
Layer RZ Inbound disables template offloads from rule #1
Throughput acceleration still enabled.
Layer OnPrem2AzureInfrastructure disables template offloads from rule #2
Throughput acceleration still enabled.
Layer Mtl2OnPremRZ disables template offloads from rule #1
Throughput acceleration still enabled.
Layer Azure2OnPrem RZ disables template offloads from rule #1
Throughput acceleration still enabled.
Layer RZ-OnPrem&Azure disables template offloads from rule #8
Throughput acceleration still enabled.


+-----------------------------------------------------------------------------+
| Command #2: fwaccel stats -s |
| |
| Check for : Accelerated conns/Totals conns: >25% good, >50% great |
| Accelerated pkts/Total pkts : >50% great |
| PXL pkts/Total pkts : >50% OK |
| F2Fed pkts/Total pkts : <30% good, <10% great |
| |
| Chapter 9: SecureXL throughput acceleration |
| Page 287, Packet/Throughput Acceleration: The Three Kernel Paths |
+-----------------------------------------------------------------------------+
| Output: |
Accelerated conns/Total conns : 46/31721 (0%)
Accelerated pkts/Total pkts : 171313823924/173420969516 (98%)
F2Fed pkts/Total pkts : 2107145592/173420969516 (1%)
F2V pkts/Total pkts : 347523193/173420969516 (0%)
CPASXL pkts/Total pkts : 51117776512/173420969516 (29%)
PSLXL pkts/Total pkts : 118964632574/173420969516 (68%)
CPAS pipeline pkts/Total pkts : 0/173420969516 (0%)
PSL pipeline pkts/Total pkts : 0/173420969516 (0%)
CPAS inline pkts/Total pkts : 0/173420969516 (0%)
PSL inline pkts/Total pkts : 0/173420969516 (0%)
QOS inbound pkts/Total pkts : 0/173420969516 (0%)
QOS outbound pkts/Total pkts : 0/173420969516 (0%)
Corrected pkts/Total pkts : 0/173420969516 (0%)


+-----------------------------------------------------------------------------+
| Command #3: grep -c ^processor /proc/cpuinfo && /sbin/cpuinfo |
| |
| Check for : If number of cores is roughly double what you are excpecting, |
| hyperthreading may be enabled |
| |
| Chapter 7: CoreXL Tuning |
| Page 239 |
+-----------------------------------------------------------------------------+
| Output: |
8
HyperThreading=disabled


+-----------------------------------------------------------------------------+
| Command #4: fw ctl affinity -l -r |
| |
| Check for : SND/IRQ/Dispatcher Cores, # of CPU's allocated to interface(s) |
| Firewall Workers/INSPECT Cores, # of CPU's allocated to fw_x |
| R77.30: Support processes executed on ALL CPU's |
| R80.xx: Support processes only executed on Firewall Worker Cores|
| |
| Chapter 7: CoreXL Tuning |
| Page 221 |
+-----------------------------------------------------------------------------+
| Output: |
CPU 0: eth1 eth2 eth3
CPU 1: fw_5
fwd lpd in.acapd mta_monitor in.asessiond usrchkd dtpsd pepd mpdaemon vpnd wsdnsd rtmd cprid fwpushd in.emaild.mta rad dtlsd core_uploader pdpd in.geod cpd cprid
CPU 2: fw_3
fwd lpd in.acapd mta_monitor in.asessiond usrchkd dtpsd pepd mpdaemon vpnd wsdnsd rtmd cprid fwpushd in.emaild.mta rad dtlsd core_uploader pdpd in.geod cpd cprid
CPU 3: fw_1
fwd lpd in.acapd mta_monitor in.asessiond usrchkd dtpsd pepd mpdaemon vpnd wsdnsd rtmd cprid fwpushd in.emaild.mta rad dtlsd core_uploader pdpd in.geod cpd cprid
CPU 4:
CPU 5: fw_4
fwd lpd in.acapd mta_monitor in.asessiond usrchkd dtpsd pepd mpdaemon vpnd wsdnsd rtmd cprid fwpushd in.emaild.mta rad dtlsd core_uploader pdpd in.geod cpd cprid
CPU 6: fw_2
fwd lpd in.acapd mta_monitor in.asessiond usrchkd dtpsd pepd mpdaemon vpnd wsdnsd rtmd cprid fwpushd in.emaild.mta rad dtlsd core_uploader pdpd in.geod cpd cprid
CPU 7: fw_0
fwd lpd in.acapd mta_monitor in.asessiond usrchkd dtpsd pepd mpdaemon vpnd wsdnsd rtmd cprid fwpushd in.emaild.mta rad dtlsd core_uploader pdpd in.geod cpd cprid
All:
Interface eth8: has multi queue enabled
Interface eth9: has multi queue enabled


+-----------------------------------------------------------------------------+
| Command #5: netstat -ni |
| |
| Check for : RX/TX errors |
| RX-DRP % should be <0.1% calculated by (RX-DRP/RX-OK)*100 |
| TX-ERR might indicate Fast Ethernet/100Mbps Duplex Mismatch |
| |
| Chapter 2: Layers 1&2 Performance Optimization |
| Page 28-35 |
| |
| Chapter 7: CoreXL Tuning |
| Page 204 |
| Page 206 (Network Buffering Misses) |
+-----------------------------------------------------------------------------+
| Output: |
Kernel Interface table
Iface MTU Met RX-OK RX-ERR RX-DRP RX-OVR TX-OK TX-ERR TX-DRP TX-OVR Flg
bond1 1500 0 131205881921 0 0 0 98197788926 0 0 0 BMmRU
bond1.2 1500 0 11075878 0 49750 0 19263187 0 0 0 BMRU
bond1.11 1500 0 10190083 0 156 0 9717573 0 0 0 BMRU
bond1.100 1500 0 77510858121 0 1307019 0 78822159513 0 0 0 BMRU
bond1.106 1500 0 4380993 0 379761 0 235075028 0 0 0 BMRU
bond1.108 1500 0 17092053715 0 183806 0 5674536270 0 0 0 BMRU
bond1.112 1500 0 21332727560 0 823 0 5877593652 0 0 0 BMRU
bond1.140 1500 0 2742324 0 917 0 2677623 0 0 0 BMRU
bond1.150 1500 0 3164913167 0 59 0 642979284 0 0 0 BMRU
bond1.152 1500 0 192985 0 0 0 149949 0 0 0 BMRU
bond1.160 1500 0 827531034 0 24259 0 229120864 0 0 0 BMRU
bond1.170 1500 0 13975269 0 31 0 12763680 0 0 0 BMRU
bond1.171 1500 0 19 0 0 0 42795 0 0 0 BMRU
bond1.355 1500 0 5484716373 0 0 0 4311281028 0 0 0 BMRU
bond1.500 1500 0 3778665907 0 2438 0 1726948704 0 0 0 BMRU
bond1.550 1500 0 565614091 0 17 0 192662881 0 0 0 BMRU
bond1.560 1500 0 585068115 0 202 0 215526710 0 0 0 BMRU
bond1.570 1500 0 815696026 0 358 0 238497128 0 0 0 BMRU
bond1.804 1500 0 5303533 0 0 0 9059566 0 0 0 BMRU
eth1 1500 0 7191113 0 6 0 8530266 0 0 0 BMRU
eth2 1500 0 42402513961 0 321122427 0 114851655091 0 0 0 BMRU
eth3 1500 0 559319557 0 333462 0 546147561 0 0 0 BMRU
eth8 1500 0 63152389162 0 0 0 13837171475 0 0 0 BMsRU
eth9 1500 0 68053487950 0 0 0 84360614272 0 0 0 BMsRU
lo 65536 0 81592231 0 0 0 81592231 0 0 0 ALMPRU

interface eth1: There were no RX drops in the past 0.5 seconds(B
interface eth1 rx_missed_errors :
interface eth1 rx_fifo_errors :
interface eth1 rx_no_buffer_count:

interface eth2: There were no RX drops in the past 0.5 seconds(B
interface eth2 rx_missed_errors :
interface eth2 rx_fifo_errors :
interface eth2 rx_no_buffer_count:

interface eth3: There were no RX drops in the past 0.5 seconds(B
interface eth3 rx_missed_errors :
interface eth3 rx_fifo_errors :
interface eth3 rx_no_buffer_count:

interface eth8: There were no RX drops in the past 0.5 seconds(B
interface eth8 rx_missed_errors : 0
interface eth8 rx_fifo_errors : 0
interface eth8 rx_no_buffer_count: 0

interface eth9: There were no RX drops in the past 0.5 seconds(B
interface eth9 rx_missed_errors : 0
interface eth9 rx_fifo_errors : 0
interface eth9 rx_no_buffer_count: 0

 

+-----------------------------------------------------------------------------+
| Command #6: fw ctl multik stat |
| |
| Check for : Large # of conns on Worker 0 - IPSec VPN/VoIP? |
| Large imbalance of connections on a single or multiple Workers |
| |
| Chapter 7: CoreXL Tuning |
| Page 241 |
| |
| Chapter 8: CoreXL VPN Optimization |
| Page 256 |
+-----------------------------------------------------------------------------+
| Output: |
ID | Active | CPU | Connections | Peak
----------------------------------------------
0 | Yes | 7 | 8073 | 11972
1 | Yes | 3 | 2891 | 9650
2 | Yes | 6 | 7990 | 10876
3 | Yes | 2 | 7732 | 10735
4 | Yes | 5 | 2764 | 8529
5 | Yes | 1 | 4610 | 9798

+-----------------------------------------------------------------------------+
| Command #7: cpstat os -f multi_cpu -o 1 -c 5 |
| |
| Check for : High SND/IRQ Core Utilization |
| High Firewall Worker Core Utilization |
| |
| Chapter 6: CoreXL & Multi-Queue |
| Page 173 |
+-----------------------------------------------------------------------------+
| Output: |

 

Processors load
---------------------------------------------------------------------------------
|CPU#|User Time(%)|System Time(%)|Idle Time(%)|Usage(%)|Run queue|Interrupts/sec|
---------------------------------------------------------------------------------
| 1| 0| 76| 24| 76| ?| 42582|
| 2| 1| 99| 1| 99| ?| 42582|
| 3| 7| 58| 35| 65| ?| 42582|
| 4| 0| 100| 0| 100| ?| 42582|
| 5| 0| 8| 92| 8| ?| 42582|
| 6| 0| 100| 0| 100| ?| 42582|
| 7| 5| 62| 33| 67| ?| 42582|
| 8| 4| 70| 26| 74| ?| 42582|
---------------------------------------------------------------------------------

 

 

Processors load
---------------------------------------------------------------------------------
|CPU#|User Time(%)|System Time(%)|Idle Time(%)|Usage(%)|Run queue|Interrupts/sec|
---------------------------------------------------------------------------------
| 1| 0| 76| 24| 76| ?| 42582|
| 2| 1| 99| 1| 99| ?| 42582|
| 3| 7| 58| 35| 65| ?| 42582|
| 4| 0| 100| 0| 100| ?| 42582|
| 5| 0| 8| 92| 8| ?| 42582|
| 6| 0| 100| 0| 100| ?| 42582|
| 7| 5| 62| 33| 67| ?| 42582|
| 8| 4| 70| 26| 74| ?| 42582|
---------------------------------------------------------------------------------

 

 

Processors load
---------------------------------------------------------------------------------
|CPU#|User Time(%)|System Time(%)|Idle Time(%)|Usage(%)|Run queue|Interrupts/sec|
---------------------------------------------------------------------------------
| 1| 0| 78| 22| 78| ?| 87287|
| 2| 0| 100| 0| 100| ?| 43644|
| 3| 12| 49| 39| 61| ?| 87290|
| 4| 0| 100| 0| 100| ?| 43646|
| 5| 0| 7| 93| 7| ?| 87296|
| 6| 0| 100| 0| 100| ?| 43648|
| 7| 2| 79| 19| 81| ?| 87299|
| 8| 3| 67| 30| 70| ?| 87303|
---------------------------------------------------------------------------------

 

 

Processors load
---------------------------------------------------------------------------------
|CPU#|User Time(%)|System Time(%)|Idle Time(%)|Usage(%)|Run queue|Interrupts/sec|
---------------------------------------------------------------------------------
| 1| 0| 78| 22| 78| ?| 87287|
| 2| 0| 100| 0| 100| ?| 43644|
| 3| 12| 49| 39| 61| ?| 87290|
| 4| 0| 100| 0| 100| ?| 43646|
| 5| 0| 7| 93| 7| ?| 87296|
| 6| 0| 100| 0| 100| ?| 43648|
| 7| 2| 79| 19| 81| ?| 87299|
| 8| 3| 67| 30| 70| ?| 87303|
---------------------------------------------------------------------------------

 

 

Processors load
---------------------------------------------------------------------------------
|CPU#|User Time(%)|System Time(%)|Idle Time(%)|Usage(%)|Run queue|Interrupts/sec|
---------------------------------------------------------------------------------
| 1| 0| 78| 22| 78| ?| 84875|
| 2| 0| 100| 0| 100| ?| 42437|
| 3| 7| 55| 38| 62| ?| 84873|
| 4| 0| 100| 0| 100| ?| 42438|
| 5| 0| 7| 93| 7| ?| 84875|
| 6| 0| 100| 0| 100| ?| 42437|
| 7| 1| 79| 20| 80| ?| 84883|
| 8| 3| 60| 37| 63| ?| 84882|
---------------------------------------------------------------------------------


+-----------------------------------------------------------------------------+
| Thanks for using s7pac |
+-----------------------------------------------------------------------------+

0 Kudos
flachance
Advisor
‎2022-10-14 09:56 AM
In response to PhoneBoy
Jump to solution

 
Preview file
22 KB
0 Kudos
PhoneBoy
Admin
Admin
‎2022-10-14 10:54 AM
In response to flachance
Jump to solution

This is a bit concerning:

Accept Templates : disabled by Firewall
                   Layer CIRB incoming disables template offloads from rule #3
                   Throughput acceleration still enabled.
                   Layer RZ Inbound disables template offloads from rule #1
                   Throughput acceleration still enabled.
                   Layer OnPrem2AzureInfrastructure disables template offloads from rule #2
                   Throughput acceleration still enabled.
                   Layer Mtl2OnPremRZ disables template offloads from rule #1
                   Throughput acceleration still enabled.
                   Layer Azure2OnPrem RZ disables template offloads from rule #1
                   Throughput acceleration still enabled.
                   Layer RZ-OnPrem&Azure disables template offloads from rule #8
                   Throughput acceleration still enabled.

 
That said, most of your packets are accelerated, so this may not be a big deal.
You do have a large number of receive drops on your bond interface and on eth2, which might be something worth investigating.

What version/JHF are we working with here on what precise hardware?
output of enabled_blades would also be helpful to further contextualize this.

Also, you have 10 connections that are taking 92% of the CPU.
What precisely are these connections for and what precise policies relate to them?
Depending on what they are (and if they are trusted), we can fully accelerate them using fast_accel to reduce the overall CPU load.

0 Kudos
flachance
Advisor
‎2022-10-14 11:03 AM
In response to PhoneBoy
Jump to solution

I looked at the mentioned layers and rule numbers where it says disables template offload but I don't see/notice anything particular about them.

We're running R80.40 JHF 173 on a pair of HP Proliant servers. The connections taking all that CPU are from the backup server so they would be trusted. Maybe it's always been like that and I just happened to notice it today but I don't think so. Is it easy to fully accelerate the connections from that server?

0 Kudos
PhoneBoy
Admin
Admin
‎2022-10-14 11:08 AM
In response to flachance
Jump to solution

Yes, using fw ctl fast_accel.
Refer to: https://supportcenter.checkpoint.com/supportcenter/portal?eventSubmit_doGoviewsolutiondetails=&solut... 

0 Kudos
flachance
Advisor
‎2022-10-14 11:15 AM
In response to PhoneBoy
Jump to solution

once added should it be effective immediatly or just on new connections?

0 Kudos
PhoneBoy
Admin
Admin
‎2022-10-14 11:23 AM
In response to flachance
Jump to solution

Given how SecureXL works in R80.20 and above, I believe it should be effective immediately.
However, it might require a new connection to be established.

0 Kudos

Leaderboard

Epsum factorial non deposit quid pro quo hic escorol.

View All ≫
Upcoming Events

    Sort by:
    CheckMates Events