Create a Post
cancel
Showing results for 
Search instead for 
Did you mean: 
Blason_R
Leader
Leader

CPU spiking on core 2 and core 6

Hi Team,

 

I have open server having 8 core processor and for last couple of days I am observing continuous CPU spike on Core 2 and core 6. This is R80.10. Surpringly my SND core is no spiking but only core 2, 6 & sometimes 7 is spiking.

This is carrying SSL VPN for around 450 users. We even increased rx-ringsize

Disabled URL filtering

Have put certain traffic on fast path using sim fastaccel

 

Any other help?

 

Thanks and Regards,
Blason R
CCSA,CCSE,CCCS
8 Replies
Vladimir
Champion
Champion

Check the traffic for the presence of the "elephant flows" to see if those are causing the issue: https://community.checkpoint.com/t5/General-Topics/R80-x-Performance-Tuning-Tip-Elephant-Flows-Heavy...

 

the_rock
Legend
Legend

I cant speak for anyone else, but the best way I found to solve issue like this is following (if its cluster, it has to be done on all members):

-cpconfig

-option for corexl, choose disable, reboot

-cpconfig again, re-enable corexl, reboot

-check again

Andy

Timothy_Hall
Champion
Champion

Almost certainly the presence of elephant flows as Vladimir said.  Use command fw ctl gconn to see what specific connections are being handled on the saturated cores, and fw ctl multik print_heavy_conn to see if the firewall has identified any elephant flows during the last 24 hours.  fastaccel can help if you correctly identify the elephant flows and fastpath them, but fastaccel will not work on corrections that must go F2F/slowpath for some reason.

Gateway Performance Optimization R81.20 Course
now available at maxpowerfirewalls.com
0 Kudos
Blason_R
Leader
Leader

Hi,

Thanks for the reply however considering R80.10 print heavy command gconn commands are not available. Is there any other way to identify?

Thanks and Regards,
Blason R
CCSA,CCSE,CCCS
Timothy_Hall
Champion
Champion

After the fact, no.  If you can catch the heavy utilization on those cores "in the act", try these live cpview screens (they won't be populated in cpview -t's historical mode):

CPU->Top Connections

Advanced->CoreXL->Instances-> FW-InstanceX->Top-FW-Lock consumers (you'll need to hit "a" to activate statistics)

If these screens don't help or are not available in your code version, you'll have to do it the old-fashioned way as described here: sk103293: How to get per CPU statistics and TOP FW-lock consumers with cpkstats

 

Gateway Performance Optimization R81.20 Course
now available at maxpowerfirewalls.com
0 Kudos
Blason_R
Leader
Leader

Yes I tried that but nothing specifically being pointed out. However I see connections destined to firewall IP with port 443 which are SSL VPN connections whenever CPUs are spiked and those are in F2F path.

Thanks and Regards,
Blason R
CCSA,CCSE,CCCS
0 Kudos
Vladimir
Champion
Champion

I've tried to reply to you earlier, but it somehow did not post: If you have SmartEvent in your environment, you can try using "High Bandwidth Applications" view, filter it down to the timeframe of the incident and deep-dive in the results to pin-down the participants.

0 Kudos
Blason_R
Leader
Leader

Hi Team,

 

Thanks for the heads-up. After struggling for about 2 weeks and really no response from TAC I finally upgraded the setup to 80.30 with maximum rx-ringsize and LACP created for LAN interface.

Seems like the no complaints received at least for past two days.

 

Thanks and Regards,
Blason R
CCSA,CCSE,CCCS
0 Kudos

Leaderboard

Epsum factorial non deposit quid pro quo hic escorol.

Upcoming Events

    CheckMates Events