Advice to troubleshoot CPU spike on SMB 1470 Gateway

[JustinLow]

Hi All,

I have an issue regarding the CPU spike on SMB 1470 gateway, I had received many alert notification from SMS regarding the CPU spike for one of the gateway. I want to troubleshoot on the root cause, however I have no clue to check further.

This is what I done for checking,

1) Checked /var/log/messages, there is no meaningful log inside it

2) Executed top command in expert mode to check CPU utilization, CPU utilization seem okay when I'm checking

3) Checked connection table, traffic is not congested

Can anyone advice on this?

[Lesley]

Use the cpview tool for this during the issue. 

or cpview -t  -> then in next window press t again, there you can enter time and date of the issue and see history. 

This tool should give you all the info you need to troubleshoot this further. 

[Chris_Atkinson]

Out of interest are you running firmware R77_990173127_20.img or something else?

Does the issue coincide with a policy install?

See also: https://community.checkpoint.com/t5/SMB-Gateways-Spark/Brief-introduction-to-SMB-performance-tuning/...

Note the 1470 will go End of Support in October 2024.

[JustinLow]

Hi Chris, 

Thank you for your reply. The firmware that the firewall running now is R77.20.81 build 541. The issue is the SMS will sending the high CPU alert email notification to me at frequent. I checked the hardware resource and all seem okay to me. I have checked the system log also but found nothing meaningful log inside the firewall. 

I have another SMB firewall model 1450 and 1470 that running firmware R77.20.80. build 392 and managed by the same SMS. The other firewalls don't have this kind of issue which will keep on sending high CPU alert notification.

I found another post from the community, https://community.checkpoint.com/t5/SMB-Gateways-Spark/R77-20-85-performance-issue-on-centrally-mana...

The replies inside post has mentioned that there is an instability performance issue on the the firmware R77.20.81 build 541. However I can't found related sk that mentioned for the instability issue from the CheckPoint official support portal.

Hope you can provide my some suggestions on this. 

[Tom_Hinoue]

R77.20.81 is very very old and you definitely need to upgrade to R77.20.87 latest image, since it includes various stability fixes compared to R77.20.81/R77.20.85.

Also it may depend on your amount of traffic, but one fix in R77.20.87 that may prevent CPU spike is that the CRL file size inspection limit was limited to only inspect CRL below 10000 entries. If you cannot upgrade than you can maybe disable CRL validation to check if the issue mitigates as well.

[JustinLow]

Hi Tom,

May I know how to verify that CPU spike is due to the CRL validation? Is there any CRL validation log inside system log or firewall log in the device?

[Tom_Hinoue]

I believe this is something not logged by default and would require specific debugs, which was done in our TAC case back then introducing the CRL entry limit in R77.20.87. You can maybe consult with TAC, but I believe they will first inform you to upgrade to R77.20.87 latest build.

Note as Chris mentioned, 700/1400 appliance will be EoS later this year, so you may want to start your plans to replace/upgrading the device to 1500/1600/1800.

[Chris_Atkinson]

Both of the listed firmware versions are no longer recommended / supported and an upgrade is appropriate for stability & performance reasons. 

Note a multi-step upgrade might be applicable in some cases (sk143274).

Refer also:

https://www.checkpoint.com/support-services/support-life-cycle-policy/#embedded-security

[JustinLow]

Hi Chris,

I will proceed to upgrade the firmware to resolve the CPU spike issue and get the firewall stable performance first. Thank you for sharing the multi-step upgrade SK.

[Lesley]

Is this topic not to much focused on finding a bug? Of course old software is not good. But checking why the box is loaded would be first step. Also does not take a lot of time or is complex. 

This CRL bug is very specific and without indication for this bug I would not recommend to troubleshoot it. 

[Chris_Atkinson]

Perhaps, but it's not without basis:

https://community.checkpoint.com/t5/SMB-Gateways-Spark/R77-20-81-build-541-high-CPU-usage/td-p/10430...

https://community.checkpoint.com/t5/Security-Gateways/1200R-cpu-usage-history-show-above-90-load-per...

[Tom_Hinoue]

I believe I heard from TAC back then that the CRL issue is not considered as as bug, but a feature added to address the impact to the CPU due to the size of the CRL file in modern internet, as performance is limited on these SMB boxes.