- Products
- Learn
- Local User Groups
- Partners
-
More
Celebrate the New Year
With CheckMates!
Value of Security
Vendor Self-Awareness
Join Us for CPX 360
23-24 February 2021
Important certificate update to CloudGuard Controller, CME,
and Azure HA Security Gateways
How to Remediate Endpoint & VPN
Issues (in versions E81.10 or earlier)
Mobile Security
Buyer's Guide Out Now
Important! R80 and R80.10
End Of Support around the corner (May 2021)
We have a CheckPoint 4200 appliance running as our gateway/firewall. Our WAN speed is 1Gbps, but we can only seem to get 100Mbps throughput from the appliance.
I have connected a computer directly to our WAN-connection to confirm WAN speed, and without going through the firewall i get the correct speed (1Gbps).
The WAN interface (eth1) says "Link Speed: 1000Mbps / Full Duplex".
I have been monitoring with CPview on the firewall, and I have not seen "Total Mbits/sec" go above 102 Mbps. To me it seems like speed is capped at 100Mbps. I am wondering what the cause of this can be, and what steps should I do to troubleshoot this issue? Appreciate any help.
Try from different clients at the same time - and add up the throughputs...
I have, and the speed still doesn't exceed 100Mbps.
Hello,
Regards,
Hello,
Q: Which version are you running?
A: R77.30.
Q: Which blades are you running?
A:
Q: Please post output of fwaccel stats -s (or stat, don't remember right now)
A:
Accelerator Status : on
Accept Templates : disabled by Firewall
disabled from rule #12
Drop Templates : disabled
NAT Templates : disabled by user
Accelerator Features : Accounting, NAT, Cryptography, Routing,
HasClock, Templates, Synchronous, IdleDetection,
Sequencing, TcpStateDetect, AutoExpire,
DelayedNotif, TcpStateDetectV2, CPLS, McastRouting,
WireMode, DropTemplates, NatTemplates,
Streaming, MultiFW, AntiSpoofing, Nac,
ViolationStats, AsychronicNotif, ERDOS,
NAT64, GTPAcceleration, SCTPAcceleration,
McastRoutingV2
Cryptography Features : Tunnel, UDPEncapsulation, MD5, SHA1, NULL,
3DES, DES, CAST, CAST-40, AES-128, AES-256,
ESP, LinkSelection, DynamicVPN, NatTraversal,
EncRouting, AES-XCBC, SHA256
Q: From expert (#) run ifconfig -a . Do you see errors on the interfaces?
A: I only see 22 errors on the trunked interface...
Q: Did you tried to connect your host directly to the firewall to perform the speed test?
A: No.
Regards.
Q: Which version are you running?
A: R77.30.
--> having support for eleven more days, so what about the future ?
The maximum speed through a 2core box like 4200 will depend on which blades are enabled (enabled_blades command), and how much traffic is being pulled into the PXL or F2F paths based on your APCL/URLF and Threat Prevention policies. Please provide the output from the "Super Seven" commands run on your firewall for further analysis:
Hello!
Super Seven output:
fwaccel stat
Accelerator Status : on
Accept Templates : disabled by Firewall
disabled from rule #12
Drop Templates : disabled
NAT Templates : disabled by user
Accelerator Features : Accounting, NAT, Cryptography, Routing,
HasClock, Templates, Synchronous, IdleDetection,
Sequencing, TcpStateDetect, AutoExpire,
DelayedNotif, TcpStateDetectV2, CPLS, McastRouting,
WireMode, DropTemplates, NatTemplates,
Streaming, MultiFW, AntiSpoofing, Nac,
ViolationStats, AsychronicNotif, ERDOS,
NAT64, GTPAcceleration, SCTPAcceleration,
McastRoutingV2
Cryptography Features : Tunnel, UDPEncapsulation, MD5, SHA1, NULL,
3DES, DES, CAST, CAST-40, AES-128, AES-256,
ESP, LinkSelection, DynamicVPN, NatTraversal,
EncRouting, AES-XCBC, SHA256
fwaccel stats -s
grep -c ^processor /proc/cpuinfo
fw ctl affinity -l -r
netstat -ni
Iface | MTU | Met | RX-OK | RX-ERR | RX-DRP | RX-OVR | TX-OK | TX-ERR | TX-DRP | TX-OVR | Flg |
Mgmt | 1500 | 0 | 352341 | 0 | 0 | 0 | 315754 | 0 | 0 | 0 | BMRU |
eth1 | 1500 | 0 | 195704407 | 0 | 3154326 | 0 | 102409603 | 0 | 0 | 0 | BMRU |
eth2 | 1500 | 0 | 251686 | 0 | 1549 | 0 | 150148 | 0 | 0 | 0 | BMRU |
eth3 | 1500 | 0 | 104346397 | 23 | 385319 | 0 | 189718282 | 0 | 0 | 0 | BMRU |
eth3.3 | 1500 | 0 | 390303 | 0 | 0 | 0 | 46120 | 0 | 0 | 0 | BMRU |
eth3.5 | 1500 | 0 | 97345634 | 0 | 0 | 0 | 181543112 | 0 | 0 | 0 | BMRU |
eth3.6 | 1500 | 0 | 5467821 | 0 | 0 | 0 | 8966394 | 0 | 0 | 0 | BMRU |
eth3.7 | 1500 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | BMRU |
eth3.9 | 1500 | 0 | 79155 | 0 | 0 | 0 | 1099 | 0 | 0 | 0 | BMRU |
eth3.10 | 1500 | 0 | 473634 | 0 | 0 | 0 | 260472 | 0 | 0 | 0 | BMRU |
eth3.15 | 1500 | 0 | 81049 | 0 | 0 | 0 | 8108 | 0 | 0 | 0 | BMRU |
eth3.20 | 1500 | 0 | 508709 | 0 | 0 | 0 | 229251 | 0 | 0 | 0 | BMRU |
lo | 16436 | 0 | 1101289 | 0 | 0 | 0 | 1101289 | 0 | 0 | 0 | LRU |
fw ctl multik stat
cpstat os -f multi_cpu -o 1
Processors load
---------------------------------------------------------------------------------
|CPU#|User Time(%)|System Time(%)|Idle Time(%)|Usage(%)|Run queue|Interrupts/sec|
---------------------------------------------------------------------------------
| 1| 1| 79| 20| 80| ?| 2183|
| 2| 4| 62| 34| 66| ?| 2183|
---------------------------------------------------------------------------------
Processors load
---------------------------------------------------------------------------------
|CPU#|User Time(%)|System Time(%)|Idle Time(%)|Usage(%)|Run queue|Interrupts/sec|
---------------------------------------------------------------------------------
| 1| 1| 79| 20| 80| ?| 2183|
| 2| 4| 62| 34| 66| ?| 2183|
---------------------------------------------------------------------------------
Processors load
---------------------------------------------------------------------------------
|CPU#|User Time(%)|System Time(%)|Idle Time(%)|Usage(%)|Run queue|Interrupts/sec|
---------------------------------------------------------------------------------
| 1| 2| 85| 13| 87| ?| 2272|
| 2| 17| 51| 32| 68| ?| 2272|
---------------------------------------------------------------------------------
Processors load
---------------------------------------------------------------------------------
|CPU#|User Time(%)|System Time(%)|Idle Time(%)|Usage(%)|Run queue|Interrupts/sec|
---------------------------------------------------------------------------------
| 1| 2| 85| 13| 87| ?| 2272|
| 2| 17| 51| 32| 68| ?| 2272|
---------------------------------------------------------------------------------
Processors load
---------------------------------------------------------------------------------
|CPU#|User Time(%)|System Time(%)|Idle Time(%)|Usage(%)|Run queue|Interrupts/sec|
---------------------------------------------------------------------------------
| 1| 0| 83| 16| 84| ?| 2235|
| 2| 11| 43| 47| 53| ?| 2235|
---------------------------------------------------------------------------------
Processors load
---------------------------------------------------------------------------------
|CPU#|User Time(%)|System Time(%)|Idle Time(%)|Usage(%)|Run queue|Interrupts/sec|
---------------------------------------------------------------------------------
| 1| 0| 83| 16| 84| ?| 2235|
| 2| 11| 43| 47| 53| ?| 2235|
---------------------------------------------------------------------------------
Processors load
---------------------------------------------------------------------------------
|CPU#|User Time(%)|System Time(%)|Idle Time(%)|Usage(%)|Run queue|Interrupts/sec|
---------------------------------------------------------------------------------
| 1| 0| 85| 14| 86| ?| 2254|
| 2| 1| 37| 63| 37| ?| 2254|
---------------------------------------------------------------------------------
Processors load
---------------------------------------------------------------------------------
|CPU#|User Time(%)|System Time(%)|Idle Time(%)|Usage(%)|Run queue|Interrupts/sec|
---------------------------------------------------------------------------------
| 1| 0| 85| 14| 86| ?| 2254|
| 2| 1| 37| 63| 37| ?| 2254|
---------------------------------------------------------------------------------
Cheers.
You are getting frame loss (RX-DRP) rates of between 0.3% and 1.6% on your interfaces due to buffering misses which is probably the main thing slowing you down. This is almost certainly due to high CPU load on your 2 cores, given the large number blades you have enabled on an old 2-core box like that 4200, 100Mbps top throughput doesn't seem that unreasonable to me. Currently you have a 2/2 CoreXL split on your box, in some cases disabling CoreXL and going to a 1/1 split helps on a 2-core box but given your high PXL% I don't think doing that will help in this case.
The 4200 only has 4GB of RAM which may not be enough for all you are trying to do. Please provide output of the free -m command to see if a memory upgrade will help.
You can probably pick up some more speed by tuning your policies, the two major areas in your case are Threat Prevention and APCL/URLF. In order to figure out where to focus your efforts, try this and report back what you see:
1) Run Internet speed test and note throughput
2) On the gateway from expert mode run commands ips off and fw amw unload
3) Wait 60 seconds
4) From a completely new browser instance run an Internet speed test and note throughput. If throughput has substantially increased you need to tune your IPS & Threat Prevention configuration.
5) Run commands ips on and fw amw fetch local
6) Wait 60 seconds
7) From a completely new browser instance run an Internet speed test and note throughput. (should be about the same as #1)
😎 On gateway object in SmartConsole, uncheck the APCL and URLF blades and reinstall policy to the gateway.
9) Wait 60 seconds
9) From a completely new browser instance run an Internet speed test and note throughput. If throughput has substantially increased you need to tune your APCL/URLF policy, typically this will involve removing the "Any Any Any Accept" rule at the bottom of your APCL/URLF policy (which is not necessary except for logging purposes), and making sure you are using object "Internet" in the Destination column of all APCL/URLF rules and NOT "Any".
10) Recheck the APCL and URLF checkboxes and reinstall policy to the gateway.
11) From a completely new browser instance run an Internet speed test and note throughput. (should be about the same as #1)
Let us know what you find out.
Thanks for the input!
I’ve done the steps you suggested, and I found this:
free -m command:
| total | used | free | shared | buffers | cached |
Mem: | 3973 | 3289 | 684 | 0 | 34 | 834 |
-/+ buffers/cache: |
| 2420 | 1553 |
|
|
|
Swap: | 10268 | 0 | 10268 |
|
|
|
Throughput tests (peaks - CPview):
Without any changes: 74 Mbps
ips off & fw amw unload: 234 Mbps
Reverted (ips on & fw amw fetch local) 93 Mbps
APCL & URLF blades disabled: 115 Mbps
Reverted (APCL & URLF enabled) 95 Mbps
So it seems like the IPS & Threat Prevention needs tuning. Do you have any suggestions for that?
I will do your suggested tuning for APCL/URLF also.
Looks like your box is not hitting swap at all which is good, no memory upgrade needed.
We'll need to do a few more tests to determine whether it is IPS specifically (more likely) or the rest of Threat Prevention (less likely) that is causing the bulk of the slowdown:
1) Run Internet speed test and note throughput
2) On the gateway from expert mode run commands ips off
3) Wait 60 seconds
4) From a completely new browser instance run an Internet speed test and note throughput. If throughput has substantially increased you need to tune your IPS configuration.
5) Run command ips on
6) Run command fw amw unload
7) Wait 60 seconds
😎 From a completely new browser instance run an Internet speed test and note throughput. If throughput has substantially increased you need to tune your TP (AV/ABOT) configuration.
9) Run command fw amw fetch local
You may well see a performance improvement at both steps #4 & #8, I'd suggest focusing on where you get the biggest increase for tuning. If turning off IPS provides most of the gain, determine which IPS profile is in use by your 4200 gateway and open it for editing. Sort the IPS protections by the "Performance Impact" rating and disable all IPS Protections with a "Critical" or "High" rating. That should help a lot.
If turning off Threat Prevention (amw) provided most of the gain, my guess is that Anti-virus is causing most of the overhead as Anti-bot tends to be pretty low impact. I'll need to see the AV & ABOT settings in the relevant TP profile applied to your gateway to make specific recommendations.
About CheckMates
Learn Check Point
Advanced Learning
WELCOME TO THE FUTURE OF CYBER SECURITY