cancel
Showing results for 
Search instead for 
Did you mean: 
Create a Post
Highlighted
exciteman
Ivory

Low throughput from 4200 appliance

We have a CheckPoint 4200 appliance running as our gateway/firewall. Our WAN speed is 1Gbps, but we can only seem to get 100Mbps throughput from the appliance.

I have connected a computer directly to our WAN-connection to confirm WAN speed, and without going through the firewall i get the correct speed (1Gbps). 

The WAN interface (eth1) says "Link Speed: 1000Mbps / Full Duplex".

I have been monitoring with CPview on the firewall, and I have not seen "Total Mbits/sec" go above 102 Mbps. To me it seems like speed is capped at 100Mbps. I am wondering what the cause of this can be, and what steps should I do to troubleshoot this issue? Appreciate any help. 

0 Kudos
10 Replies

Re: Low throughput from 4200 appliance

Try from different clients at the same time - and add up the throughputs...

0 Kudos
exciteman
Ivory

Re: Low throughput from 4200 appliance

I have, and the speed still doesn't exceed 100Mbps.

0 Kudos

Re: Low throughput from 4200 appliance

Hello,

  • Which version are you running? 
  • Which blades are you running?
  • Please post output of fwaccel stats -s (or stat, don't remember right now)
  • From expert (#) run ifconfig -a . Do you see errors on the interfaces? 
  • Output from top
  • Did you tried to connect your host directly to the firewall to perform the speed test?

Regards,

 

____________
https://www.linkedin.com/in/federicomeiners/
0 Kudos
exciteman
Ivory

Re: Low throughput from 4200 appliance

Hello,

 

Q: Which version are you running? 

A: R77.30.

Q: Which blades are you running?

A: Enabled blades.PNG

Q: Please post output of fwaccel stats -s (or stat, don't remember right now)

A:

Spoiler

Accelerator Status : on
Accept Templates : disabled by Firewall
disabled from rule #12
Drop Templates : disabled
NAT Templates : disabled by user

Accelerator Features : Accounting, NAT, Cryptography, Routing,
HasClock, Templates, Synchronous, IdleDetection,
Sequencing, TcpStateDetect, AutoExpire,
DelayedNotif, TcpStateDetectV2, CPLS, McastRouting,
WireMode, DropTemplates, NatTemplates,
Streaming, MultiFW, AntiSpoofing, Nac,
ViolationStats, AsychronicNotif, ERDOS,
NAT64, GTPAcceleration, SCTPAcceleration,
McastRoutingV2
Cryptography Features : Tunnel, UDPEncapsulation, MD5, SHA1, NULL,
3DES, DES, CAST, CAST-40, AES-128, AES-256,
ESP, LinkSelection, DynamicVPN, NatTraversal,
EncRouting, AES-XCBC, SHA256

 

Q: From expert (#) run ifconfig -a . Do you see errors on the interfaces? 

A: I only see 22 errors on the trunked interface...

Q: Did you tried to connect your host directly to the firewall to perform the speed test?

A: No.

 

Regards.

0 Kudos

Re: Low throughput from 4200 appliance

Q: Which version are you running? 

A: R77.30.

--> having support for eleven more days, so what about the future ?

0 Kudos

Re: Low throughput from 4200 appliance

The maximum speed through a 2core box like 4200 will depend on which blades are enabled (enabled_blades command), and how much traffic is being pulled into the PXL or F2F paths based on your APCL/URLF and Threat Prevention policies.  Please provide the output from the "Super Seven" commands run on your firewall for further analysis:

https://community.checkpoint.com/t5/General-Topics/Super-Seven-Performance-Assessment-Commands-s7pac...

 

"IPS Immersion Training" Self-paced Video Class
Now Available at http://www.maxpowerfirewalls.com
0 Kudos
exciteman
Ivory

Re: Low throughput from 4200 appliance

Hello!

Super Seven output:

fwaccel stat

Spoiler

Accelerator Status : on
Accept Templates : disabled by Firewall
disabled from rule #12
Drop Templates : disabled
NAT Templates : disabled by user

Accelerator Features : Accounting, NAT, Cryptography, Routing,
HasClock, Templates, Synchronous, IdleDetection,
Sequencing, TcpStateDetect, AutoExpire,
DelayedNotif, TcpStateDetectV2, CPLS, McastRouting,
WireMode, DropTemplates, NatTemplates,
Streaming, MultiFW, AntiSpoofing, Nac,
ViolationStats, AsychronicNotif, ERDOS,
NAT64, GTPAcceleration, SCTPAcceleration,
McastRoutingV2


Cryptography Features : Tunnel, UDPEncapsulation, MD5, SHA1, NULL,
3DES, DES, CAST, CAST-40, AES-128, AES-256,
ESP, LinkSelection, DynamicVPN, NatTraversal,
EncRouting, AES-XCBC, SHA256

fwaccel stats -s

Spoiler
Accelerated conns/Total conns : 0/607 (0%)
Accelerated pkts/Total pkts : 40/4009649 (0%)
F2Fed pkts/Total pkts : 216182/4009649 (5%)
PXL pkts/Total pkts : 3793427/4009649 (94%)
QXL pkts/Total pkts : 0/4009649 (0%)

grep -c ^processor /proc/cpuinfo

Spoiler
2

fw ctl affinity -l -r

Spoiler
CPU 0: eth2 eth3
fw_1
CPU 1: eth1 Mgmt
fw_0
All: usrchkd mpdaemon in.acapd vpnd lpd rad fwd in.msd fwpushd cprid cpd

netstat -ni

Spoiler
Kernel Interface table
IfaceMTUMetRX-OKRX-ERRRX-DRPRX-OVRTX-OKTX-ERRTX-DRPTX-OVRFlg
Mgmt15000352341000315754000BMRU
eth115000195704407031543260102409603000BMRU
eth215000251686015490150148000BMRU
eth315000104346397233853190189718282000BMRU
eth3.31500039030300046120000BMRU
eth3.51500097345634000181543112000BMRU
eth3.61500054678210008966394000BMRU
eth3.71500000000000BMRU
eth3.915000791550001099000BMRU
eth3.1015000473634000260472000BMRU
eth3.1515000810490008108000BMRU
eth3.2015000508709000229251000BMRU
lo16436011012890001101289000LRU

fw ctl multik stat

Spoiler
ID | Active | CPU | Connections | Peak
----------------------------------------------
0 | Yes | 1 | 271 | 3596
1 | Yes | 0 | 368 | 3657

cpstat os -f multi_cpu -o 1

Spoiler

Processors load
---------------------------------------------------------------------------------
|CPU#|User Time(%)|System Time(%)|Idle Time(%)|Usage(%)|Run queue|Interrupts/sec|
---------------------------------------------------------------------------------
| 1| 1| 79| 20| 80| ?| 2183|
| 2| 4| 62| 34| 66| ?| 2183|
---------------------------------------------------------------------------------

 

 

Processors load
---------------------------------------------------------------------------------
|CPU#|User Time(%)|System Time(%)|Idle Time(%)|Usage(%)|Run queue|Interrupts/sec|
---------------------------------------------------------------------------------
| 1| 1| 79| 20| 80| ?| 2183|
| 2| 4| 62| 34| 66| ?| 2183|
---------------------------------------------------------------------------------

 

 

Processors load
---------------------------------------------------------------------------------
|CPU#|User Time(%)|System Time(%)|Idle Time(%)|Usage(%)|Run queue|Interrupts/sec|
---------------------------------------------------------------------------------
| 1| 2| 85| 13| 87| ?| 2272|
| 2| 17| 51| 32| 68| ?| 2272|
---------------------------------------------------------------------------------

 

 

Processors load
---------------------------------------------------------------------------------
|CPU#|User Time(%)|System Time(%)|Idle Time(%)|Usage(%)|Run queue|Interrupts/sec|
---------------------------------------------------------------------------------
| 1| 2| 85| 13| 87| ?| 2272|
| 2| 17| 51| 32| 68| ?| 2272|
---------------------------------------------------------------------------------

 

 

Processors load
---------------------------------------------------------------------------------
|CPU#|User Time(%)|System Time(%)|Idle Time(%)|Usage(%)|Run queue|Interrupts/sec|
---------------------------------------------------------------------------------
| 1| 0| 83| 16| 84| ?| 2235|
| 2| 11| 43| 47| 53| ?| 2235|
---------------------------------------------------------------------------------

 

 

Processors load
---------------------------------------------------------------------------------
|CPU#|User Time(%)|System Time(%)|Idle Time(%)|Usage(%)|Run queue|Interrupts/sec|
---------------------------------------------------------------------------------
| 1| 0| 83| 16| 84| ?| 2235|
| 2| 11| 43| 47| 53| ?| 2235|
---------------------------------------------------------------------------------

 

 

Processors load
---------------------------------------------------------------------------------
|CPU#|User Time(%)|System Time(%)|Idle Time(%)|Usage(%)|Run queue|Interrupts/sec|
---------------------------------------------------------------------------------
| 1| 0| 85| 14| 86| ?| 2254|
| 2| 1| 37| 63| 37| ?| 2254|
---------------------------------------------------------------------------------

 

 

Processors load
---------------------------------------------------------------------------------
|CPU#|User Time(%)|System Time(%)|Idle Time(%)|Usage(%)|Run queue|Interrupts/sec|
---------------------------------------------------------------------------------
| 1| 0| 85| 14| 86| ?| 2254|
| 2| 1| 37| 63| 37| ?| 2254|
---------------------------------------------------------------------------------

Cheers.

0 Kudos

Re: Low throughput from 4200 appliance

You are getting frame loss (RX-DRP) rates of between 0.3% and 1.6% on your interfaces due to buffering misses which is probably the main thing slowing you down.  This is almost certainly due to high CPU load on your 2 cores, given the large number blades you have enabled on an old 2-core box like that 4200, 100Mbps top throughput doesn't seem that unreasonable to me.  Currently you have a 2/2 CoreXL split on your box, in some cases disabling CoreXL and going to a 1/1 split helps on a 2-core box but given your high PXL% I don't think doing that will help in this case.

The 4200 only has 4GB of RAM which may not be enough for all you are trying to do.  Please provide output of the free -m command to see if a memory upgrade will help.

You can probably pick up some more speed by tuning your policies, the two major areas in your case are Threat Prevention and APCL/URLF. In order to figure out where to focus your efforts, try this and report back what you see:

1) Run Internet speed test and note throughput

2) On the gateway from expert mode run commands ips off and fw amw unload

3) Wait 60 seconds

4) From a completely new browser instance run an Internet speed test and note throughput.  If throughput has substantially increased you need to tune your IPS & Threat Prevention configuration.

5) Run commands ips on and fw amw fetch local

6) Wait 60 seconds

7) From a completely new browser instance run an Internet speed test and note throughput. (should be about the same as #1)

😎 On gateway object in SmartConsole, uncheck the APCL and URLF blades and reinstall policy to the gateway.

9) Wait 60 seconds

9) From a completely new browser instance run an Internet speed test and note throughput. If throughput has substantially increased you need to tune your APCL/URLF policy, typically this will involve removing the "Any Any Any Accept" rule at the bottom of your APCL/URLF policy (which is not necessary except for logging purposes), and making sure you are using object "Internet" in the Destination column of all APCL/URLF rules and NOT "Any".

10) Recheck the APCL and URLF checkboxes and reinstall policy to the gateway.

11) From a completely new browser instance run an Internet speed test and note throughput. (should be about the same as #1)

Let us know what you find out.

 

"IPS Immersion Training" Self-paced Video Class
Now Available at http://www.maxpowerfirewalls.com
0 Kudos
exciteman
Ivory

Re: Low throughput from 4200 appliance

Thanks for the input!

I’ve done the steps you suggested, and I found this:

free -m command:

 

total

used

free

shared

buffers

cached

Mem:

3973

3289

684

0

34

834

-/+ buffers/cache:

 

2420

1553

 

 

 

Swap:

10268

0

10268

 

 

 

 

Throughput tests (peaks - CPview):

Without any changes: 74 Mbps

ips off & fw amw unload: 234 Mbps

Reverted (ips on & fw amw fetch local) 93 Mbps

APCL & URLF blades disabled: 115 Mbps

Reverted (APCL & URLF enabled) 95 Mbps

 

So it seems like the IPS & Threat Prevention needs tuning. Do you have any suggestions for that?

I will do your suggested tuning for APCL/URLF also.

0 Kudos

Re: Low throughput from 4200 appliance

Looks like your box is not hitting swap at all which is good, no memory upgrade needed.

We'll need to do a few more tests to determine whether it is IPS specifically (more likely) or the rest of Threat Prevention (less likely) that is causing the bulk of the slowdown:

1) Run Internet speed test and note throughput

2) On the gateway from expert mode run commands ips off

3) Wait 60 seconds

4) From a completely new browser instance run an Internet speed test and note throughput.  If throughput has substantially increased you need to tune your IPS configuration.

5) Run command ips on

6) Run command fw amw unload

7) Wait 60 seconds

😎 From a completely new browser instance run an Internet speed test and note throughput.  If throughput has substantially increased you need to tune your TP (AV/ABOT) configuration.

9) Run command fw amw fetch local

You may well see a performance improvement at both steps #4 & #8, I'd suggest focusing on where you get the biggest increase for tuning.  If turning off IPS provides most of the gain, determine which IPS profile is in use by your 4200 gateway and open it for editing.  Sort the IPS protections by the "Performance Impact" rating and disable all IPS Protections with a "Critical" or "High" rating.  That should help a lot.

If turning off Threat Prevention (amw) provided most of the gain, my guess is that Anti-virus is causing most of the overhead as Anti-bot tends to be pretty low impact.  I'll need to see the AV & ABOT settings in the relevant TP profile applied to your gateway to make specific recommendations.

 

"IPS Immersion Training" Self-paced Video Class
Now Available at http://www.maxpowerfirewalls.com
0 Kudos