Low Speedtest result when behind CheckPoint Firewall

[Theo]

Subscribed Internet Rate: 10Mbps Download, 10Mbps Upload

Clients behind the CheckPoint firewall only achieves max. of 3Mbps download and upload. If I attached a laptop directly to internet line, I got consistent 10M/10M (download/upload). But, if I used the same laptop and connect to Eth port of the CheckPoint I got the same as the clients behind the firewall.

Security Gateway: CheckPoint 1450

OS: Gaia

Version: R77.20.81 (990172541) 

Enabled Blades:

• Firewall
• IPSec VPN
• IPS
• Anti-Bot
• Anti-Virus
• identity Awareness
• Application Control
• URL Filtering
• QoS

[PhoneBoy]

Moving this to SMB and SMP‌ section.

Which speed test site did you use?

How is the PC connecting to the 1450 (specifically what is speed and duplex setting)?

[Theo]

I used speedtest.net

The laptop is connected using an ethernet cable (CAT5) to LAN3 of CheckPoint 1450. The Speed is set to Auto Full Duplex.

[Chris_Atkinson]

I would start with reviewing sk125097 for relevance.

Also is the connection PPPoE based?

[Theo]

Thanks. I'm using Static connection.

I couldn't find this "Threat Prevention Anti-Virus policy - Resource classification mode" in my Advance Settings. The suggestion of Pedro Espindola‌ below helps

[Danny]

You've activated almost all Software Blades available. On an SMB 1450 Embedded GAiA appliance (Datasheet) using an ARM926EJ-S processor with 1GB RAM (see 1400 FAQ). Try disabling if disabling all blades but FW speeds up your performance. Then investigate which blades requires most of your performance.

[Casey_Engel]

I would start with Anti-Bot and Anti-Virus. Also take a look at #top when running the test

[Pedro_Espindola]

What happens if you disable QoS?

[Theo]

Pedro Espindola wrote:

What happens if you disable QoS?

This works!

[Pedro_Espindola]

It seems QoS limits speed of a single connection to 25 - 30% of the link speed for whatever reason, even if it is the only connection.

What were your QoS configs?

I have to do some testing, but it seems there are some hidden default QoS rules for traffic shaping besides the ones we can configure.

[Theo]

I'm still doing the troubleshooting, the download returned to 3M while upload seems fine at 8 or 9M.

My QoS only have Default as I temporary deleted the created previously (for test purposes)

Now, download drops again to max. 3M

[John_Pinegar]

I have a bunch of 1450 appliances deployed and have run into this issue in the past. The first thing I would do is set the speed and duplex settings manually (Device >> Internet >> Edit >> Advanced >> Port Settings). Let me know if that solves your problem. Thanks.

[PhoneBoy]

I was thinking of this on the client side, but yeah, also good to check this on the WAN link as well.

[Theo]

I did but got the same results. Thanks for the tip

[Theo]

What a strange behavior, I started to disabled Anti-Bot, Antivirus, QoS and noticed that the speed went up for a couple of minutes but was also revert to maximum of 3Mbps download and upload. 

[Timothy_Hall]

Please provide output of netstat -ni, and ethtool (interfacename) for all interfaces, it sounds like your network is not running cleanly.

--

CheckMates Break Out Sessions Speaker

CPX 2019 Las Vegas & Vienna - Tuesday@13:30

[HristoGrigorov]

netstat -ni is unfortunately not available on SMB. ethtool is there but won't give any useful info.

I wonder what build is Darius running? 

Also does that happen if you download something from the appliance itself ? I mean login with ssh and use wget or curl.

What IPS profile is in use ?

fwaccel stats -s may be and also have you tried to connect ONLY your laptop behind the appliance and directly to it not through some switch ?

[Timothy_Hall]

How about ifconfig (interfacename) or ifconfig -a?  That should provide enough detail...

--

CheckMates Break Out Sessions Speaker

CPX 2019 Las Vegas & Vienna - Tuesday@13:30

[HristoGrigorov]

Yeah, ifconfig -a would be nice to see. May want to replace your public IP with something else for privacy.

[Theo]

Hi Timothy Hall and Hristo Grigorov thanks for the comments guys

I update the version in my original post it's R77.20.81 (990172541)

I copied only important ports, LAN1 and LAN2 uses to connect Switch. Btw, I changed the MAC and IP into xxx for privacy indeed.

WAN       Link encap:Ethernet  HWaddr xx:xx:xx:xx:xx:xx 

          inet addr:xxx.xxx.xxx.xxx  Bcast:xxx.xxx.xxx.xxx  Mask:xxx.xxx.xxx.xxx

          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

          RX packets:29392574 errors:1439911 dropped:755 overruns:0 frame:1557862

          TX packets:25045004 errors:0 dropped:0 overruns:0 carrier:0

          collisions:0 txqueuelen:1000

          RX bytes:4292707451 (3.9 GiB)  TX bytes:2376566533 (2.2 GiB)

eth1      Link encap:Ethernet  HWaddr xx:xx:xx:xx:xx:xx

          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

          RX packets:28961827 errors:0 dropped:1 overruns:0 frame:0

          TX packets:29090006 errors:0 dropped:0 overruns:0 carrier:0

          collisions:0 txqueuelen:1000

          RX bytes:3400537063 (3.1 GiB)  TX bytes:4190357047 (3.9 GiB)

LAN1      Link encap:Ethernet  HWaddr xx:xx:xx:xx:xx:xx

          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

          RX packets:24087372 errors:0 dropped:1 overruns:0 frame:0

          TX packets:27842428 errors:0 dropped:0 overruns:0 carrier:0

          collisions:0 txqueuelen:0

          RX bytes:1511988096 (1.4 GiB)  TX bytes:3947024923 (3.6 GiB)

LAN2      Link encap:Ethernet  HWaddr xx:xx:xx:xx:xx:xx

          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

          RX packets:4866498 errors:0 dropped:6 overruns:0 frame:0

          TX packets:1247572 errors:0 dropped:0 overruns:0 carrier:0

          collisions:0 txqueuelen:0

          RX bytes:1771935171 (1.6 GiB)  TX bytes:533676195 (508.9 MiB)

I disabled QoS yesterday and I noticed the speed jumped now to 9M/9M. I also noticed that there is an entry in Limits for Office365 Applications with 3M/3M and I temporary delete this for test purposes

[HristoGrigorov]

Your WAN interface isn't looking very good. There are errors, drops, etc. How is it connected to Internet ? Going through switch I guess? You should definitely try to force port settings as suggested above. 

Do you have QoS defined on the WAN port itself? You can see that in appliance WebUI. Limits in application policy are not the same as QoS btw.

[Theo]

Hristo thanks!

My WAN port is indeed connected to a Nortel ESU 1800 switch that is managed by the ISP.

So the current setup is Checkpoint WAN <----> Nortel Switch <----> ISP Fiber Optic

• I already disabled Auto negotiation and is set WAN Link Speed to 100M/Full Duplex.
• There was no QoS defined for WAN port

[HristoGrigorov]

Nice. Let us know if issue is resolved.

[Antonio_Ballest]

Hi Darius,

As far as I know, if you fix speed and duplex settings in your firewall it must be fixed in the Nortel switch too, otherwise you will get lots of errors in your link, as your NIC will not negotiate this parameters with the other end. Contact your ISP and ask them to tell you the right settings.

By the way, have you tried changing the patch cable between your firewall and the Nortel switch?

[Timothy_Hall]

Make a note of the RX errors/overruns/frame counters for interface WAN and see if they are still incrementing after running a few speed tests.  You definitely had a WAN duplex mismatch at one point, if you hardcode the 1400 to 100/full it will not autonegotiate with the Nortel at all, and the Nortel will assume that it is attached to a hub and go 100/half.  Thankfully this annoying behavior was fixed in the GigE spec.  If you want to hardcode you need to make sure that both ends are hardcoded to 100/full and that the error counters are not moving at all during speed tests first, before you try to do anything else. 

Enabling QoS is generally not recommended unless you need to do bandwidth guarantees, LLQ, or DiffServ, as it causes a big dent in firewall performance as you have observed especially on the smaller boxes.  If you need to enforce bandwidth limits (the most common use of QoS in most environments) just do a Limit action directly in your APCL/URLF policy rules.

--

CheckMates Break Out Sessions Speaker

CPX 2019 Las Vegas & Vienna - Tuesday@13:30

[JD_Tomzak]

Also, you might try bumping up the receive buffers on the WAN interface. (if they still increment up after all this)

show interface WAN rx-ringsize 

Set interface WAN rx-ringsize 1024

save config

[Balamurugan_M]

Hi,

Have you verified tracker logs for any restricted traffic due to some policy?

Once I had a similar issue due to IPS policy. Kindly do review ur policy settings.

Thanks.

[Perkin_Foo]

after go thought all thread, it seems 2 issues pointed out
1) WAN packet loss, not sure if that just a matter of layer 1 issue? replace cable / port help?

2) Danny pointed out you have enable most of the blade, have you try to disable the blade to test the speed first? to me, it seems you turn on most the functions on an entry level FW.  for me the actual bandwidth will be the 30% the datasheet claimed. 

[Theo]

1. i refer this to ISP and asked to change the cable between the gateway and the switch;

2. I disabled QoS from the list and noticed the speed rised up to closed to ISP rate.

[Gali_Fein]

Hi Darius,

Can you please update what is the status the issue ? 

We'd like to see what is the minimal configuration that still causes low bandwidth, constantly or from time to time. According to your email from Jan 2, 2019 10:20 PM, disabling QoS, Anti-virus and Anti-Bot does not remove the problem completely. What about other blades? Please, try to test disabling APPI and Identity Awareness, one by one and altogether (while the three mentioned above are off)

What is the current QoS blade configuration ?

Is the appliance centrally managed ?

Thanks,

Gali