low throughput on some external ip addresses

[flwsterN]

Hi,

I have one external interface, to this interface our ISP is static routing 8 ipv4 addresses /31 mask.

This is over a gigabit link. We have different servers using these addresses behind NAT. 3 ipv4 addresses is working fine and we are getting gbit connection, however on the other 5 ipv4s im getting 1-2mbit download and 200-500mbit upload.

Lets say the server is using a internal network of 192.168.100.0/24, if i hide NAT this network behind the "working" ipv4 addresses it gets gbit. Otherwise back to the 1-2mbit down and 200-500 upload. This must be a issue with the ISP right? What could cause a problem like this?

[PhoneBoy]

It could be an ISP issue, but we should rule out anything specific with the gateway.
Let’s start with:

• Version/JHF level of gateway
• Output of Super Seven commands when attempting access via one of the “bad” IPs
• Output of enabled_blades commands 

This should help us narrow down the issue.

[flwsterN]

Hi PhoneBoy!

R81.10 Take 87
enabled_blades
fw vpn

superseven script output in the txt file!

[PhoneBoy]

The gateway is barely breaking a sweat, so this isn't a performance related issue.
It may be something in the networking configuration with your ISP like @Timothy_Hall said.

[Timothy_Hall]

I have been in contact with the OP and done some probing from the Shadow Peak lab, and it definitely looks like an upstream ISP issue for his routed /31's.  Helping them assemble an airtight case that the ISP will be unable to dismiss as not their problem.  🙂

[Timothy_Hall · Accepted solution]

With the blades you have enabled, nothing in your Check Point firewall policy/feature config should be causing the performance effect you are seeing.

Sounds like the subnet mask on your firewall's external interface is not matching what your ISP has for you on their router, a /31 (255.255.255.254) is 2 total addresses not 8.  If you are set for a /31 and your ISP is set for /29 (255.255.255.248), the "good/fast" addresses probably fall within your /31 while the slow ones fall outside that.  In that case the ISP router may be proxy ARPing for every address on the Internet for your slow addresses which will cause some problems.  You may also be stepping on the addresses assigned as the network number (old broadcast - lowest IP address in the range) for your subnet, as well as the broadcast (highest IP address) which may cause a variety of nasty broadcast storm-type effects that impact performance.

Depending on your ISP they may have given you two Internet-routable netblocks: a small WAN/transit netblock (like a /30) that should be implemented between your firewall's external interface and their router, and another larger LAN netblock (like a /29 or /28) that will be routed by the ISP across the WAN netblock for transit to your firewall.  Feel free to PM me the information your ISP gave you as far as Internet-routable addresses they assigned you, and the external interface configuration of your firewall with no redaction.  I wouldn't recommend posting that info publicly.

[flwsterN]

Hi Timothy!

Thanks for the reply.

I sent you a message!

[flwsterN]

Thank you for all the help Timothy!

[the_rock]

I agree with the guys. Based on fact that ONLY vpn blade is on (since fw is always on by default), looking at your superseven output, there is literally no load on your gateway at all, so its highly unlikely its CP issue.

[the_rock]

Forgot to mention, sorry...it might be worth sending us below outputs if you can. Say your external interface is eth1 (just as an example), could you send us below?

ethtool -i eth1

ethtool -S eth1

clish -c "show interface eth1"

ifconfig -a eth1

cpstat os -f all

Cheers,

Andy

[flwsterN]

I sent you a PM! with some tcpdump on video.

[the_rock]

Let me check : - )

Will respond there.

Andy

[the_rock]

Hey guys,

Just to update, Benjamin and I had remote session and we are pretty confident that this ISP issue based on the testing.

-sxl on or off, same problem

-rule is there to alow full communication, same problem

-ICMP enabled globally, no change

-fw up_execute shows traffic allowed for IPs we tested

-fw ctl zdebug does not drop anything on affected IP addresses

-fw monitor -F we ran, shows traffic for icmp requests, but not constant replies

-verified routing, its very basic and definitely no assymetric routing issues

Based on all above, I am positive this is not the CP fw issue.

Andy

[flwsterN]

Thank you for all the help Andy.

[the_rock]

No problem. Here comes my corny joke of the century that everyone on this planet is sick of...for you, no charge, except iphone charge ; - )