Traffic sync between 2 AIX servers is slow when handled by checkpoint 9400

[MarcuzShinz]

Dear Guy,

Have a nice day!

We are currently deploying Check Point 9400 and encountering issues with the synchronization traffic of AIX servers. These servers synchronize using the ssh_v2 service. Let me describe the previous situation:

1. Previously, the customer used Check Point 4800 with only the Firewall Blade enabled. They had two systems in DC and DR, each containing AIX servers that synchronized data daily. When this traffic was processed by Check Point 4800, it reached approximately 200Mb/s.
2. After replacing Check Point 4800 with Check Point 9400, also with only the Firewall Blade enabled, the synchronization traffic between AIX servers across the DC and DR sites dropped to 20~30Mb/s.

Has anyone encountered this issue before? please see the image attached

[PhoneBoy]

What was the version running on the 4800 versus what is running on the 9400?
Sounds like the traffic might be hitting F2F path for some reason.
You might try to use fast_accel to ensure the flow is accelerated: https://support.checkpoint.com/results/sk/sk156672

[MarcuzShinz]

Dear PhoneBoy,

Version on cp4800 is R77.30, on cp9400 is R81.20.

I have checked on cp4800 with command fwaccel stat, securexl stopped the first rule.

[G_W_Albrecht]

Open an SR# with CP TAC to get this resolved asap.

[MarcuzShinz]

Sure, I opened case with TAC but they response very slow, two week but they cannot resolved the issue.

[JaAnd]

I fully agree that Fast_Accel rules might be the right way to go, as I have successfully used it to boost performance in backups, and also in some VoIP areas. As always use with care, as it makes affected traffic processed faster, but not without a cost - bypassing most of the wonderfulness of Check Point's security oriented code 😉

I am also waiting if the same will be ever possible with Maestro FF, as currently it only supports traffic that is not traversing local networks. Who knows, what feature GAIA releases will bring to us.

[Chris_Atkinson]

When SecureXL works in the User Mode (UPPAK), the Security Gateway performance for the Slow Path traffic (F2F) is lower compared to the Kernel Mode (KPPAK).

Suggest ensuring the policy is constructed / optimized in a manner to avoid scenarios impacting SecureXL.

[Timothy_Hall]

We need to know what path your replication connection is operating in.  Start a replication connection and make sure it is alive by running fw ctl multik gconn.  If you can't see the connection here it is not alive.

Next run the following while the connection is alive:

fw tab -t connections -z

fwaccel conns

netstat -ni

If you see the replication connection in the output of the first command, the traffic is slowpath and a reason will be provided.  fast_accel will not work for this traffic.

If you can't find the connection in the output of the first command it will be shown by the second one which means medium or fastpath. 

Please post the output of wherever you find the connection, along with the third command.  If fwaccel stat is complaining about stopping on rule #1 it is referring to accept templating which has nothing do with which path the connection ends up in.  With just the firewall blade enabled it should be fastpath but Core Activations & Inspection Settings can interfere with this.

[MarcuzShinz]

Since we have clear source and destination, how can we check with these commands.

[MarcuzShinz]

I got cpinfo on both devices while processing traffic, about securexl it shows as below

on cp-4800

on cp-9400

[Timothy_Hall]

Nothing is wrong with SecureXL on your new box (it is still in KPPAK mode for some reason), please provide the outputs I requested in my prior message.

[MarcuzShinz]

Dear Legend,

I provide about image output of command your share. Vlan of server DR iss vlan 160 and server is 172.29.4.3. Server on DC is 172.27.4.10 & 172.27.4.11

[Timothy_Hall]

Based on your outputs the problematic traffic is already in the fastpath, assuming you did not already explicitly force it there with a fast_accel rule.  Your network interfaces are running clean.  I don't see how the firewall can be the cause of your performance issues for this traffic, however I don't understand why KPPAK mode is still being utilized over UPPAK mode for your Quantum Force appliance unless you are not running the latest recommended Jumbo HFA for R81.20.  Are you?  Did you set KPPAK mode explicitly?

If you are running the latest recommended Jumbo HFA for R81.20, TAC will have to look at this...but my gut instinct is that the firewall is not responsible for the slow performance.

[Lesley]

Share this output and:

cpinfo -y all (to see if you have a Jumbo installed and if so a new one)

enabled_blades double check that only fw is enabled

ethtool -g INTERFACE (to checkt he rx buffers) this is follow-up question depending on nestat -ni output