AnsweredAssumed Answered

Latency on Links

Question asked by Terry Tish on Jan 16, 2019
Latest reply on Jan 24, 2019 by Terry Tish

Hello,

 

We have been experiencing latency on our Bonded links. It has had a major impact on product and caused us to move our bonded gateway back to our core switch. I simply am not familiar enough with the different core configurations to make an assessment here. Originally I setup our bond with multi-queuing. Can someone assist? Here is the Super 7 output:

 

[Expert@CSBFW-PROD-A:0]# /usr/bin/s7pac
+-----------------------------------------------------------------------------+
| Super Seven Performance Assessment Commands v0.3 (Thanks to Timothy Hall) |
+-----------------------------------------------------------------------------+
| Inspecting your environment: OK |
| This is a firewall....(continuing) |
| |
| Referred pagenumbers are to be found in the following book: |
| Max Power: Check Point Firewall Performance Optimization - Second Edition |
| Available at http://www.maxpowerfirewalls.com/ |
| |
+-----------------------------------------------------------------------------+
| Command #1: fwaccel stat |
| |
| Check for : Accelerator Status must be enabled (R77.xx/R80.10 versions) |
| Status must be enabled (R80.20) |
| Accept Templates must be enabled |
| Message "disabled" from (low rule number) = bad |
| |
| Chapter 9: SecureXL throughput acceleration |
| Page 278 |
+-----------------------------------------------------------------------------+
| Output: |
Accelerator Status : on
Accept Templates : enabled
Drop Templates : disabled
NAT Templates : disabled by user

Accelerator Features : Accounting, NAT, Cryptography, Routing,
HasClock, Templates, Synchronous, IdleDetection,
Sequencing, TcpStateDetect, AutoExpire,
DelayedNotif, TcpStateDetectV2, CPLS, McastRouting,
WireMode, DropTemplates, NatTemplates,
Streaming, MultiFW, AntiSpoofing, Nac,
ViolationStats, AsychronicNotif, ERDOS,
NAT64, GTPAcceleration, SCTPAcceleration,
McastRoutingV2
Cryptography Features : Tunnel, UDPEncapsulation, MD5, SHA1, NULL,
3DES, DES, CAST, CAST-40, AES-128, AES-256,
ESP, LinkSelection, DynamicVPN, NatTraversal,
EncRouting, AES-XCBC, SHA256


+-----------------------------------------------------------------------------+
| Command #2: fwaccel stats -s |
| |
| Check for : Accelerated conns/Totals conns: >25% good, >50% great |
| Accelerated pkts/Total pkts : >50% great |
| PXL pkts/Total pkts : >50% OK |
| F2Fed pkts/Total pkts : <30% good, <10% great |
| |
| Chapter 9: SecureXL throughput acceleration |
| Page 287, Packet/Throughput Acceleration: The Three Kernel Paths |
+-----------------------------------------------------------------------------+
| Output: |
Accelerated conns/Total conns : 0/35 (0%)
Accelerated pkts/Total pkts : 0/3983956 (0%)
F2Fed pkts/Total pkts : 3983956/3983956 (100%)
PXL pkts/Total pkts : 0/3983956 (0%)
QXL pkts/Total pkts : 0/3983956 (0%)


+-----------------------------------------------------------------------------+
| Command #3: grep -c ^processor /proc/cpuinfo && /sbin/cpuinfo |
| |
| Check for : If number of cores is roughly double what you are excpecting, |
| hyperthreading may be enabled |
| |
| Chapter 7: CorexL Tuning |
| Page 239 |
+-----------------------------------------------------------------------------+
| Output: |
6
HyperThreading=disabled


+-----------------------------------------------------------------------------+
| Command #4: fw ctl affinity -l -r |
| |
| Check for : SND/IRQ/Dispatcher Cores, # of CPU's allocated to interface(s) |
| Firewall Workers/INSPECT Cores, # of CPU's allocated to fw_x |
| R77.30: Support processes executed on ALL CPU's |
| R80.xx: Support processes only executed on Firewall Worker Cores|
| |
| Chapter 7: CoreXL Tuning |
| Page 221 |
+-----------------------------------------------------------------------------+
| Output: |
CPU 0: eth1-04
CPU 1: Sync Mgmt
CPU 2:
CPU 3:
CPU 4:
CPU 5:
All:


+-----------------------------------------------------------------------------+
| Command #5: netstat -ni |
| |
| Check for : RX/TX errors |
| RX-DRP % should be <0.1% calculated by (RX-DRP/RX-OK)*100 |
| TX-ERR might indicate Fast Ethernet/100Mbps Duplex Mismatch |
| |
| Chapter 2: Layers 1&2 Performance Optimization |
| Page 28-35 |
| |
| Chapter 7: CoreXL Tuning |
| Page 204 |
+-----------------------------------------------------------------------------+
| Output: |
Kernel Interface table
Iface MTU Met RX-OK RX-ERR RX-DRP RX-OVR TX-OK TX-ERR TX-DRP TX-OVR Flg
Mgmt 1500 0 43600162 2 0 0 36013528 0 0 0 BMRU
Sync 1500 0 61606193 0 0 0 283637876 0 0 0 BMRU
eth1-04 1500 0 5426765033 0 495 495 4930747640 0 0 0 BMRU
lo 16436 0 8398237 0 0 0 8398237 0 0 0 LRU

interface eth1-04: There are no RX drops

 

+-----------------------------------------------------------------------------+
| Command #6: fw ctl multik stat |
| |
| Check for : Large # of conns on Worker 0 - IPSec VPN/VoIP? |
| Large imbalance of connections on a single or multiple Workers |
| |
| Chapter 7: CoreXL Tuning |
| Page 241 |
| |
| Chapter 8: CoreXL VPN Optimization |
| Page 256 |
+-----------------------------------------------------------------------------+
| Output: |
fw: CoreXL is disabled


+-----------------------------------------------------------------------------+
| Command #7: cpstat os -f multi_cpu -o 1 -c 5 |
| |
| Check for : High SND/IRQ Core Utilization |
| High Firewall Worker Core Utilization |
| |
| Chapter 7: CoreXL Tuning |
| Page 197 |
+-----------------------------------------------------------------------------+
| Output: |

 

Processors load
---------------------------------------------------------------------------------
|CPU#|User Time(%)|System Time(%)|Idle Time(%)|Usage(%)|Run queue|Interrupts/sec|
---------------------------------------------------------------------------------
| 1| 8| 8| 84| 16| ?| 4|
| 2| 5| 6| 89| 11| ?| 4|
| 3| 4| 5| 91| 9| ?| 4|
| 4| 4| 4| 92| 8| ?| 4|
| 5| 4| 4| 92| 8| ?| 4|
| 6| 4| 4| 93| 7| ?| 4|
---------------------------------------------------------------------------------

 

 

Processors load
---------------------------------------------------------------------------------
|CPU#|User Time(%)|System Time(%)|Idle Time(%)|Usage(%)|Run queue|Interrupts/sec|
---------------------------------------------------------------------------------
| 1| 8| 8| 84| 16| ?| 4|
| 2| 5| 6| 89| 11| ?| 4|
| 3| 4| 5| 91| 9| ?| 4|
| 4| 4| 4| 92| 8| ?| 4|
| 5| 4| 4| 92| 8| ?| 4|
| 6| 4| 4| 93| 7| ?| 4|
---------------------------------------------------------------------------------

 

 

Processors load
---------------------------------------------------------------------------------
|CPU#|User Time(%)|System Time(%)|Idle Time(%)|Usage(%)|Run queue|Interrupts/sec|
---------------------------------------------------------------------------------
| 1| 14| 16| 70| 30| ?| 2726|
| 2| 4| 11| 86| 14| ?| 2726|
| 3| 5| 8| 87| 13| ?| 5452|
| 4| 5| 6| 89| 11| ?| 2726|
| 5| 8| 9| 84| 16| ?| 2726|
| 6| 10| 9| 81| 19| ?| 5452|
---------------------------------------------------------------------------------

 

 

Processors load
---------------------------------------------------------------------------------
|CPU#|User Time(%)|System Time(%)|Idle Time(%)|Usage(%)|Run queue|Interrupts/sec|
---------------------------------------------------------------------------------
| 1| 14| 16| 70| 30| ?| 2726|
| 2| 4| 11| 86| 14| ?| 2726|
| 3| 5| 8| 87| 13| ?| 5452|
| 4| 5| 6| 89| 11| ?| 2726|
| 5| 8| 9| 84| 16| ?| 2726|
| 6| 10| 9| 81| 19| ?| 5452|
---------------------------------------------------------------------------------

 

 

Processors load
---------------------------------------------------------------------------------
|CPU#|User Time(%)|System Time(%)|Idle Time(%)|Usage(%)|Run queue|Interrupts/sec|
---------------------------------------------------------------------------------
| 1| 10| 26| 64| 36| ?| 694|
| 2| 3| 9| 89| 11| ?| 694|
| 3| 5| 8| 87| 13| ?| 694|
| 4| 5| 10| 86| 14| ?| 694|
| 5| 6| 14| 80| 20| ?| 695|
| 6| 7| 13| 80| 20| ?| 697|
---------------------------------------------------------------------------------


+-----------------------------------------------------------------------------+
| Thanks for using s7pac |

Outcomes