Create a Post
cancel
Showing results for 
Search instead for 
Did you mean: 
Thecoder
Collaborator

SecureXL is enabled, but the traffic is not accelerated

SecureXL is enabled, but the traffic is not accelerated. i restarted securexl.but the result not changed. corexl and ht are active. and we have performance problems.

[Expert@fw1:0]# fwaccel stats -s

Accelerated conns/Total conns : 519/114126 (0%)

Accelerated pkts/Total pkts   : 1306379/17442674 (7%)

F2Fed pkts/Total pkts   : 11676500/17442674 (66%)

PXL pkts/Total pkts   : 4459795/17442674 (25%)

QXL pkts/Total pkts   : 0/17442674 (0%)

[Expert@fw1:0]# fwaccel stats -p

F2F packets:

--------------

Violation             Packets            Violation             Packets       

--------------------  ---------------    --------------------  ---------------

pkt is a fragment                 389    pkt has IP options                225

ICMP miss conn                  80304    TCP-SYN miss conn             5282924

TCP-other miss conn           7127805    UDP miss conn                 1435788

other miss conn                  5652    VPN returned F2F                   15

ICMP conn is F2Fed              24867    TCP conn is F2Fed           167500203

UDP conn is F2Fed               29421    other conn is F2Fed                 0

uni-directional viol                0    possible spoof viol                 0

TCP state viol                 103953    out if not def/accl              5518

bridge, src=dst                     0    routing decision err               87

sanity checks failed                0    temp conn expired                   3

fwd to non-pivot                    0    broadcast/multicast                 0

cluster message                     0    partial conn                    92673

PXL returned F2F               192634    cluster forward                     0

chain forwarding                    0    Tmpl no-match range                 0

Tmpl no-match time                  0    general reason                    306

route change                        0    inbound zone change                 0

outbound zone change                0   

24 Replies
Timothy_Hall
Champion
Champion

Please post the output of the following commands for further analysis:

fw ver

free -m
netstat -ni
enabled_blades
fwaccel stat
fw ctl multik stat
fw ctl affinity -l -r
fw ctl multik get_mode (R77.30) or fw ctl multik dynamic_dispatching get_mode (R80.10+)
cpstat os -f multi_cpu -o 1
cpconfig (the menu displayed by this command)

--
Second Edition of my "Max Power" Firewall Book
Now Available at http://www.maxpowerfirewalls.com

Gateway Performance Optimization R81.20 Course
now available at maxpowerfirewalls.com
Joshua_Hatter
Employee
Employee

Also might be useful to have #fwaccel conns

Thecoder
Collaborator

fw ver
This is Check Point's software version R80.10 - Build 056
[Expert@fw1:0]# free -m
total used free shared buffers cached
Mem: 64207 35142 29064 0 903 12673
-/+ buffers/cache: 21565 42641
Swap: 32765 0 32765
[Expert@fw1:0]# netstat -ni
Kernel Interface table
Iface MTU Met RX-OK RX-ERR RX-DRP RX-OVR TX-OK TX-ERR TX-DRP TX-OVR Flg
Mgmt 1500 0 118199136 0 0 0 1672154958 0 0 0 BMRU
bond100 1500 0 125776735 0 0 0 1903103776 0 0 0 BMmRU
eth1-01 1500 0 6990601996 0 0 0 7257672547 0 0 0 BMRU
eth1-02 1500 0 2708488932 0 0 0 2616334278 0 0 0 BMRU
eth2-01 1500 0 5308103110 0 127382 127382 5964648490 0 0 0 BMRU
eth2-02 1500 0 4477011 0 0 0 4658400 0 0 0 BMRU
eth3-01 1500 0 0 0 0 0 0 0 0 0 BMU
eth3-03 1500 0 303419 0 0 0 299866 0 0 0 BMsRU
eth3-04 1500 0 125473320 0 0 0 1902803949 0 0 0 BMsRU
lo 16436 0 17779742 0 0 0 17779742 0 0 0 LRU
[Expert@fw1:0]# enabled_blades
fw vpn cvpn urlf av appi ips identityServer SSL_INSPECT anti_bot ThreatEmulation mon vpn
[Expert@fw1:0]# fwaccel stat
Accelerator Status : on
Accept Templates : enabled
Drop Templates : enabled
NAT Templates : enabled
NMR Templates : enabled
NMT Templates : enabled

Accelerator Features : Accounting, NAT, Cryptography, Routing,
HasClock, Templates, Synchronous, IdleDetection,
Sequencing, TcpStateDetect, AutoExpire,
DelayedNotif, TcpStateDetectV2, CPLS, McastRouting,
WireMode, DropTemplates, NatTemplates,
Streaming, MultiFW, AntiSpoofing, Nac,
ViolationStats, AsychronicNotif, ERDOS,
McastRoutingV2, NMR, NMT, NAT64, GTPAcceleration,
SCTPAcceleration
Cryptography Features : Tunnel, UDPEncapsulation, MD5, SHA1, NULL,
3DES, DES, CAST, CAST-40, AES-128, AES-256,
ESP, LinkSelection, DynamicVPN, NatTraversal,
EncRouting, AES-XCBC, SHA256

cpconfig
This program will let you re-configure
your Check Point products configuration.


Configuration Options:
----------------------
(1) Licenses and contracts
(2) SNMP Extension
(3) PKCS#11 Token
(4) Random Pool
(5) Secure Internal Communication
(6) Disable cluster membership for this gateway
(7) Enable Check Point Per Virtual System State
(8) Enable Check Point ClusterXL for Bridge Active/Standby
(9) Hyper-Threading
(10) Disable Check Point SecureXL
(11) Check Point CoreXL
(12) Automatic start of Check Point Products

(13) Exit

cpstat os -f multi_cpu -o 1

Processors load
---------------------------------------------------------------------------------
|CPU#|User Time(%)|System Time(%)|Idle Time(%)|Usage(%)|Run queue|Interrupts/sec|
---------------------------------------------------------------------------------
| 1| 1| 13| 86| 14| ?| 74156|
| 2| 0| 16| 84| 16| ?| 74164|
| 3| 25| 49| 26| 74| ?| 74169|
| 4| 20| 50| 30| 70| ?| 74175|
| 5| 19| 51| 30| 70| ?| 74180|
| 6| 21| 50| 29| 71| ?| 74187|
| 7| 27| 42| 31| 69| ?| 74194|
| 8| 15| 60| 25| 75| ?| 74199|
| 9| 16| 56| 28| 72| ?| 74204|
| 10| 22| 44| 34| 66| ?| 74210|
| 11| 23| 45| 32| 68| ?| 74216|
| 12| 25| 37| 38| 62| ?| 74222|
| 13| 27| 37| 36| 64| ?| 74226|
| 14| 22| 47| 31| 69| ?| 74232|
| 15| 19| 48| 33| 67| ?| 74240|
| 16| 24| 41| 35| 65| ?| 74246|
| 17| 5| 5| 90| 10| ?| 74252|
| 18| 16| 11| 74| 26| ?| 74255|
| 19| 22| 49| 29| 71| ?| 74261|
| 20| 20| 52| 27| 73| ?| 74266|
| 21| 14| 64| 22| 78| ?| 74272|
| 22| 23| 46| 31| 69| ?| 74276|
| 23| 24| 45| 32| 68| ?| 74281|
| 24| 23| 48| 30| 70| ?| 74287|
| 25| 18| 55| 27| 73| ?| 74292|
| 26| 22| 45| 33| 67| ?| 74299|
| 27| 23| 43| 33| 67| ?| 74306|
| 28| 26| 40| 34| 66| ?| 74313|
| 29| 24| 46| 30| 70| ?| 74321|
| 30| 22| 45| 33| 67| ?| 74325|
| 31| 25| 40| 35| 65| ?| 74331|
| 32| 21| 46| 33| 67| ?| 74335|
---------------------------------------------------------------------------------

Processors load
---------------------------------------------------------------------------------
|CPU#|User Time(%)|System Time(%)|Idle Time(%)|Usage(%)|Run queue|Interrupts/sec|
---------------------------------------------------------------------------------
| 1| 1| 13| 86| 14| ?| 74156|
| 2| 0| 16| 84| 16| ?| 74164|
| 3| 25| 49| 26| 74| ?| 74169|
| 4| 20| 50| 30| 70| ?| 74175|
| 5| 19| 51| 30| 70| ?| 74180|
| 6| 21| 50| 29| 71| ?| 74187|
| 7| 27| 42| 31| 69| ?| 74194|
| 8| 15| 60| 25| 75| ?| 74199|
| 9| 16| 56| 28| 72| ?| 74204|
| 10| 22| 44| 34| 66| ?| 74210|
| 11| 23| 45| 32| 68| ?| 74216|
| 12| 25| 37| 38| 62| ?| 74222|
| 13| 27| 37| 36| 64| ?| 74226|
| 14| 22| 47| 31| 69| ?| 74232|
| 15| 19| 48| 33| 67| ?| 74240|
| 16| 24| 41| 35| 65| ?| 74246|
| 17| 5| 5| 90| 10| ?| 74252|
| 18| 16| 11| 74| 26| ?| 74255|
| 19| 22| 49| 29| 71| ?| 74261|
| 20| 20| 52| 27| 73| ?| 74266|
| 21| 14| 64| 22| 78| ?| 74272|
| 22| 23| 46| 31| 69| ?| 74276|
| 23| 24| 45| 32| 68| ?| 74281|
| 24| 23| 48| 30| 70| ?| 74287|
| 25| 18| 55| 27| 73| ?| 74292|
| 26| 22| 45| 33| 67| ?| 74299|
| 27| 23| 43| 33| 67| ?| 74306|
| 28| 26| 40| 34| 66| ?| 74313|
| 29| 24| 46| 30| 70| ?| 74321|
| 30| 22| 45| 33| 67| ?| 74325|
| 31| 25| 40| 35| 65| ?| 74331|
| 32| 21| 46| 33| 67| ?| 74335|
---------------------------------------------------------------------------------

Processors load
---------------------------------------------------------------------------------
|CPU#|User Time(%)|System Time(%)|Idle Time(%)|Usage(%)|Run queue|Interrupts/sec|
---------------------------------------------------------------------------------
| 1| 0| 13| 87| 13| ?| 54603|
| 2| 0| 15| 85| 15| ?| 54608|
| 3| 20| 52| 27| 73| ?| 54612|
| 4| 25| 49| 26| 74| ?| 54617|
| 5| 23| 47| 30| 70| ?| 54620|
| 6| 23| 46| 31| 69| ?| 54624|
| 7| 22| 51| 26| 74| ?| 54629|
| 8| 26| 44| 30| 70| ?| 54635|
| 9| 25| 46| 30| 70| ?| 54641|
| 10| 17| 56| 27| 73| ?| 54647|
| 11| 26| 43| 32| 68| ?| 54654|
| 12| 27| 41| 32| 68| ?| 54660|
| 13| 27| 45| 29| 71| ?| 54664|
| 14| 23| 50| 27| 73| ?| 54669|
| 15| 19| 53| 28| 72| ?| 54673|
| 16| 15| 63| 21| 79| ?| 54679|
| 17| 0| 2| 98| 2| ?| 54684|
| 18| 0| 3| 97| 3| ?| 54690|
| 19| 18| 55| 27| 73| ?| 54694|
| 20| 23| 48| 29| 71| ?| 54700|
| 21| 12| 67| 20| 80| ?| 54706|
| 22| 31| 32| 37| 63| ?| 54710|
| 23| 27| 37| 36| 64| ?| 54714|
| 24| 21| 51| 28| 72| ?| 54719|
| 25| 18| 54| 27| 73| ?| 54724|
| 26| 22| 52| 26| 74| ?| 54730|
| 27| 27| 41| 32| 68| ?| 54737|
| 28| 23| 47| 30| 70| ?| 54742|
| 29| 30| 40| 30| 70| ?| 54744|
| 30| 25| 43| 32| 68| ?| 54748|
| 31| 25| 47| 28| 72| ?| 54752|
| 32| 22| 50| 29| 71| ?| 54756|
---------------------------------------------------------------------------------

Processors load
---------------------------------------------------------------------------------
|CPU#|User Time(%)|System Time(%)|Idle Time(%)|Usage(%)|Run queue|Interrupts/sec|
---------------------------------------------------------------------------------
| 1| 0| 13| 87| 13| ?| 54603|
| 2| 0| 15| 85| 15| ?| 54608|
| 3| 20| 52| 27| 73| ?| 54612|
| 4| 25| 49| 26| 74| ?| 54617|
| 5| 23| 47| 30| 70| ?| 54620|
| 6| 23| 46| 31| 69| ?| 54624|
| 7| 22| 51| 26| 74| ?| 54629|
| 8| 26| 44| 30| 70| ?| 54635|
| 9| 25| 46| 30| 70| ?| 54641|
| 10| 17| 56| 27| 73| ?| 54647|
| 11| 26| 43| 32| 68| ?| 54654|
| 12| 27| 41| 32| 68| ?| 54660|
| 13| 27| 45| 29| 71| ?| 54664|
| 14| 23| 50| 27| 73| ?| 54669|
| 15| 19| 53| 28| 72| ?| 54673|
| 16| 15| 63| 21| 79| ?| 54679|
| 17| 0| 2| 98| 2| ?| 54684|
| 18| 0| 3| 97| 3| ?| 54690|
| 19| 18| 55| 27| 73| ?| 54694|
| 20| 23| 48| 29| 71| ?| 54700|
| 21| 12| 67| 20| 80| ?| 54706|
| 22| 31| 32| 37| 63| ?| 54710|
| 23| 27| 37| 36| 64| ?| 54714|
| 24| 21| 51| 28| 72| ?| 54719|
| 25| 18| 54| 27| 73| ?| 54724|
| 26| 22| 52| 26| 74| ?| 54730|
| 27| 27| 41| 32| 68| ?| 54737|
| 28| 23| 47| 30| 70| ?| 54742|
| 29| 30| 40| 30| 70| ?| 54744|
| 30| 25| 43| 32| 68| ?| 54748|
| 31| 25| 47| 28| 72| ?| 54752|
| 32| 22| 50| 29| 71| ?| 54756|
---------------------------------------------------------------------------------

Processors load
---------------------------------------------------------------------------------
|CPU#|User Time(%)|System Time(%)|Idle Time(%)|Usage(%)|Run queue|Interrupts/sec|
---------------------------------------------------------------------------------
| 1| 0| 13| 87| 13| ?| 25536|
| 2| 0| 14| 86| 14| ?| 25541|
| 3| 21| 49| 29| 71| ?| 25545|
| 4| 21| 46| 33| 67| ?| 25549|
| 5| 25| 41| 35| 65| ?| 25551|
| 6| 23| 49| 28| 72| ?| 25554|
| 7| 24| 44| 32| 68| ?| 25559|
| 8| 21| 51| 28| 72| ?| 25564|
| 9| 18| 52| 30| 70| ?| 25569|
| 10| 25| 43| 32| 68| ?| 25576|
| 11| 25| 40| 36| 64| ?| 25580|
| 12| 24| 43| 33| 67| ?| 25587|
| 13| 20| 45| 35| 65| ?| 25591|
| 14| 19| 53| 27| 73| ?| 25599|
| 15| 17| 52| 32| 68| ?| 25603|
| 16| 24| 39| 37| 63| ?| 25609|
| 17| 0| 2| 98| 2| ?| 25616|
| 18| 0| 1| 99| 1| ?| 25623|
| 19| 24| 44| 32| 68| ?| 25631|
| 20| 20| 50| 30| 70| ?| 25639|
| 21| 22| 51| 27| 73| ?| 25646|
| 22| 26| 39| 36| 64| ?| 25652|
| 23| 23| 42| 36| 64| ?| 25657|
| 24| 23| 45| 32| 68| ?| 25662|
| 25| 17| 51| 32| 68| ?| 25667|
| 26| 21| 41| 38| 62| ?| 25673|
| 27| 25| 40| 35| 65| ?| 25679|
| 28| 23| 41| 36| 64| ?| 25686|
| 29| 25| 41| 35| 65| ?| 25692|
| 30| 19| 45| 36| 64| ?| 25700|
| 31| 18| 53| 30| 70| ?| 25706|
| 32| 24| 42| 34| 66| ?| 25710|
---------------------------------------------------------------------------------

Processors load
---------------------------------------------------------------------------------
|CPU#|User Time(%)|System Time(%)|Idle Time(%)|Usage(%)|Run queue|Interrupts/sec|
---------------------------------------------------------------------------------
| 1| 0| 13| 87| 13| ?| 25536|
| 2| 0| 14| 86| 14| ?| 25541|
| 3| 21| 49| 29| 71| ?| 25545|
| 4| 21| 46| 33| 67| ?| 25549|
| 5| 25| 41| 35| 65| ?| 25551|
| 6| 23| 49| 28| 72| ?| 25554|
| 7| 24| 44| 32| 68| ?| 25559|
| 8| 21| 51| 28| 72| ?| 25564|
| 9| 18| 52| 30| 70| ?| 25569|
| 10| 25| 43| 32| 68| ?| 25576|
| 11| 25| 40| 36| 64| ?| 25580|
| 12| 24| 43| 33| 67| ?| 25587|
| 13| 20| 45| 35| 65| ?| 25591|
| 14| 19| 53| 27| 73| ?| 25599|
| 15| 17| 52| 32| 68| ?| 25603|
| 16| 24| 39| 37| 63| ?| 25609|
| 17| 0| 2| 98| 2| ?| 25616|
| 18| 0| 1| 99| 1| ?| 25623|
| 19| 24| 44| 32| 68| ?| 25631|
| 20| 20| 50| 30| 70| ?| 25639|
| 21| 22| 51| 27| 73| ?| 25646|
| 22| 26| 39| 36| 64| ?| 25652|
| 23| 23| 42| 36| 64| ?| 25657|
| 24| 23| 45| 32| 68| ?| 25662|
| 25| 17| 51| 32| 68| ?| 25667|
| 26| 21| 41| 38| 62| ?| 25673|
| 27| 25| 40| 35| 65| ?| 25679|
| 28| 23| 41| 36| 64| ?| 25686|
| 29| 25| 41| 35| 65| ?| 25692|
| 30| 19| 45| 36| 64| ?| 25700|
| 31| 18| 53| 30| 70| ?| 25706|
| 32| 24| 42| 34| 66| ?| 25710|
---------------------------------------------------------------------------------

Processors load
---------------------------------------------------------------------------------
|CPU#|User Time(%)|System Time(%)|Idle Time(%)|Usage(%)|Run queue|Interrupts/sec|
---------------------------------------------------------------------------------
| 1| 9| 20| 71| 29| ?| 57968|
| 2| 5| 15| 80| 20| ?| 57977|
| 3| 26| 47| 27| 73| ?| 57984|
| 4| 29| 41| 31| 69| ?| 57991|
| 5| 13| 65| 22| 78| ?| 57999|
| 6| 28| 44| 29| 71| ?| 58004|
| 7| 20| 55| 25| 75| ?| 58009|
| 8| 19| 54| 28| 72| ?| 58017|
| 9| 20| 51| 29| 71| ?| 58027|
| 10| 19| 50| 30| 70| ?| 58034|
| 11| 22| 44| 35| 65| ?| 58044|
| 12| 23| 39| 39| 61| ?| 58052|
| 13| 21| 48| 32| 68| ?| 58059|
| 14| 25| 42| 33| 67| ?| 58066|
| 15| 17| 56| 27| 73| ?| 58073|
| 16| 20| 52| 27| 73| ?| 58078|
| 17| 1| 5| 95| 5| ?| 58082|
| 18| 0| 7| 93| 7| ?| 58089|
| 19| 24| 48| 28| 72| ?| 58097|
| 20| 23| 49| 28| 72| ?| 58103|
| 21| 19| 49| 32| 68| ?| 58110|
| 22| 28| 41| 31| 69| ?| 58119|
| 23| 26| 43| 31| 69| ?| 58126|
| 24| 25| 44| 32| 68| ?| 58133|
| 25| 22| 48| 31| 69| ?| 58138|
| 26| 20| 51| 28| 72| ?| 58143|
| 27| 24| 43| 34| 66| ?| 58149|
| 28| 21| 47| 32| 68| ?| 58156|
| 29| 24| 42| 35| 65| ?| 58162|
| 30| 22| 47| 31| 69| ?| 58171|
| 31| 17| 52| 30| 70| ?| 58178|
| 32| 19| 49| 33| 67| ?| 58184|
---------------------------------------------------------------------------------

Processors load
---------------------------------------------------------------------------------
|CPU#|User Time(%)|System Time(%)|Idle Time(%)|Usage(%)|Run queue|Interrupts/sec|
---------------------------------------------------------------------------------
| 1| 9| 20| 71| 29| ?| 57968|
| 2| 5| 15| 80| 20| ?| 57977|
| 3| 26| 47| 27| 73| ?| 57984|
| 4| 29| 41| 31| 69| ?| 57991|
| 5| 13| 65| 22| 78| ?| 57999|
| 6| 28| 44| 29| 71| ?| 58004|
| 7| 20| 55| 25| 75| ?| 58009|
| 8| 19| 54| 28| 72| ?| 58017|
| 9| 20| 51| 29| 71| ?| 58027|
| 10| 19| 50| 30| 70| ?| 58034|
| 11| 22| 44| 35| 65| ?| 58044|
| 12| 23| 39| 39| 61| ?| 58052|
| 13| 21| 48| 32| 68| ?| 58059|
| 14| 25| 42| 33| 67| ?| 58066|
| 15| 17| 56| 27| 73| ?| 58073|
| 16| 20| 52| 27| 73| ?| 58078|
| 17| 1| 5| 95| 5| ?| 58082|
| 18| 0| 7| 93| 7| ?| 58089|
| 19| 24| 48| 28| 72| ?| 58097|
| 20| 23| 49| 28| 72| ?| 58103|
| 21| 19| 49| 32| 68| ?| 58110|
| 22| 28| 41| 31| 69| ?| 58119|
| 23| 26| 43| 31| 69| ?| 58126|
| 24| 25| 44| 32| 68| ?| 58133|
| 25| 22| 48| 31| 69| ?| 58138|
| 26| 20| 51| 28| 72| ?| 58143|
| 27| 24| 43| 34| 66| ?| 58149|
| 28| 21| 47| 32| 68| ?| 58156|
| 29| 24| 42| 35| 65| ?| 58162|
| 30| 22| 47| 31| 69| ?| 58171|
| 31| 17| 52| 30| 70| ?| 58178|
| 32| 19| 49| 33| 67| ?| 58184|
---------------------------------------------------------------------------------

Processors load
---------------------------------------------------------------------------------
|CPU#|User Time(%)|System Time(%)|Idle Time(%)|Usage(%)|Run queue|Interrupts/sec|
---------------------------------------------------------------------------------
| 1| 36| 33| 30| 70| ?| 61929|
| 2| 63| 29| 8| 92| ?| 61936|
| 3| 26| 48| 27| 73| ?| 61942|
| 4| 27| 46| 27| 73| ?| 61949|
| 5| 27| 45| 28| 72| ?| 61953|
| 6| 19| 58| 24| 76| ?| 61959|
| 7| 24| 52| 24| 76| ?| 61965|
| 8| 25| 49| 26| 74| ?| 61972|
| 9| 21| 52| 27| 73| ?| 61978|
| 10| 21| 53| 26| 74| ?| 61982|
| 11| 13| 68| 19| 81| ?| 61989|
| 12| 29| 43| 29| 71| ?| 61993|
| 13| 27| 44| 30| 70| ?| 61999|
| 14| 24| 46| 30| 70| ?| 62006|
| 15| 23| 49| 28| 72| ?| 62012|
| 16| 19| 55| 26| 74| ?| 62019|
| 17| 26| 28| 46| 54| ?| 62024|
| 18| 26| 24| 50| 50| ?| 62031|
| 19| 30| 49| 21| 79| ?| 62036|
| 20| 28| 45| 27| 73| ?| 62042|
| 21| 27| 47| 26| 74| ?| 62048|
| 22| 25| 46| 29| 71| ?| 62055|
| 23| 26| 46| 28| 72| ?| 62061|
| 24| 23| 55| 22| 78| ?| 62068|
| 25| 26| 45| 29| 71| ?| 62075|
| 26| 23| 51| 26| 74| ?| 62082|
| 27| 19| 45| 36| 64| ?| 62089|
| 28| 29| 43| 28| 72| ?| 62096|
| 29| 29| 41| 31| 69| ?| 62102|
| 30| 18| 57| 25| 75| ?| 62107|
| 31| 26| 42| 32| 68| ?| 62110|
| 32| 19| 52| 29| 71| ?| 62115|
---------------------------------------------------------------------------------

Processors load
---------------------------------------------------------------------------------
|CPU#|User Time(%)|System Time(%)|Idle Time(%)|Usage(%)|Run queue|Interrupts/sec|
---------------------------------------------------------------------------------
| 1| 36| 33| 30| 70| ?| 61929|
| 2| 63| 29| 8| 92| ?| 61936|
| 3| 26| 48| 27| 73| ?| 61942|
| 4| 27| 46| 27| 73| ?| 61949|
| 5| 27| 45| 28| 72| ?| 61953|
| 6| 19| 58| 24| 76| ?| 61959|
| 7| 24| 52| 24| 76| ?| 61965|
| 8| 25| 49| 26| 74| ?| 61972|
| 9| 21| 52| 27| 73| ?| 61978|
| 10| 21| 53| 26| 74| ?| 61982|
| 11| 13| 68| 19| 81| ?| 61989|
| 12| 29| 43| 29| 71| ?| 61993|
| 13| 27| 44| 30| 70| ?| 61999|
| 14| 24| 46| 30| 70| ?| 62006|
| 15| 23| 49| 28| 72| ?| 62012|
| 16| 19| 55| 26| 74| ?| 62019|
| 17| 26| 28| 46| 54| ?| 62024|
| 18| 26| 24| 50| 50| ?| 62031|
| 19| 30| 49| 21| 79| ?| 62036|
| 20| 28| 45| 27| 73| ?| 62042|
| 21| 27| 47| 26| 74| ?| 62048|
| 22| 25| 46| 29| 71| ?| 62055|
| 23| 26| 46| 28| 72| ?| 62061|
| 24| 23| 55| 22| 78| ?| 62068|
| 25| 26| 45| 29| 71| ?| 62075|
| 26| 23| 51| 26| 74| ?| 62082|
| 27| 19| 45| 36| 64| ?| 62089|
| 28| 29| 43| 28| 72| ?| 62096|
| 29| 29| 41| 31| 69| ?| 62102|
| 30| 18| 57| 25| 75| ?| 62107|
| 31| 26| 42| 32| 68| ?| 62110|
| 32| 19| 52| 29| 71| ?| 62115|
---------------------------------------------------------------------------------

Processors load
---------------------------------------------------------------------------------
|CPU#|User Time(%)|System Time(%)|Idle Time(%)|Usage(%)|Run queue|Interrupts/sec|
---------------------------------------------------------------------------------
| 1| 14| 44| 42| 58| ?| 55755|
| 2| 2| 24| 73| 27| ?| 55761|
| 3| 25| 40| 35| 65| ?| 55765|
| 4| 18| 47| 34| 66| ?| 55770|
| 5| 21| 44| 35| 65| ?| 55774|
| 6| 22| 39| 39| 61| ?| 55779|
| 7| 19| 51| 30| 70| ?| 55785|
| 8| 23| 39| 39| 61| ?| 55789|
| 9| 25| 37| 38| 62| ?| 55791|
| 10| 16| 53| 31| 69| ?| 55797|
| 11| 26| 34| 40| 60| ?| 55803|
| 12| 24| 35| 40| 60| ?| 55808|
| 13| 20| 40| 40| 60| ?| 55813|
| 14| 21| 40| 39| 61| ?| 55818|
| 15| 19| 41| 40| 60| ?| 55823|
| 16| 20| 42| 37| 63| ?| 55830|
| 17| 23| 31| 46| 54| ?| 55835|
| 18| 92| 8| 0| 100| ?| 55839|
| 19| 22| 45| 34| 66| ?| 55844|
| 20| 18| 50| 32| 68| ?| 55851|
| 21| 23| 43| 34| 66| ?| 55854|
| 22| 29| 35| 36| 64| ?| 55858|
| 23| 25| 34| 41| 59| ?| 55861|
| 24| 22| 45| 32| 68| ?| 55866|
| 25| 25| 38| 37| 63| ?| 55870|
| 26| 23| 41| 36| 64| ?| 55874|
| 27| 24| 39| 37| 63| ?| 55879|
| 28| 26| 35| 39| 61| ?| 55884|
| 29| 22| 41| 38| 62| ?| 55889|
| 30| 19| 46| 35| 65| ?| 55892|
| 31| 15| 52| 33| 67| ?| 55898|
| 32| 19| 44| 38| 62| ?| 55902|
---------------------------------------------------------------------------------

Processors load
---------------------------------------------------------------------------------
|CPU#|User Time(%)|System Time(%)|Idle Time(%)|Usage(%)|Run queue|Interrupts/sec|
---------------------------------------------------------------------------------
| 1| 14| 44| 42| 58| ?| 55755|
| 2| 2| 24| 73| 27| ?| 55761|
| 3| 25| 40| 35| 65| ?| 55765|
| 4| 18| 47| 34| 66| ?| 55770|
| 5| 21| 44| 35| 65| ?| 55774|
| 6| 22| 39| 39| 61| ?| 55779|
| 7| 19| 51| 30| 70| ?| 55785|
| 8| 23| 39| 39| 61| ?| 55789|
| 9| 25| 37| 38| 62| ?| 55791|
| 10| 16| 53| 31| 69| ?| 55797|
| 11| 26| 34| 40| 60| ?| 55803|
| 12| 24| 35| 40| 60| ?| 55808|
| 13| 20| 40| 40| 60| ?| 55813|
| 14| 21| 40| 39| 61| ?| 55818|
| 15| 19| 41| 40| 60| ?| 55823|
| 16| 20| 42| 37| 63| ?| 55830|
| 17| 23| 31| 46| 54| ?| 55835|
| 18| 92| 8| 0| 100| ?| 55839|
| 19| 22| 45| 34| 66| ?| 55844|
| 20| 18| 50| 32| 68| ?| 55851|
| 21| 23| 43| 34| 66| ?| 55854|
| 22| 29| 35| 36| 64| ?| 55858|
| 23| 25| 34| 41| 59| ?| 55861|
| 24| 22| 45| 32| 68| ?| 55866|
| 25| 25| 38| 37| 63| ?| 55870|
| 26| 23| 41| 36| 64| ?| 55874|
| 27| 24| 39| 37| 63| ?| 55879|
| 28| 26| 35| 39| 61| ?| 55884|
| 29| 22| 41| 38| 62| ?| 55889|
| 30| 19| 46| 35| 65| ?| 55892|
| 31| 15| 52| 33| 67| ?| 55898|
| 32| 19| 44| 38| 62| ?| 55902|
---------------------------------------------------------------------------------

Processors load
---------------------------------------------------------------------------------
|CPU#|User Time(%)|System Time(%)|Idle Time(%)|Usage(%)|Run queue|Interrupts/sec|
---------------------------------------------------------------------------------
| 1| 25| 41| 34| 66| ?| 58551|
| 2| 13| 24| 64| 36| ?| 58557|
| 3| 22| 39| 40| 60| ?| 58561|
| 4| 15| 46| 39| 61| ?| 58564|
| 5| 19| 39| 42| 58| ?| 58567|
| 6| 12| 52| 37| 63| ?| 58572|
| 7| 17| 48| 35| 65| ?| 58575|
| 8| 18| 45| 37| 63| ?| 58579|
| 9| 26| 38| 36| 64| ?| 58583|
| 10| 20| 50| 29| 71| ?| 58587|
| 11| 26| 39| 35| 65| ?| 58593|
| 12| 26| 38| 36| 64| ?| 58600|
| 13| 26| 37| 37| 63| ?| 58605|
| 14| 25| 36| 40| 60| ?| 58609|
| 15| 21| 47| 32| 68| ?| 58613|
| 16| 24| 38| 38| 62| ?| 58618|
| 17| 29| 31| 40| 60| ?| 58622|
| 18| 91| 9| 0| 100| ?| 58626|
| 19| 20| 42| 38| 62| ?| 58632|
| 20| 14| 53| 34| 66| ?| 58638|
| 21| 19| 41| 41| 59| ?| 58642|
| 22| 16| 49| 36| 64| ?| 58646|
| 23| 23| 34| 44| 56| ?| 58650|
| 24| 20| 38| 42| 58| ?| 58655|
| 25| 27| 39| 34| 66| ?| 58659|
| 26| 21| 44| 35| 65| ?| 58664|
| 27| 24| 38| 37| 63| ?| 58668|
| 28| 25| 43| 33| 67| ?| 58672|
| 29| 26| 36| 38| 62| ?| 58675|
| 30| 23| 44| 33| 67| ?| 58680|
| 31| 28| 36| 36| 64| ?| 58684|
| 32| 25| 41| 34| 66| ?| 58690|

0 Kudos
Timothy_Hall
Champion
Champion

Networker Networker‌ you didn't post the output of fw ctl affinity -l -r, but based on the CPU behavior it would appear you have a 2/30 split of SND/IRQ cores to Firewall Workers.  The low RX-DRP rate reported by netstat seems to indicate that the SND/IRQ cores are servicing the network ring buffers adequately.

However I'm noticing a lot of user/process space CPU usage on the Firewall Worker cores and given you have HTTPS Inspection enabled, I'd guess that the wstlsd process assigned to each worker core in the cause.  All traffic that is subject to HTTPS Inspection will go F2F.  Selecting "Categorize HTTPS Sites" without enabling full-fledged HTTPS Inspection will also cause this to a lesser degree.  Please review your HTTPS Inspection policy, you should NEVER use the object "Any" in the source or destination fields as it will pull all kinds of traffic into F2F.

The only other blade you have enabled that can cause major trips up into process space through F2F is Threat Emulation via the dplu and ted processes.  I'd say your next step is to identify via top what processes are eating process/user space CPU time on the worker cores (2-31) and that should get you going in the right direction.

In regards to the zero templating rate (Accelerated conns/Total conns) noted by another poster, this is caused by enabling the anti-bot blade and is noted in the second edition of my book.  Also if you have anything other than "Firewall" selected in your first policy layer (Network) that will also crater the SecureXL templating rate to zero.  Doesn't matter all that much performance-wise though due to the new Column-based rulebase matching feature in R80.10.

--
Second Edition of my "Max Power" Firewall Book
Now Available at http://www.maxpowerfirewalls.com

Gateway Performance Optimization R81.20 Course
now available at maxpowerfirewalls.com
0 Kudos
Thecoder
Collaborator

hi Timm

i attached new log file about your requests. https inspection active on fw. i saw that there are many wstlsd process in result of top command. And we have two access policy layers:security and application. firewall layer is only opened in security layer. 

0 Kudos
Thecoder
Collaborator

i exchanged https inspection rules(sources=any->all network range destination=ayn->internet)

but amount of traffic of f2f didnt change.

0 Kudos
Timothy_Hall
Champion
Champion

Networker Networker‌ after looking at your revised file you have a 4/28 split.  Pretty sure your high F2F is caused by HTTPS Inspection and the associated process space trip up to wstlsd, unfortunately there is very little you can do about it other than optimizing your HTTPS Inspection policy as mentioned.  pkxld is running so you definitely have Gaia executing in 64-bit mode and are taking full advantage of the various processor extensions and AES-NI.  The only way to conclusively determine if HTTPS Inspection is causing the high F2F is to disable it on the gateway/cluster temporarily, run fwaccel stats -r, wait a few minutes then run fwaccel stats -s again.

The zero SecureXL templating rate (Accelerated conns/Total conns) is being caused by anti-bot and has nothing to do with the high F2F percentage.

--
Second Edition of my "Max Power" Firewall Book
Now Available at http://www.maxpowerfirewalls.com

Gateway Performance Optimization R81.20 Course
now available at maxpowerfirewalls.com
0 Kudos
Timothy_Hall
Champion
Champion

I almost put this in my reply above, but wasn't sure how much was public yet about R80.20.  But now that the R80.20 has entered public EA and the release notes are semi-public, I can state that there is some relief coming for the high CPU impact of HTTPS Inspection, even if your firewall does not have the new Falcon accelerator card (but of course the card will *really* help a lot).  Still going through all the new R80.20 stuff but looks quite promising thus far.

--
Second Edition of my "Max Power" Firewall Book
Now Available at http://www.maxpowerfirewalls.com

Gateway Performance Optimization R81.20 Course
now available at maxpowerfirewalls.com
0 Kudos
Jelle_Hazenberg
Collaborator
Collaborator

Hi,

Since when did you have the performance problems (Is there something changed in the environment)?

Which blades are there enabled? (enter enabled_blades in expert mode)

Show the ouput of the "sar" command

Look at the output of fwaccel conns and compare them to the following rules:


• The first packet of any new TCP or UDP session, unless there is a template
• Connections destined to or from the Security Gateway
• Connections that require Security servers
• Connections that have a Handler (ICMP, FTP,)
• Some IPS features (IP, ID, TTL)
• Multicast packets

Best luck with the troubleshooting!

Sounds like an interesting but yet irritating problem...

Greetz,

Jelle

0 Kudos
Thecoder
Collaborator

we opened strict inspection and ips profile. enabled_blades commands applied in expert mode

i attached sar log file to original post.

0 Kudos
Martin_Valenta
Advisor

Well if you have enabled NGTP blades you cannot expect much to be accelarated. All NGTP traffic goes to PXL. You can verify that once you disable those blades, all will go to Accelerated path

0 Kudos
Thecoder
Collaborator

i disabled ips and threat prevention for test. but the result not changed. not accelerated.

0 Kudos
Martin_Valenta
Advisor

and with expert command: fwaccel off/on ?

0 Kudos
Thecoder
Collaborator

yes

0 Kudos
Martin_Valenta
Advisor

fwaccel stat - shows what?

0 Kudos
Thecoder
Collaborator

result of fwaccel stat not changed. accelerated conn=%0

0 Kudos
Martin_Valenta
Advisor

That would mean that traffic passed throw gateway cannot be accelarated. What kind of services are passing firewall?

0 Kudos
G_W_Albrecht
Legend
Legend

Thecoder
Collaborator

yes, traffic passed throw gateway but not accelarated. our top protocol http,https,vpn,imap,pop3,sql,kerboros,naspter,dns,echo-request

0 Kudos
Markus_Genser
Contributor

Do you use ISP redundancy with load sharing?

0 Kudos
Danny_Olson
Contributor

Yeah, you can only accelerate traffic, or inspect it but not both. I have similar stats (below), and no traffic being accelerated, but we average 50MB of throughput going though the box (on a 12600), so we can get away with it. I have FW, VPN, APP/URLF, IA, AV, AB, IPS. What amount of throughput are you seeing though the box? Are you seeing the performance issues all the time, or just certain times? Do you have high connection rate? Dynamic dispatcher address a lot of issues. Are you seeing drops/overruns?  Maybe your SND/Dispatcher ratio could use some adjusting? 

Accelerated conns/Total conns : 49/37356 (0%)
Accelerated pkts/Total pkts : 3376791/195771942 (1%)
F2Fed pkts/Total pkts : 11224710/195771942 (5%)
PXL pkts/Total pkts : 181170441/195771942 (92%)
QXL pkts/Total pkts : 0/195771942 (0%)

0 Kudos
Thecoder
Collaborator

our throughput is between 800mb-1gb.we are seeing the performance issues in working time. we have average 100000 connection. i appiled corexl. and allocated cpu for important intarfaces

0 Kudos
PhoneBoy
Admin
Admin

Even with all the blades enabled, you shouldn't see the amount of F2F traffic you're seeing.

0 Kudos
Thecoder
Collaborator

probably we found the problem. we realized that there were too many /connect traffics. and we disabled to redirect to the captive portal. And The cpu down from %80 to %45

0 Kudos

Leaderboard

Epsum factorial non deposit quid pro quo hic escorol.

Upcoming Events

    CheckMates Events