Solved: Re: CPU utilization

cyberfinder · ‎2021-12-10

could you please advice us below error is it normal behavior or need to do a fine tuning,

coreXL has been disabled, VSX 23500

CPU 30-Day Average - OK
CPU 30-Day Peak - WARNING
CPU 0 Peak Usage: 100
CPU 1 Peak Usage: 100
CPU 2 Peak Usage: 99
CPU 3 Peak Usage: 98
CPU 4 Peak Usage: 99
CPU 5 Peak Usage: 100
CPU 6 Peak Usage: 100
CPU 7 Peak Usage: 99
CPU 8 Peak Usage: 80
CPU 9 Peak Usage: 100
CPU 10 Peak Usage: 87
11 core(s) out of 40 went over 80% in the last month.

fw ctl affinity -l -v
Interface Mgmt (irq 63): CPU 0
VS_0 fwk: CPU 8 9 10 11 12 13 14 15 16 17 18 19
VS_1 fwk: CPU 8 9 10 11 12 13 14 15 16 17 18 19
VS_2 fwk: CPU 8 9 10 11 12 13 14 15 16 17 18 19
VS_3 fwk: CPU 8 9 10 11 12 13 14 15 16 17 18 19
VS_4 fwk: CPU 8 9 10 11 12 13 14 15 16 17 18 19
VS_5 fwk: CPU 8 9 10 11 12 13 14 15 16 17 18 19
VS_6 fwk: CPU 8 9 10 11 12 13 14 15 16 17 18 19
VS_7 fwk: CPU 8 9 10 11 12 13 14 15 16 17 18 19
VS_8 fwk: CPU 8 9 10 11 12 13 14 15 16 17 18 19
VS_9 fwk: CPU 8 9 10 11 12 13 14 15 16 17 18 19
VS_10 fwk: CPU 8 9 10 11 12 13 14 15 16 17 18 19
VS_11 fwk: CPU 8 9 10 11 12 13 14 15 16 17 18 19
VS_12 fwk: CPU 8 9 10 11 12 13 14 15 16 17 18 19
VS_13 fwk: CPU 8 9 10 11 12 13 14 15 16 17 18 19
VS_14 fwk: CPU 8 9 10 11 12 13 14 15 16 17 18 19
VS_15 fwk: CPU 8 9 10 11 12 13 14 15 16 17 18 19
VS_16 fwk: CPU 8 9 10 11 12 13 14 15 16 17 18 19
VS_17 fwk: CPU 8 9 10 11 12 13 14 15 16 17 18 19
VS_18 fwk: CPU 8 9 10 11 12 13 14 15 16 17 18 19
VS_19 fwk: CPU 8 9 10 11 12 13 14 15 16 17 18 19
VS_20 fwk: CPU 8 9 10 11 12 13 14 15 16 17 18 19
VS_21 fwk: CPU 8 9 10 11 12 13 14 15 16 17 18 19
VS_22 fwk: CPU 8 9 10 11 12 13 14 15 16 17 18 19
VS_23 fwk: CPU 8 9 10 11 12 13 14 15 16 17 18 19
VS_24 fwk: CPU 8 9 10 11 12 13 14 15 16 17 18 19
VS_25 fwk: CPU 8 9 10 11 12 13 14 15 16 17 18 19
VS_26 fwk: CPU 8 9 10 11 12 13 14 15 16 17 18 19
VS_27 fwk: CPU 8 9 10 11 12 13 14 15 16 17 18 19
VS_28 fwk: CPU 8 9 10 11 12 13 14 15 16 17 18 19
VS_29 fwk: CPU 8 9 10 11 12 13 14 15 16 17 18 19
VS_30 fwk: CPU 8 9 10 11 12 13 14 15 16 17 18 19
VS_31 fwk: CPU 8 9 10 11 12 13 14 15 16 17 18 19
Interface Sync: has multi queue enabled
Interface eth2-01: has multi queue enabled
Interface eth2-02: has multi queue enabled
Interface eth2-03: has multi queue enabled
Interface eth2-04: has multi queue enabled
Interface eth1-01: has multi queue enabled
Interface eth1-02: has multi queue enabled
Interface eth1-03: has multi queue enabled
Interface eth1-04: has multi queue enabled

cpmq get -v
Note: 'cpmq' is deprecated and no longer supported. For multiqueue management, please use 'mq_mng'
Current multiqueue status:
Total 40 cores. Multiqueue 28 cores
i/f type state mode cores
------------------------------------------------------------------------------------------------
Mgmt igb Up Off 0
Sync igb Up Auto (2/2) 0,20
eth1-01 ixgbe Up Auto (16/16) 0,20,1,21,2,22,3,23,4,24,5,25,
6,26,7,27
eth1-02 ixgbe Up Auto (16/16) 0,20,1,21,2,22,3,23,4,24,5,25,
6,26,7,27
eth1-03 ixgbe Up Auto (16/16) 0,20,1,21,2,22,3,23,4,24,5,25,
6,26,7,27
eth1-04 ixgbe Up Auto (16/16) 0,20,1,21,2,22,3,23,4,24,5,25,
6,26,7,27
eth2-02 igb Up Auto (8/8) 0,20,1,21,2,22,3,23
eth2-03 igb Up Auto (8/8) 0,20,1,21,2,22,3,23

emmap · ‎2021-12-12

From the output there it looks like you have hyperthreading enabled but are not allowing the use of all the CPU cores. You definitely could use some tuning of affinities here.

View solution in original post

HeikoAnkenbrand · ‎2022-01-01

Here are some tips:

1) VS0 is an administrative instance and no SecureXL is used here therefore 99% F2Fed traffic.
Set the CoreXL cores to the minimum. In some cases I have even disabled CoreXL for VS0.

2) Use Hyper-Threading on applications and disable it on Open Servers if necessary.

3) If the automatic affinity does not work, set the affinity manually for all VS. But I think this is a bug in this version.

4) Set the CoreXL number in the SmartConsole correctly for each VS. Do not overbook the CoreXL cores.

5) Check SecureXL and CoreXL distribution for all virtual systems from VS1 to VSx (not VS0) and set the affinity manually if necessary.

6) For manually affinity, make sure that the Hyper-Threading cores match the physical cores.

7) The multiqueue settings for the ixgbe and igb network cards look good in the last screenshot.

➜ CCSM Elite, CCME, CCTE ➜ www.checkpoint.tips

View solution in original post

PhoneBoy · ‎2021-12-12

Super Seven command output?
https://community.checkpoint.com/t5/Scripts/S7PAC-Super-Seven-Performance-Assessment-Commands/m-p/40...

cyberfinder · ‎2021-12-13

Accept Templates : enabled
Drop Templates : disabled
NAT Templates : enabled

fwaccel stats -s
Accelerated conns/Total conns : 0/0 (0%)
Accelerated pkts/Total pkts : 0/3113220103 (0%)
F2Fed pkts/Total pkts : 3113210107/311329999 (99%)
F2V pkts/Total pkts : 0/3113220103 (0%)
CPASXL pkts/Total pkts : 0/3113220103 (0%)
PSLXL pkts/Total pkts : 0/3113220103 (0%)
CPAS pipeline pkts/Total pkts : 0/3113220103 (0%)
PSL pipeline pkts/Total pkts : 0/3113220103 (0%)
CPAS inline pkts/Total pkts : 0/3113220103 (0%)
PSL inline pkts/Total pkts : 0/3113220103 (0%)
QOS inbound pkts/Total pkts : 0/3113220103 (0%)
QOS outbound pkts/Total pkts : 0/3113220103 (0%)
Corrected pkts/Total pkts : 0/3113220103 (0%)

grep -c ^processor /proc/cpuinfo
40

/sbin/cpuinfo
HyperThreading=enabled

fw ctl affinity -l -r
CPU 0: Mgmt
CPU 1:
CPU 2:
CPU 3:
CPU 4:
CPU 5:
CPU 6:
CPU 7:
CPU 8:
CPU 9:
CPU 10:
CPU 11:
CPU 12:
CPU 13:
CPU 14:
CPU 15:
CPU 16:
CPU 17:
CPU 18:
CPU 19:
CPU 20:
CPU 21:
CPU 22:
CPU 23:
CPU 24:
CPU 25:
CPU 26:
CPU 27:
CPU 28:
CPU 29:
CPU 30:
CPU 31:
CPU 32:
CPU 33:
CPU 34:
CPU 35:
CPU 36:
CPU 37:
CPU 38:
CPU 39:
All:
Interface Sync: has multi queue enabled
Interface eth2-01: has multi queue enabled
Interface eth2-02: has multi queue enabled
Interface eth2-03: has multi queue enabled
Interface eth2-04: has multi queue enabled
Interface eth1-01: has multi queue enabled
Interface eth1-02: has multi queue enabled
Interface eth1-03: has multi queue enabled
Interface eth1-04: has multi queue enabled

netstat -ni
Kernel Interface table
Iface MTU Met RX-OK RX-ERR RX-DRP RX-OVR TX-OK TX-ERR TX-DRP TX-OVR Flg
Mgmt 1500 0 3387314698 0 1235992 0 507436495 0 0 0 BMRU
Sync 1500 0 12868659837 0 136913 0 19588613103 0 0 0 BMRU
bond1 1500 0 9790681557828 45 2099301 0 9823567402850 0 0 0 BMmRU
eth1-01 1500 0 3337377268774 5 343926 0 500475092615 0 0 0 BMsRU
eth1-02 1500 0 1918030761240 10 343924 0 3607939305888 0 0 0 BMsRU
eth1-03 1500 0 1817018918925 21 343926 0 3301129162365 0 0 0 BMsRU
eth1-04 1500 0 2718254559263 9 343926 0 2414023836483 0 0 0 BMsRU
eth2-02 1500 0 118944801943 0 0 92 76048481553 0 0 0 BMRU
eth2-03 1500 0 29098955107 0 16483356 0 27793925672 0 0 0 BMRU
lo 65536 0 61909008 0 0 0 61909008 0 0 0 ALdPNRU

fw ctl multik stat
CoreXL is disabled

cpstat os -f multi_cpu -o 1 -c 5

Processors load
---------------------------------------------------------------------------------
|CPU#|User Time(%)|System Time(%)|Idle Time(%)|Usage(%)|Run queue|Interrupts/sec|
---------------------------------------------------------------------------------
| 1| 0| 6| 94| 6| ?| 342518|
| 2| 0| 53| 47| 53| ?| 342517|
| 3| 0| 8| 92| 8| ?| 342517|
| 4| 0| 6| 94| 6| ?| 342516|
| 5| 0| 16| 84| 16| ?| 342515|
| 6| 0| 4| 96| 4| ?| 342514|
| 7| 0| 5| 95| 5| ?| 342514|
| 8| 0| 6| 94| 6| ?| 342515|
| 9| 4| 7| 89| 11| ?| 342515|
| 10| 5| 6| 89| 11| ?| 342514|
| 11| 7| 9| 84| 16| ?| 342514|
| 12| 6| 7| 87| 13| ?| 342513|
| 13| 6| 8| 87| 13| ?| 342511|
| 14| 5| 7| 88| 12| ?| 342511|
| 15| 6| 6| 87| 13| ?| 342510|
| 16| 5| 7| 88| 12| ?| 342519|
| 17| 5| 7| 88| 12| ?| 342518|
| 18| 7| 9| 84| 16| ?| 342518|
| 19| 6| 9| 85| 15| ?| 342518|
| 20| 5| 7| 87| 13| ?| 342516|
| 21| 0| 7| 93| 7| ?| 342514|
| 22| 0| 13| 87| 13| ?| 342514|
| 23| 0| 8| 92| 8| ?| 342512|
| 24| 1| 8| 92| 8| ?| 342510|
| 25| 0| 4| 96| 4| ?| 342509|
| 26| 1| 5| 94| 6| ?| 342509|
| 27| 0| 30| 70| 30| ?| 342508|
| 28| 0| 8| 92| 8| ?| 342507|
| 29| 0| 0| 100| 0| ?| 342506|
| 30| 0| 0| 100| 0| ?| 342505|
| 31| 1| 0| 99| 1| ?| 342504|
| 32| 0| 0| 100| 0| ?| 342504|
| 33| 1| 0| 99| 1| ?| 342501|
| 34| 0| 0| 100| 0| ?| 342501|
| 35| 1| 0| 99| 1| ?| 342499|
| 36| 0| 0| 100| 0| ?| 342497|
| 37| 1| 0| 99| 1| ?| 342496|
| 38| 2| 0| 98| 2| ?| 342494|
| 39| 3| 0| 96| 4| ?| 342493|
| 40| 3| 0| 97| 3| ?| 342492|
---------------------------------------------------------------------------------

Processors load
---------------------------------------------------------------------------------
|CPU#|User Time(%)|System Time(%)|Idle Time(%)|Usage(%)|Run queue|Interrupts/sec|
---------------------------------------------------------------------------------
| 1| 0| 6| 94| 6| ?| 342528|
| 2| 0| 53| 47| 53| ?| 342527|
| 3| 0| 8| 92| 8| ?| 342527|
| 4| 0| 6| 94| 6| ?| 342526|
| 5| 0| 16| 84| 16| ?| 342525|
| 6| 0| 4| 96| 4| ?| 342524|
| 7| 0| 5| 95| 5| ?| 342524|
| 8| 0| 6| 94| 6| ?| 342525|
| 9| 4| 7| 89| 11| ?| 342525|
| 10| 5| 6| 89| 11| ?| 342524|
| 11| 7| 9| 84| 16| ?| 342524|
| 12| 6| 7| 87| 13| ?| 342523|
| 13| 6| 8| 87| 13| ?| 342521|
| 14| 5| 7| 88| 12| ?| 342521|
| 15| 6| 6| 87| 13| ?| 342520|
| 16| 5| 7| 88| 12| ?| 342519|
| 17| 5| 7| 88| 12| ?| 342518|
| 18| 7| 9| 84| 16| ?| 342518|
| 19| 6| 9| 85| 15| ?| 342518|
| 20| 5| 7| 87| 13| ?| 342516|
| 21| 0| 7| 93| 7| ?| 342514|
| 22| 0| 13| 87| 13| ?| 342514|
| 23| 0| 8| 92| 8| ?| 342512|
| 24| 1| 8| 92| 8| ?| 342510|
| 25| 0| 4| 96| 4| ?| 342509|
| 26| 1| 5| 94| 6| ?| 342509|
| 27| 0| 30| 70| 30| ?| 342508|
| 28| 0| 8| 92| 8| ?| 342507|
| 29| 0| 0| 100| 0| ?| 342506|
| 30| 0| 0| 100| 0| ?| 342505|
| 31| 1| 0| 99| 1| ?| 342504|
| 32| 0| 0| 100| 0| ?| 342504|
| 33| 1| 0| 99| 1| ?| 342501|
| 34| 0| 0| 100| 0| ?| 342501|
| 35| 1| 0| 99| 1| ?| 342499|
| 36| 0| 0| 100| 0| ?| 342497|
| 37| 1| 0| 99| 1| ?| 342496|
| 38| 2| 0| 98| 2| ?| 342494|
| 39| 3| 0| 96| 4| ?| 342493|
| 40| 3| 0| 97| 3| ?| 342492|
---------------------------------------------------------------------------------

Processors load
---------------------------------------------------------------------------------
|CPU#|User Time(%)|System Time(%)|Idle Time(%)|Usage(%)|Run queue|Interrupts/sec|
---------------------------------------------------------------------------------
| 1| 0| 6| 94| 6| ?| 327013|
| 2| 0| 53| 47| 53| ?| 654040|
| 3| 1| 8| 92| 8| ?| 327023|
| 4| 0| 5| 95| 5| ?| 654050|
| 5| 0| 16| 84| 16| ?| 654042|
| 6| 0| 4| 96| 4| ?| 327034|
| 7| 0| 6| 94| 6| ?| 327035|
| 8| 0| 5| 96| 4| ?| 327023|
| 9| 6| 7| 87| 13| ?| 327004|
| 10| 6| 7| 87| 13| ?| 327001|
| 11| 6| 7| 88| 12| ?| 326984|
| 12| 5| 7| 88| 12| ?| 326982|
| 13| 6| 6| 87| 13| ?| 326970|
| 14| 7| 8| 85| 15| ?| 326972|
| 15| 6| 7| 87| 13| ?| 326976|
| 16| 4| 6| 89| 11| ?| 326976|
| 17| 7| 9| 84| 16| ?| 326978|
| 18| 5| 7| 88| 12| ?| 326977|
| 19| 6| 8| 86| 14| ?| 326976|
| 20| 6| 7| 87| 13| ?| 653944|
| 21| 0| 9| 92| 8| ?| 326981|
| 22| 0| 12| 88| 12| ?| 653974|
| 23| 3| 10| 87| 13| ?| 326984|
| 24| 0| 9| 91| 9| ?| 653993|
| 25| 0| 4| 96| 4| ?| 653993|
| 26| 0| 3| 97| 3| ?| 326995|
| 27| 0| 31| 69| 31| ?| 327008|
| 28| 0| 6| 94| 6| ?| 327018|
| 29| 0| 1| 99| 1| ?| 327016|
| 30| 0| 0| 100| 0| ?| 327012|
| 31| 4| 0| 96| 4| ?| 327012|
| 32| 6| 0| 94| 6| ?| 327004|
| 33| 2| 0| 98| 2| ?| 326992|
| 34| 3| 0| 97| 3| ?| 326988|
| 35| 0| 0| 100| 0| ?| 327003|
| 36| 1| 0| 99| 1| ?| 327015|
| 37| 1| 0| 99| 1| ?| 327017|
| 38| 0| 0| 100| 0| ?| 327025|
| 39| 3| 0| 97| 3| ?| 327033|
| 40| 1| 0| 99| 1| ?| 327044|
---------------------------------------------------------------------------------

Processors load
---------------------------------------------------------------------------------
|CPU#|User Time(%)|System Time(%)|Idle Time(%)|Usage(%)|Run queue|Interrupts/sec|
---------------------------------------------------------------------------------
| 1| 0| 7| 93| 7| ?| 410032|
| 2| 0| 56| 44| 56| ?| 820145|
| 3| 3| 10| 88| 12| ?| 410090|
| 4| 2| 8| 91| 9| ?| 410105|
| 5| 0| 18| 82| 18| ?| 820277|
| 6| 0| 6| 94| 6| ?| 820337|
| 7| 0| 6| 94| 6| ?| 820387|
| 8| 0| 8| 92| 8| ?| 820462|
| 9| 7| 13| 80| 20| ?| 410271|
| 10| 9| 13| 78| 22| ?| 410315|
| 11| 10| 11| 80| 20| ?| 820703|
| 12| 8| 8| 84| 16| ?| 410393|
| 13| 5| 8| 88| 12| ?| 410428|
| 14| 4| 7| 90| 10| ?| 410468|
| 15| 3| 4| 93| 7| ?| 820980|
| 16| 6| 7| 87| 13| ?| 410499|
| 17| 12| 20| 69| 31| ?| 821036|
| 18| 6| 9| 84| 16| ?| 410530|
| 19| 8| 10| 81| 19| ?| 410543|
| 20| 6| 11| 83| 17| ?| 410574|
| 21| 0| 8| 92| 8| ?| 821160|
| 22| 0| 13| 87| 13| ?| 821182|
| 23| 1| 11| 89| 11| ?| 410619|
| 24| 5| 8| 87| 13| ?| 410619|
| 25| 1| 4| 94| 6| ?| 410637|
| 26| 0| 10| 90| 10| ?| 410646|
| 27| 0| 30| 70| 30| ?| 821299|
| 28| 0| 12| 88| 12| ?| 821294|
| 29| 0| 0| 100| 0| ?| 410642|
| 30| 0| 0| 100| 0| ?| 410654|
| 31| 1| 0| 99| 1| ?| 410656|
| 32| 0| 0| 100| 0| ?| 410673|
| 33| 0| 0| 100| 0| ?| 410694|
| 34| 0| 0| 100| 0| ?| 410696|
| 35| 0| 0| 100| 0| ?| 410694|
| 36| 0| 1| 99| 1| ?| 410696|
| 37| 0| 0| 100| 0| ?| 410707|
| 38| 1| 0| 99| 1| ?| 410708|
| 39| 8| 0| 92| 8| ?| 410701|
| 40| 2| 0| 97| 3| ?| 410700|
---------------------------------------------------------------------------------

PhoneBoy · ‎2021-12-13

Here's your issue right here:

F2Fed pkts/Total pkts : 3113210107/311329999 (99%)

And I don't think you disable CoreXL with VSX.
What version/JHF is this?
If you're not on at least R80.40, you should be.

emmap · ‎2021-12-13

CoreXL on VS0 should not be enabled, there's no need for it. As VS0 is generally only there as an administrative instance, receiving connections to it but not routing anything through it, it's normal for it to be pretty much 100% F2F.

cyberfinder · ‎2021-12-16

R80.40 JHF 120 , do i need to enable coreXL ?

genisis__ · ‎2021-12-19

I would installed the Latest 'GA' jumbo (HFA139), for us we are currently running HFA125 only because 139 did not go into GA before our scheduled window (Also was this a clean install or upgrade).

Also another option is to install R81 with the latest GA jumbo and take advantage of dynamic split.

You should, if possible monitor the amount of CPU is used on a per VS basis (fwk process), you may find some VSs need more fwk processes then other, so could tune in this way, manual affinity is of course another way, but can be a pain.

VS0 - as its been stated, in general should not have COREXL on.

What does 'fwaccel stat' show for the specific VS's?

cyberfinder · ‎2021-12-31

Name Value Name Value
---------------------------- ------------ ---------------------------- ------------

Accelerated Path
--------------------------------------------------------------------------------------
accel packets 23789168 accel bytes 8555520695
outbound packets 23789168 outbound bytes 8888505053
conns created 139286 conns deleted 138967
C total conns 319 C TCP conns 311
C non TCP conns 8 nat conns 0
dropped packets 33 dropped bytes 4636
fragments received 0 fragments transmit 0
fragments dropped 0 fragments expired 0
IP options stripped 0 IP options restored 0
IP options dropped 0 corrs created 0
corrs deleted 0 C corrections 0
corrected packets 0 corrected bytes 0

Accelerated VPN Path
--------------------------------------------------------------------------------------
C crypt conns 0 enc bytes 0
dec bytes 0 ESP enc pkts 0
ESP enc err 0 ESP dec pkts 0
ESP dec err 0 ESP other err 0
espudp enc pkts 0 espudp enc err 0
espudp dec pkts 0 espudp dec err 0
espudp other err 0

Medium Streaming Path
--------------------------------------------------------------------------------------
CPASXL packets 0 PSLXL packets 4571
CPASXL async packets 0 PSLXL async packets 4571
CPASXL bytes 0 PSLXL bytes 471123
C CPASXL conns 0 C PSLXL conns 0
CPASXL conns created 0 PSLXL conns created 597
PXL FF conns 0 PXL FF packets 0
PXL FF bytes 0 PXL FF acks 0
PXL no conn drops 0

Pipeline Streaming Path
--------------------------------------------------------------------------------------
PSL Pipeline packets 0 PSL Pipeline bytes 0
CPAS Pipeline packets 0 CPAS Pipeline bytes 0

Inline Streaming Path
--------------------------------------------------------------------------------------
PSL Inline packets 0 PSL Inline bytes 0
CPAS Inline packets 0 CPAS Inline bytes 0

Buffer Path
--------------------------------------------------------------------------------------
Buffer path buffers 0 Buffer path bytes 0

TLS PARSER
--------------------------------------------------------------------------------------
RECORD INFO 0

TLS DECRYPT
--------------------------------------------------------------------------------------
TLS INSPECTION 0 TLS HANDSHAKE 0
TLS RECORD LAYER 0 TLS CRYPTO 0

HTTP DISP
--------------------------------------------------------------------------------------
ACTIVATE WS MAIN 0 EXEC NO HTTP CMI CONTEXT 0

WS LITE
--------------------------------------------------------------------------------------
WS TX COMPLETED 0 WS FORWARD TO MAIN 0
WS NOTIFY TIMEOUT 0 WS HANDLE EVENT 0
WS CHUNKED ERROR 0 WS GZIP EVENT 0
WS ADD MAC HEADER 0 WS IS STICKY ACTIVE 0
WS TIER1 JOB ERROR 0 WS TIER1 HAS MATCHES 0
CML MATCHES 0 TOTAL UPLOADED JOBS 0
TOTAL JOBS 0

ADVP
--------------------------------------------------------------------------------------
ADVP FORW TO MAIN 0 ADVP HOLD TIMEOUT 0

QoS Paths
--------------------------------------------------------------------------------------
QoS General Information:
------------------------
Total QoS Conns 0 QoS Classify Conns 0
QoS Classify flow 0 Reclassify QoS policy 0

FireWall QoS Path:
------------------
Enqueued IN packets 0 Enqueued OUT packets 0
Dequeued IN packets 0 Dequeued OUT packets 0
Enqueued IN bytes 0 Enqueued OUT bytes 0
Dequeued IN bytes 0 Dequeued OUT bytes 0

Accelerated QoS Path:
---------------------
Enqueued IN packets 0 Enqueued OUT packets 0
Dequeued IN packets 0 Dequeued OUT packets 0
Enqueued IN bytes 0 Enqueued OUT bytes 0
Dequeued IN bytes 0 Dequeued OUT bytes 0

Firewall Path
--------------------------------------------------------------------------------------
F2F packets 2882668 F2F bytes 777963162
TCP violations 0 F2V conn match pkts 1
F2V packets 139288 F2V bytes 9359370

GTP
--------------------------------------------------------------------------------------
gtp tunnels created 0 gtp tunnels 0
gtp accel pkts 0 gtp f2f pkts 0
gtp spoofed pkts 0 gtp in gtp pkts 0
gtp signaling pkts 0 gtp tcpopt pkts 0
gtp apn err pkts 0

General
--------------------------------------------------------------------------------------
memory used 252459652 C tcp handshake conns 0
C tcp established conns 309 C tcp closed conns 2
C tcp pxl handshake conns 0 C tcp pxl established conns 0
C tcp pxl closed conns 0 DNS DoR stats 0

(*) Statistics marked with C refer to current value, others refer to total value