topic Imbalance of f2f and accelerated traffic between security group members in Hyperscale Firewall (Maestro)

Imbalance of f2f and accelerated traffic between security group members

kamilazat — Mon, 08 Apr 2024 13:05:58 GMT

I see a big imbalance in fwaccel stats -s output from two SGMs (Maestro R81.10 JHF Take95). There are no other members in the SG. Here are the outputs:

Member 1

----------------------

fwaccel stats -s

----------------------

Accelerated conns/Total conns : 0/0 (0%)

LightSpeed conns/Total conns : 0/0 (0%)

Accelerated pkts/Total pkts : 0/73614996 (0%)

LightSpeed pkts/Total pkts : 0/73614996 (0%)

F2Fed pkts/Total pkts : 73614996/73614996 (100%)

F2V pkts/Total pkts : 0/73614996 (0%)

CPASXL pkts/Total pkts : 0/73614996 (0%)

PSLXL pkts/Total pkts : 0/73614996 (0%)

CPAS pipeline pkts/Total pkts : 0/73614996 (0%)

PSL pipeline pkts/Total pkts : 0/73614996 (0%)

CPAS inline pkts/Total pkts : 0/73614996 (0%)

PSL inline pkts/Total pkts : 0/73614996 (0%)

QOS inbound pkts/Total pkts : 0/73614996 (0%)

QOS outbound pkts/Total pkts : 0/73614996 (0%)

Corrected pkts/Total pkts : 0/73614996 (0%)

Member 2

----------------------

fwaccel stats -s

----------------------

Accelerated conns/Total conns : 4476/4498 (99%)

LightSpeed conns/Total conns : 0/4498 (0%)

Accelerated pkts/Total pkts : 1779594854/2163662486 (82%)

LightSpeed pkts/Total pkts : 0/2163662486 (0%)

F2Fed pkts/Total pkts : 384067632/2163662486 (17%)

F2V pkts/Total pkts : 58094845/2163662486 (2%)

CPASXL pkts/Total pkts : 0/2163662486 (0%)

PSLXL pkts/Total pkts : 15942198/2163662486 (0%)

CPAS pipeline pkts/Total pkts : 0/2163662486 (0%)

PSL pipeline pkts/Total pkts : 0/2163662486 (0%)

CPAS inline pkts/Total pkts : 0/2163662486 (0%)

PSL inline pkts/Total pkts : 0/2163662486 (0%)

QOS inbound pkts/Total pkts : 0/2163662486 (0%)

QOS outbound pkts/Total pkts : 0/2163662486 (0%)

Corrected pkts/Total pkts : 0/2163662486 (0%)

Accept and NAT Templates are enabled on both members. They both appear as Active-Active in cphaprob state output, which, I assume, is expected. However, I’m having a hard time understanding the underlying reason for this behavior. Is this by design or am I missing something?

By the way, I remember reading a post by Tim Hall mentioning that seeing 100% f2f traffic can be expected, but I couldn’t find any resources to either explain it or back it up. Maybe it could be related to my case.

Cheers!

Re: Imbalance of f2f and accelerated traffic between security group members

Machine_Head — Mon, 08 Apr 2024 14:18:54 GMT

Try resetting the counters and then seeing again, you might get a similar number but best to start from scratch

g_fwaccel stats -r

Might be possible that member 2 is processing most of the traffic, perhaps check "asg perf -v -p -c"

Re: Imbalance of f2f and accelerated traffic between security group members

Dario_Perez — Mon, 08 Apr 2024 14:20:25 GMT

First check if both members are sync and if they can see the total of packet "asg perf -v" if you can see there are let say 2k packets and 100 are on SGM1 and 1900 on SGM, yes are not balance but if you have like 10 and only 2/8 is not big deal, also check the size of the packet. and if the secureXL is enable on both.

Also fwaccel stats -s is not the right way to measure the balance of traffic should be asg perf -v

Check if you have the L4 enabled on system as well

gclish -c "show distribution l4-mode"

Re: Imbalance of f2f and accelerated traffic between security group members

Timothy_Hall — Mon, 08 Apr 2024 14:30:13 GMT

100% F2F/slowpath is expected on a standby member of a traditional ClusterXL HA cluster because it is only handling connections to and from itself in standby mode, and those non-transiting connections are always handled F2F.

For Maestro specifically it is possible that the orchestrator is not sending any connections to member 1, and therefore it is only handling connections to and from itself such as HyperSync. Is there a reasonable diversity of IP addresses talking to each other through the security group? Or is this a lab environment with only a few stations passing traffic?

Use the asg search command to take a quick look at what transiting connections (if any) are being handled on member 1. Would also recommend running show distribution verification verbose to ensure you don't have a distribution issue, here are the two pages from my Gateway Performance Optimization Course covering these commands for Maestro:

Re: Imbalance of f2f and accelerated traffic between security group members

kamilazat — Tue, 09 Apr 2024 08:28:58 GMT

Thank you for the information. We are in the process of a production environment analysis. It's a relatively big environment, and I can see close to 1000 different IPs on accelerated connections list. I think the answer for me lies somewhere in distribuion modes, which I'm not even sure if configured at all.

At the same time, your explanation about the non-transiting traffic would explain the 100% f2f, and the number of the packets (73m on M1 vs 2b on M2).

Thank you again for the information!