BOND configuration with RX Drop Situation

[Chinmaya_Naik]

Hi Checkmates

We come across to know that one of our interface (10G) is over utilized due to which we are getting RX Drop on that particular interface.

To overcome the issue we are plane to create BOND interface (20G) with Active/Active Mode.

In General when we configure BOND with "2" interfaces then the default minimum number of "Slave interface required is 1" so basically when one interface of that BOND goes down then Fail-over will not going to happen.

Now here we have a challenge because in our case  if one interface is goes down on that BOND then we sure getting huge RX drop during our production because one interface can not handle the traffic because its already fully utilized.

So as a solution we are planing to change the "Minimum Slave interface required set to 2" (By change in the file "$FWDIR/conf/cpha_bond_ls_config.conf). So in this case one interface is goes down then fail-over will happen and Standby will handle the traffic.

We can not use "3" interface in that BOND because we only have one 10G interface left.

So my query is that :

1) As by default BOND configuration follow N-1 slave interface required like if we have "3" interface in one BOND then by default  "Slave interface required is 2" so will there be any impact if we configure the minimum no of the interface as "2" with the total no of the interface in a bond is "2" ?

2) When we change the value of the required interface to 2 with the change in the file "$FWDIR/conf/cpha_bond_ls_config.conf" is their any impact

Kindly suggest what is the Best Practice when configured the bond and also how to overcome on this kind of scenario ?

We are using 21400 Appliance with SAM mode enable on that interface which running with R80.10 ISO with take_283 jumbo (Note: we are not able to upgrade the OS because of SAM limitation)

Regards

@Chinmaya_Naik 

[Timothy_Hall]

Where are the RX-DRPs occurring when only one physical interface of the bond is present?  On the actual SAM card physical interfaces or the backplane interfaces (eth-bp1d1/eth-bp1d2/bp_lag)?  Multi-Queue is enabled by default on interfaces associated with the SAM card when it is active and administrators are not allowed to change that. 

But what you can change is the number of SND/IRQ cores.  Are you still running with the default 2/10 split on your 21400?  (fw ctl affinity -l -r -v).  If so you may want to change that split to 4/8 (assuming the current 10 firewall instances are at least 30% idle during your busiest periods), this will hopefully keep RX-DRPs from piling up when one of your bond's physical interfaces goes down. 

I think this would be a better approach than causing a full failover in the event of a physical interface failure, unless the amount of real traffic the bond is trying to pass is consistently in excess of 10Gbps.  Be aware however that for some reason I'm vaguely recalling that SAM-based interfaces can only support a maximum of 2 queues for Multi-Queue, but I can't find any confirmation of that.  I may be thinking of something else, TAC should be able to provide an answer to this question.

[Chinmaya_Naik]

Hi @Timothy_Hall 

Thank you very much for the suggestion

Where are the RX-DRPs occurring when only one physical interface of the bond is present?

--> Its occur on the SAM Card Physical Interface not on the backplane interfaces (eth-bp1d1/eth-bp1d2/bp_lag).

So Previously one interface is present which over utilized so we are adding one more interface and create a BOND with Active/Active Mode.

We already change the core configuration (4 SND and 8 Firewall Workers)

But one point I will highlight that currently we are only using Firewall,Identity Awareness and Monitoring Blade so normally most of the traffic will handle by SecureXL.

Did you think that increasing SND on this moment will help (6/6 Split) ?

Please suggest because we don't want change the Bond Configuration because the plane configuration will fail-over the gateways when one interface goes down.

@Chinmaya_Naik 

[Timothy_Hall]

I don't understand how you could be experiencing RX-DRPs on the physical interfaces of the SAM card, as those interfaces are supposed to be able to handle wire speed in SAM mode.  I don't think that changing to a 6/6 split will help, based on this quote from sk94267:

"As a final note, this Multi-Queue support on backplane interfaces of SAM card should not be confused with Multi-Queue on SAM card ports with enabled SAM mode, which is not supported, nor is it needed."

You should be able to verify that MQ is not active on the SAM physical interfaces with cpmq -v.

This is a rare scenario where you may want to increase the ring buffer size on the SAM physical interfaces, assuming that is allowed for SAM interfaces.  I think in R80.10 this can only be done with the ethool -G command as the clish commands to do this were not added until later releases.  To make this ring buffer size change permanent you'll need to add the ethtool -G command to /etc/rc.local.  However you should check if you have an option called "rx-ringsize" for the clish command set interface and set it that way if supported.

If increasing the RX ring buffer to the maximum size still does not help, then I'd go ahead and move forward with your plan to cause a failover if one of your bond's physical interfaces fails.