ClusterXL Loadsharing Unicast Mechanism

[Amir_Arama]

Hi,

i want to understand further the mechanism of loadsharing unicast with Vmac.
for example, how a pivot member forward packet to a non pivot member? i thought that the Pivot change the destination mac of the packet to the physical mac of the non pivot and send the packet out from the same interface the packet was originally received. is it true?
also on Arp requests sent by the non pivot member does it came out with the physical IP and Mac of the non pivot member? or does it translate itself to the VIP and Vmac ?
i encounter an issue on interface that i configured the physical IP on a different subnet from the VIP, that the non pivot failed to learn the arp of the next hop on this interface (vlan interface actually). on tcpdump i saw arp request sent from the physical Mac to Broadcast Mac, and with physical IP of the nic (which is in different subnet)
for example: who has 192.168.1.1 tell 3.3.3.1 ? and no response was sent to those arp requests.
GWs version is r80.10 take 121. GAIA.

Thanks

[_Val_]

Hi,

Your understanding of LS Unicast is more or less accurate. It is important to understand that pivot is handling all incoming connections, including bidirectional communications. All packets that need to cross FW always are received by pivot before anything else.

Any who-has ARP is send from a physical interface with its MAC as source. This is valid for VLAN interfaces as well, they do not have separate MACs. Your current issue may or may not be related to ClusterXL at all.

The VLAN interface in question, what is it used for? Is it part of cluster topology? How is it configured?

[Maarten_Sjouw]

The issue with different VIP than physical network IP is a problem that I had issues with as well. in R80.10 I could not get it working at all. I would treat that as a separate problem not related to the Load Sharing. did you see the manual?

Can I ask why you would want to use load sharing in the first place? First of all you need to be in charge of the switches and the FW's if any problems occur, due to the whole Pivot system it only makes sense when you have 3 or more boxes.

[Amir_Arama]

Hi,
to Valeri:
this vlan used to connects the FWs with a Modem (the next hop) that connects to remote 3rd party company.
sure it's part of cluster topology. each member has private ip: lets say 1.1.1.1-1.1.1.2/30, and the VIP is on different subnet lets say: 192.168.1.1/30 and the next hop is 192.168.1.2/30.

to Maarten:
the different vip to physical is working for me in other r80.10 FWs clusters that work in HA mode without any issue. (did you add a localscope route?)
the issue seems to be related because when i clusterXL_admin down one member, the issue is resolved and the traffic is working, but when i bring it back up, (so two are active) it works randomally (only when packet go through the pivot).
also on the pivot i see the arp entry for the next hop, and on the non pivot i don't.

we set the LS mode historically when we had two servers that wasn't strong enough and we needed to distribute the load.

[Maarten_Sjouw]

Still I don't think this will ever gonna work, both members will be sending traffic with the same source IP, but with their own MAC. The next hop router will be confused all the time. If there is no real need to use Load Sharing I would not use it, as it will only complicate things.

Yes we did use the localscope route and it worked until the ARP cache was cleared (the cisco default of 4 hours) after doing a arping from the active member.

The gateways just did not respond to the ARP requests.

[Amir_Arama]

did you use vmac? in my case the pivot is sending the arp's with the vmac and the vip, and this is what should be, and not confused the router.

[AlekseiShelepov]

I can see already that you are using different subnets on external interface, vlan interface, and load sharing. If you add bond interface to this, you will get a setup that wasn't working for me on R77.30. So, is it a bond interface?

Well, should the default gateway reply to an ARP request coming from a different network, not configured on this interface (who has 192.168.1.1 tell 3.3.3.1)? Can you see ARP replies to local (3.3.3.X) address on the pivot member?

As Valery said, the pivot member should transfer some connections to the secondary member. And during a failover everything works fine, you said. ARP requests doesn't look like an issue to me. Maybe load balancing is not working as expected, that's why you started to dig deeper?

You can try adding a static ARP entry for the default gateway.

I can suggest (as usual) to try to use parameter fwha_forw_packet_to_not_active. Maybe the pivot firewall somehow intercept this ARP reply. Although, I think it is very unlikely that it would help in this case.

I would also advise agains using load sharing on a cluster of two firewalls. It doesn't bring really higher performance, but adds difficulties to troubleshoot.

[Amir_Arama]

Hi Aleksei,

it's not a bond interface. and this exact configuration worked very well for me for a long time on r77.30, after reinstalling servers with r80.10 the issue started.

i don't see any response to the local address on the pivot or non pivot

i started to dig because i got complains after the upgrade that browsing to a web server located behind this interface/modem works randomally, with hangups.

i tried adding static arp but gaia rejected this because the ip is not associated with any network interface.

fw ctl set int fwha_forw_packet_to_not_active 1 didn't help 😕

[Maarten_Sjouw]

Maybe this one will help:

Gratuitous ARP to force a new MAC addess towards the router

Enable binding to non-local IP addresses on-the-fly:
cat /proc/sys/net/ipv4/ip_nonlocal_bind
echo 1 > /proc/sys/net/ipv4/ip_nonlocal_bind                 <---- 0=off, 1=on
cat /proc/sys/net/ipv4/ip_nonlocal_bind

arping -c 4 -A -I eth1 172.16.1.1

[Chinmaya_Naik]

Hii  Amir Arama‌ 

How did you resolve this issue?

I am also facing the same challenge. 

Please help

fw ctl set int fw_allow_simultaneous_ping   and  fw ctl set int fwha_forw_packet_to_not_active 1 is resolved my issue ?

or below this work

Gratuitous ARP to force a new MAC addess towards the router

Enable binding to non-local IP addresses on-the-fly:
cat /proc/sys/net/ipv4/ip_nonlocal_bind
echo 1 > /proc/sys/net/ipv4/ip_nonlocal_bind <---- 0=off, 1=on
cat /proc/sys/net/ipv4/ip_nonlocal_bind

#Chinmaya Naik

[Amir_Arama]

honestly i just moved back to HA mode..