Switch or Cable between ClusterXL members for the sync network?

[HeikoAnkenbrand]

A historical question in the last years is!

Do you need a switch or a cable between ClusterXL members for the sync network?

Regards,

Heiko

CAT5e cable or fiber	43
Layer 2 switch	61

[Dan_Deutzmann]

I think cabel is easy going!

[Vladimir]

ClusterXL provides not only HA, but a load sharing capabilities with more than 2 members.

[HeikoAnkenbrand]

I just want to point out that the active gateway in a HA cluster goes into the following status  "Active attention" if there is no connection via the CAT5e cable. This has no direct effect, but the active machine also reports an problem. Active attention means that a problem has been detected, but the cluster member is still forwarding packets because it is the only machine in the cluster or there are no other active machines in the cluster. In any other situation the state of the machine would be down.

I like to avoid this and use a switch. From my point of view, however, both are possible without any problems. I think switch "yes or no" is a fundamental discussion.

Regards

Heiko

[_Val_]

Moved to General Topics

[_Val_]

Concerning the subject of the poll, the answer should not be binary. What's important here is understanding. 

The main question is what happens if sync network fails. Both cluster member should detect the failure and perform a proper failover, putting the failing member down. This is theory.

In case of the cross cable, the sync failure means only cut cable of failing NIC. In both cases, Active member will remain active and standby will go into Down state. This is good for clustering, but not that good for diagnostics. You still have to figure out which cluster member has an issue. 

In case of a switch in between and ONLY in case you have configured a pingable host (look here for details: ClusterXL ATRG) you can actually say which side is having issues. If it's Active member, you will have a failover and stay on fully operational cluster member since, which gives you a better position to resolve the issue.

The cross cable option is simple and reliable, yet may require some additional work in case of NIC failure.

Switch with a pingable IP is more complex but helps with diagnostics. 

The choice is yours.

I would actually go for an option 3: get Sync over bonded interfaces with a cross cable. It is relatively simple to configure and also covers NIC failure on Sync network much better

[Norbert_Bohusch]

Another reason to have a switch (or at least two to be correct) for the sync link, might be that the two members are not at the same physical location and there is a limited number of L2 links between the locations.

[HeikoAnkenbrand]

My real opinion:

I agree with Valeri Loukine and Norbert Bohusch. I think it has to be the right network design. Both solutions work with the right configuration. In case of a switch in between and ONLY in case you have configured a pingable host (look here for details: ClusterXL ATRG).

And now something to laugh about:

Connector for switch and two sync interfaces and power over sync:

Attention! Please do not copy it and plug it into an electrical socket. It could be fatal.

Regards,

Heiko

[Nüüül]

I´d prefer the switch. But this is a personal thing...

[PhoneBoy]

Fixed a couple of typos in the original post

I remember using ATM interfaces for sync on old Nokia appliances back in the day, largely because I had no other use for them. 

That stopped working when the sync traffic became multicast (in NG, I think).

Unicast sync is coming back in R80.20

[Lari_Luoma]

I connect them directly if possible. In old times with IPSO this could cause issues, but not anymore.

[Gaurav_Pandya]

Hi,

This is all depends on your architecture. Both methods are good but if you face any issue with sync then based on issue we can change the connection method.

[Yuri_Slobodyany]

Can't really say because:

- Clustering will work just fine in both set ups

- Whether to place a switch and cabling decisions are usually not the questions a firewall guy has a vote on, designer designs and we work with what we are given...

If it was few years back I'd rephrase the poll as:

- Broadcast ?

- Multicast ?

[phlrnnr]

I like the bonded sync interface option.  Whether you use crossover cables or switches depends on the architecture / setup of the closet(s).  But this gives you a nice setup for single NIC failure without forcing clusterXL to trigger a gateway as down.

[Mark_Mitchell]

I think ultimately it is a technical design decision. Personally I have deployed Check Point Gateway HA clusters using both local single site (direct cabling and also switches) clustering and then geographically dispersed cluster. Both work excellently when the environment is suitable.

If installing a cluster into a geo dispersed fashion (2 data centre's in different cities) then care will need to be taken with the clustering to ensure that adequate bandwidth exists between those 2 sites to avoid any packet loss and then undesired fail-over scenarios. To summarize scenarios scenarios I personally have deployed clusters:

• 2 Data Centre's in different Cities connected by direct circuits and VLAN's spanned between the 2 DC's
• Single site 2x Check Point gateways deployed in a HA cluster same rack directly attached Sync interfaces
• Single site with 2x Server Rooms with Gateway in each and VLAN's spanned across switching hardware from one room to the other. 

My 2 pennies worth  both have their merits and work as good as each other if the environment has the right conditions.

[John_Fenoughty]

Historically, I have always used a direct Ethernet cable (crossover in the old days - standard Ethernet now of course).

it has been bugging me in recent times as Heiko stated that the remaining firewall goes ‘active attention’ when its partner is down. Even more than this, the alert emails (and logs too) can be confusing for the network admin who sees ‘errors’ coming from both firewalls.

So, I have been requesting a switch to put between the two firewalls, but of course this actually creates a new point of failure that didn’t exist in the simple direct-connect method...

Therefore: I think there’s only one ‘perfect’ solution that ticks all boxes: A pair of switches trunked together with a two cable Ether-channel, and a pair of bonded interfaces on each firewall connected to these switches.

We did this recently on a client site by VLANing off a few ports from the switches being used to aggregate the Internet-side routers. This saves rack-space, power etc. and made life easy while achieving what we wanted.

Using this two-switch, bonded interfaces method we introduce no extra single points of failure but also have no errors on the primary when rebooting secondary (or vice versa), however there remain two questions for me...

1. Which is ‘better’: Each firewall’s primary interface to the same switch or crisscross firewall-1, primary interface to switch a and firewall-2, primary interface to switch b? Pros and cons?

2. This is a bit of an issue for me generally as well as for this conversation:  if using a bonded interface and we lose either the primary or secondary interface of the bond (switch, cable or interface failure) then we won’t get an alert because while an interface has gone down and resilience is reduced) the bond remains up and so nothing is logged or alerted.

[_Val_]

CCP is running on all cluster attached interfaces, sync and production. If you only cut sync cable, Active goes to Active Attention and Standby goes into Down mode. Both will report sync interface down, as they should. Why is that confusing, may I ask?

[John_Fenoughty]

It doesn’t confuse me, it makes perfect sense - but when a person gets an email alert reporting problems with both firewalls I have seen a tendency for them to mis-interpret it.

It's because they see issues on both firewalls, and I think it is nicer for them to only see alerts from the one going down - it's just a cosmetic thing really. I certainly wouldn't want to change the behaviour of the firewalls, they are doing exactly the right thing but with a bit of judicious use of switches we can make it look 'cleaner' in the logs and alerts.

[_Val_]

HI,

you do want to see messages from both members in case of split brain situation. If they are fully isolated, each will go into Active Attention state.