Elastic XL and sync interface

[JMarkus]

Hello Checkmates,

as far as I have seen the sync interface to be used when you setup a elastic XL cluster is only the one called "sync".

What if I dont want to use this one ? A simple reason for this: I need a fiber interface and not a copper.

According to some lab I did, it does not work. You can setup your first member SMO, change the bond 1024 with the sync interface you want to use. But then the second member freshly imaged, without first time wizard done, will never be visible (the interface I want to use for sync are connected together of course).

Ok, you can first setup your cluster with the dedicated "sync" interface, and change it after and it is working. But this is a workaround, not a solution. What is you want to add some additional members later ? Or do a RMA ? 

Regards

[ZaferGr]

Hello,

It was easy to perform the operation you mentioned in Maestro or a regular cluster.

For ElasticXL, it might make more sense to set up bonding.
The sync IP address is defined by default. “ElasticXL Cluster automatically configures the IP address of the sync network to 192.0.2.0/24.”

I also recommend reviewing this support article:
https://support.checkpoint.com/results/sk/sk183513

[JMarkus]

Your sk does not help. I am talking of same 9xxx appliances in R82.10.

Sync interface is by default a bond, the bond group 1024 where it puts the physical interface named "sync" (renamed eth1-Sync).

Once you have setup your cluster with the sync interface, you can modify the bond 1024 and put what you want, change for a fiber interface, change the mode from HA to LACP. It works, I tested.

My issue is that I want a new member (new or RMA) uses directly my sync setup (not the interface named sync).

The R82.10 Scalable Platforms Administration Guide says this:

"The "Sync" ports of all ElasticXL Cluster Members in the same ElasticXL Cluster must connect to the same Layer 2 broadcast domain (a dedicated Layer 2 switch, or a dedicated VLAN)."

Do we really have this limitation that we can only use the sync port ? What if I dont have copper ports on my switches ? 

Also with ElasticXL you have the "site" notion, it means distant sites. When you have distant sites it is connected by fibers, "always".

[ZaferGr]

Hello Markus

I couldn't find an official statement addressing this request. Perhaps there are some gaps in this area because it's a new technology.

If this request is to be implemented, it will likely be resolved with a patch file.

Let's keep each other updated

[Bob_Zimmerman]

To cleanly provision new members, you must use the interface named Sync or Sync1 (on boxes like the 19000 line which have more than one). The easiest way to ensure this is to have a mostly-copper switch used just for provisioning with the copper ports to the members of a given cluster on the same VLAN as that cluster's real sync interfaces. Connect the interface named Sync to that switch, then when members clone the Lightshot from the pivot and revert to it, the real members of bond 1024 will be added to it and set up. Somewhat inconvenient, but you need copper ports for the LOM anyway, so this is just a few more. You will almost certainly need console access to the new member to deal with sk184626, (the default Lightshot LV doesn't expand quickly enough, causing joins to fail) so it's not as zero-touch as originally advertised.

If this definitely isn't an option, you won't be able to provision new members seamlessly, but you can use the process for open servers in sk183513 to manually manipulate the interface naming to make any interface 'eth1-Sync' for the purposes of joining the cluster. It takes some manual futzing, but it's not especially complex, particularly since you probably need the console access for the Lightshot LV thing anyway.

You said "the interface I want to use for sync are connected together of course". Do you mean via a direct fiber connection with no switch between them? If so, that's a bad idea. The supported topologies for the synchronization network all have a switch in them.

[SLA1]

Thanks for your answer Bob.

I shouldn't have added this remark, it was confusing. I just wanted to say that the interfaces I wanted to use
are connected together, via a switch yes.

For the fact that we have copper ports anyway for LOM, well, yes, but it could de different dedicated out of band management
switches that you dont want to use for your sync, it is often the case actually.

For what is described in sk183513 for open server with renaming the interface it can be investigated, but I dont know if you can rename appliance
interfaces. Also this is not the same scenario. In the open server case explained you define the interfaces admin, sync and it stays. Here it
would be to pretend to my appliance that eth1-Sync is my fiber interface, and then it would be overridden when doing the provisioning...
It can be tested, but I doubt it will work. Or maybe you meant renaming my fiber interface in eth1-Sync an all my appliances, like that it stays
and nothing is overridden at provisioning. It should be specified in a sk if this is the solution.

But frankly I dont see the point of this zero touch provisioning. Nobody is adding new node in his cluster or doing RMA every day ?
What the problem to power on the box, run the first time wizard, select ElasticXL member, sync interface is xxxx, and that's it.
Or the equivalent editing a config file. You anyway have to power one the box check the version and possibly reimage it in the correct version.

@responsible of this community site, it would be good to add an ElasticXL TAG

[SLA1]

Thanks for your answer Bob.

I shouldn't have added this remark, it was confusing. I just wanted to say that the interfaces I wanted to use
are connected together, via a switch yes.

For the fact that we have copper ports anyway for LOM, well, yes, but it could de different dedicated out of band management
switches that you dont want to use for your sync, it is often the case actually.

For what is described in sk183513 for open server with renaming the interface it can be investigated, but I dont know if you can rename appliance
interfaces. Also this is not the same scenario. In the open server case explained you define the interfaces admin, sync and it stays. Here it
would be to pretend to my appliance that eth1-Sync is my fiber interface, and then it would be overridden when doing the provisioning...
It can be tested, but I doubt it will work. Or maybe you meant renaming my fiber interface in eth1-Sync an all my appliances, like that it stays
and nothing is overridden at provisioning. It should be specified in a sk if this is the solution.

But frankly I dont see the point of this zero touch provisioning. Nobody is adding new node in his cluster or doing RMA every day ?
What the problem to power on the box, run the first time wizard, select ElasticXL member, sync interface is xxxx, and that's it.
Or the equivalent editing a config file. You anyway have to power one the box check the version and possibly reimage it in the correct version.

[Bob_Zimmerman]

Check Point's "appliances" (except the 3900 and Spark lines) are just open servers with nonstandard PCIe slots, mediocre LOM, and the ports on the wrong end. At a software level, they're exactly the same, so the same process would work to rename some random interface to eth1-Sync. Once the new member clones the Lightshot from the pivot and reverts to it, it'll have the normal interface naming (the Lightshot includes the udev rules) and the normal sync bond membership.

The point of provisioning being low-touch is there's less opportunity to forget something. As you say, you don't add a new member every day, so when you do, you're almost guaranteed to have forgotten any unusual stuff you do to add a member. When you RMA a box (or want to swap hardware to stay ahead of EoL), you should ideally be able to just plug the replacement in, approve the replacement in the cluster, and that's all. The Lightshot LV limitation means you won't be able to get quite to that ideal, so an additional step may be acceptable.

[JMarkus]

"When you RMA a box (or want to swap hardware to stay ahead of EoL), you should ideally be able to just plug the replacement in, approve the replacement in the cluster, and that's all."

I dont agree with this. Nobody is asking for this. You have anyway to check the version and reimage if needed. I am sure nobody will consider to be a problem to have to run the 1st time wizard, select "ElasticXL member of and existing cluster", "sync interface are xxx", and then you plug it.

I will do some test with the interfaces renaming when I have time.

In the meantime I discovered another issue with the sync interface. My understanding of the management interface magg1 is that it was out of band. But it is not, it is inband unfortunately. So as I wanted out of band I setup MDPS (Management Data Plane Separation). But when you do this the system consider only the physical "Sync" interface and not the bond1024 sync interface. As a result if you dont use the physical "sync" interface you loose your cluster. I have a TAC case currently opened for this, how to put other snyc interfaces in the mgmt plane

Again the renaming could help here, not the one where you change the name on the second member for provisioning and it is overwritten. You would need to have the sync interface of all your node changed. But even if working it would need to be in an SK or somewhere in order to be sure it is Checkpoint supported in a production environement. 

Another issue is that you normally always double the sync interfaces (I do). So even if you can make one interface of your choice the eth1-sync interface, there is still something more needed to add you second interface.

I find there are many design flaws with the sync interface.

However

[Bob_Zimmerman]

---

@JMarkus wrote:

"When you RMA a box (or want to swap hardware to stay ahead of EoL), you should ideally be able to just plug the replacement in, approve the replacement in the cluster, and that's all."

I dont agree with this. Nobody is asking for this.

---

Lots of big customers are absolutely asking for this. Inconsistencies when rebuilding a box after RMA or hardware swap are a major source of weird outages and performance issues. Avoiding these is the big selling point for ElasticXL in the first place.

[Steffen_Appel]

Yes you can shuffle the interfaces in /etc/udev/rules.d/00-OS-XX.rules