replacing appliances in a SecurityGroup with newer hardware

[Wolfgang]

Following sk162373 - Quantum Maestro supported combinations of mix-and-match appliances it's possible to mix different appliances for migration.

"Quantum Maestro supports all other combinations for migration purposes. For example, suppose you would like to replace a 6500 Security Appliance model with a 16200 Security Appliance model: In this scenario, you can include the two Security Appliance models in the same Security Group, allow it to clone the configuration to the 16200 Security Appliance model, remove the 6500 Security Appliance model, and continue to work with the 16200 Security Appliance model only. This migration procedure should include CXL / SND cores configuration and adjustment on new Security Appliances models, which requires downtime because of the reboot. As a result, you must do the migration during a maintenance window."

But now we are on the way doing such migration and have some problems. Often we got an advice from TAC not
adding the appliances to the old SG, creating a new SG with the new appliances should be the way. But this will
result in a needed longer maintenance schedule and much more work to do.
And this will be contrary to the scalability of Maestro.

As an example we want to replace 6600 with 9300 and 16600 with 9700.

Any experience with that or any suggestions ?

[AkosBakos · Accepted solution]

Hi @Wolfgang 

We do have experience with the change (CP6500 -> CP9300):

A short summary of the chnage: 

• removed the 2 old SGMs from site A
• put the new SGM to site A
• Came up in down state (it took ~10 minutes)
• #cphaprob synscat showed policy installation error
  • realized that the original lic expired - we knew that therefore created EVAL earlier-  and the EVAL license has just expired - (that1s the fun fact) 
    • we assume that because of the lack of valid lic (in this situation only InitialPolicy was availabe on the SGM) the SGM couldn't pull from its own lic from the usercenter (the initialPolicy didn't allower the traffic to usercenter)
• We created a new EVAL for the new SGM
• install the lic with cplic put
• rebooted the SGM, then it came up in Active state
• install the necessary jumbo take 
• than manual failover to the new SGM

The arrange of the other three SGMs into Security Group were easy after this experience.

Cleaunup: we remoed all EVAL licenses from the SGM-s g_cplic del <signature>

If you have question, and if you want, contact me in private, or drop an update here.

Here is the tread: https://community.checkpoint.com/t5/General-Topics/MAESTRO-SGM-change/m-p/233169#M38960

Akos

[alexgnunez2]

Definitavemente tuve un problema para levantar el cluster de Maestro , el comportamiento de los miembros del cluster (show cluster status) quedaba en LOST, despues de instalar la licencia de evaluacion el cluster Maestro levanto (SGMs) .

[AkosBakos]

@alexgnunez2 

Me alegra poder ayudarte, si tienes alguna pregunta no dudes en escribirme.
(traducido por el traductor de google)

Akos

[emmap]

I've done it recently, I found it easier to add a new one, patch it up to required JHF level, remove all the old ones, then add the remaining new ones with auto-clone enabled, then check/enable dynamic balancing as required. 

[Wolfgang]

@emmap that's what we expected. We can add the 9700 to the SGM with 16600 but nothing happened and the appliance does not reboot to join the SG. SGM is shown as member of the SG, not seen in Cluster state, no error in messages. If we remove the 9700 from the SGM the mini reset is run, the appliance reboots and is back online again as unattached.

[emmap]

I've heard of that happening, you can try rebooting the SMO SGM while it's waiting to add, I've heard of that helping, or even I've heard it just starts after 2 hours. I had it happen to me but it was the second new SGM, so I went to what I said above and removed all the old ones before auto-cloning it into the group, which worked fine. The slient_install log file showed nothing useful unfortunately, I don't know what causes the issue.

[CPengine]

I agree with @emmap's method, especially if you are moving from 6000 series to the 9000 series. We have seen success with the steps below:

Maestro Replacement Procedure: 

• Unbox new firewalls and use a USB to reinstall the base image or reset to factory default 

• Remove the SMO from the security group. Wait a few minutes, unplug, and remove. 

• Connect the new firewall, and power on. Give about 8-10 minutes to show up in MHO 

• Ensure that "auto-clone" is off. 

• Add the new firewall to the security group 

• Wait 8-10 minutes for the firewall to join the security group and return to SMO status 

• Once firewall is joined to the security group, upgrade the firewall to the latest jumbo hotfix 

• (NOTE: Based on traffic utilization, you can choose to remove the other members one at a time or all at once) 

• Remove remaining SGMs from the security group. Wait a few minutes, unplug, and remove. 

• Turn "auto-clone" on the SMO 

• Connect the new SGMs, and power on. Give about 8-10 minutes to show up in MHO 

• Add the new SGMs to the security group 

• Wait 20-25 minutes for the SGMs to join the security group and auto-clone from the SMO  
• (run# asg monitor)  

• (NOTE: Console into the SMG to watch progress during auto-cloning) 

• Once SGMs are joined to the security group, check to make sure they were auto-cloned and upgraded to the latest jumbo hotfix (cpinfo –y all) 

• Turn “auto-clone” off

• Push policy on the security group and monitor traffic. 

• Replacement complete 

[RickLin]

I have a question , how to let new 9000 appliance get the lowest ID be the SMO role when the Security Group still have old 6000 appliances?

Could CheckPoint teach us what are the right steps?

[AkosBakos]

Hi,

if you remove one SGM from the SG, the newly attached will get the lowest ID first.

so if you remove 1_1, the new will be 1_1.

Last time we unattached all SGMs from the site, then attached the Quantum Force appliances. 
works fine

akos

[RickLin]

Unattach old appliance then attach new appliance then publish (once) or Unattach then publish, wait a moment, then attach new appliance and publish again(twice)?

[AkosBakos]

Hi,

I would do one-by-one. I would start the next change when the newly added SGM is in active state.

"Last time we unattached all SGMs from the site, then attached the Quantum Force appliances. 
works fine" -> standby site -> removed all SGMs -> attached one SGM -> waited until came up to Active -> install the required jumbo hotfix -> failover to the new Quantum Force 9300 -> short smoke test -> then remove all old SGMs -> only the 9300 is in duty -> attach the remaining 9300s to the SG.

If you have more questions you can contact me directly (private message)

Akos

[CPengine]

@AkosBakos  is correct. The MHO knows who the SMO is by where you have the DAC cables plugged in. So removing the 1_1 and replacing it, the new firewall will come up as 1_1.

Whatever you do, please do not try to auto clone a 9000 series from a 6000 series. It will not work, as the OS version is slightly different. I would imagine that you can't auto clone a 9000 series with any other model. 9000 series have there own SK page in which you can download the OS version.

Just let the 9000 series join the security group, upgrade it to the latest JHF, then you can auto clone the rest of the new replacement firewalls from the new 9000 series SMO.

Hope this helps.

[AkosBakos]

Yes, good summary. Only the configuration "clone" works between 6000 series and 9000 series, so you have to install jumbo manually after attaching the SG ne SGM.

Akos