R77.30 to R80.10 Mgmt HA upgrade step-by-step

[Usman_Shaikh]

Hi Guys..My first post on the forum so please bear with me

Goal: Upgrading our R77.30 Mgmt HA deployment (Virtual Machines) to R80.10 (virtual machines) HA..

Reading through various CP docs and the forums, I have prepared a step-by-step procedure and I shall be grateful if someone can validate this for me.

Hopefull this would be of help to others as well

Procedure

Primary Server

1. Run pre_upgrade_verifier tool on source R77.30 Primary (Active) Mgmt server and resolve any errors/warnings
2. Provision two new VMs using recommendations specified in R80.10 release notes (pg 16)
3. Perform clean install of an R80.10 SMS on the first VM using iso and assign temporary IP address but same hostname as R77.30 source server
4. Export the database using migrate_export from source (R77.30) Primary/Active SMS and note down the MD5 hash
5. Transfer the exported DB to target (R80.10) SMS server and verify MD5 hash. (This is why we need temporary IP in Step 2)
   
6. Disconnect target R80.10 SMS server from the network
7. From VMWare console, change the IP address for R80.10 SMS to be the same as R77.30
8. Run migrate_import to import the DB on the R80.10 SMS
9. Disconnect source (R77.30) SMS HA pair from the network. At this point firewalls are logging locally
10. Bring new R80.10 SMS online
11. Ensure SIC is maintained (since hosntame and IP are the same, SIC should not be affected.. right?) and firewalls are logging to the New R80.10 server
12. Install a policy to validate migration

Management HA

1.  Perform a clean install of R80.10 SMS on new Secondary SMS server with same IP as current (R77.30) standby SMS server
2. Establish SIC with new R80.10 Secondary SMS server
3. Perform Synchronisation from Primary to Secondary server

Rollback

1.  Disconnect the new SMS and bring R77.30 SMS servers online and test SIC, logging and policy installation

[G_W_Albrecht]

Thank you for this extensive posting ! But i have an additional question: If using Smart-1 appliances for SMS, Management HA with a second Smart-1 maybe a good option. On the other hand, out of my experience Management HA will not help so very much if the Primary SMS performs an unrecoverable crash with no available backup (read sk114933 How to promote the Secondary Management server to become the Primary server for the painfull steps needed to do so) - and especially Full Management HA is a kind of deployment that sometimes seem to ask for troubles.

If the SMS is already virtualized, i would rather suggest to use the VM snapshot feature instead of Management HA. But what is your opinion and the communities opinion ?

[Vladimir]

Having management HA in virtual environments may still be warranted if you have either multi-site environment with 0 downtime requirements or are running scripts from secondary.

That being said, this is my preferred approach for VM SMS:

"For one company on a budget, in order to provide redundancy for their single Management Server, I’ve implemented a virtual Gaia appliance with multiple interfaces. Each of the interfaces was assigned an IP from a network in each location. The routable loopback address was used for management.

I then had that address advertised via OSPF to the internal routing area via whatever interface was connected at the time. The VM was replicated to other sites and scheduled backups were copied there as well.

During primary sites’ failure, client could power-up the VM, restore latest backup, connect to the same IP and continue managing their gateways."

[G_W_Albrecht]

Very true! And thank you for the multi-site environment thoughts.

[Usman_Shaikh]

Thankyou both for your prompt responses..much appreciated

I understand the concerns regarding HA and promotion of secondary to primary; however in this case we have to go for a like-for-like deployment so the options are limited

.....coming back to the original question, does the plan look OK to you guys ?

[G_W_Albrecht]

Hi,

looks fine to me.

[Usman_Shaikh]

Many Thanks ..

[Vladimir]

I personally, dislike changing IPs of the management servers mid-migration, but this early in the process it may not be detrimental.

You can use secondary NIC on VM with different IP for file transfer and your primary IP on a dead-end network.

Once file(s) are transferred, shutdown VM and remove secondary NIC.

Additionally, while SIC will work, you may still have to reinitialize it, as I have seen the situations when after successful migration, you are getting error messages:

"Issue Description: Moving Management HA to new appliances. Gaia configs replicated, IPs and hostnames are identical. Migrate Import/Export successful. SIC from target primary server to all components of infrastructure tested and confirmed working. After policy package push to the gateway at DR site, receiving error: ^Internal SSL authentication SSL error [Got alert from peer that the certificate expired]^
similar to one described in SK102975."

So the solution ended-up being SIC reset, even though the communication between SMS and the gateays was working fine:

...it appears that you just need to reset the SIC so that the certification gets redone. To do this without reseting the firewall follow the steps in sk86521. I also have included the steps below.

1. On the Security Gateway(S), run these commands:
a. [Expert@HostName]# cp_conf sic init New_Activation_Key norestart
b. [Expert@HostName]# cpwd_admin stop -name CPD -path "$CPDIR/bin/cpd_admin" -command "cpd_admin stop"
c. [Expert@HostName]# cpwd_admin start -name CPD -path "$CPDIR/bin/cpd" -command "cpd"
2. In SmartDashboard:
A. Click on the Security Gateway object.
B. Click on 'Communication'. 
C. Click 'Reset' and confirm.
D. Enter the New_Activation_Key (that was used in the 'cp_conf sic init ...' command on Security Gateway).
E. Click on 'Initialize'.
F. Install policy, if needed.

Please Note:
• Make sure you are resetting SIC to the same Management Server IP address. Using this procedure, the firewall still has the last installed policy.
• If the user has a "Stealth Rule" or a "Cleanup Rule", the current policy may only allow for communication between the Gateway and IP address of Management Server.
• If changing the IP address of the Management Server, this traffic will be dropped on the Stealth or Cleanup Rule.

[Usman_Shaikh]

You're a legend.. I will give it a go soon and will come back to you guys with the results

Thanks

Usman