R81 Upgrade Tips

[superd]

Hi all, 

I have a number of R81 upgrades coming up over coming months for various clients, mainly from R80.x.

Aside from the upgrade guide (which I will read), Im looking for some tech tips, or best practices for increasing the chances of a smooth upgrade i.e. pre / post checks, HA best practices etc. stuff that may not neccassarily be in the upgrade guides.

Ive had a number of issues with upgrades between R80 versions (some documented here, and still ongoing), so I really want to try gather as much prep as I can from the experts here.

Also, if theres any known issues / gotchas when going from R80.x to R81, that would be great. 

(Im hoping if we can get some good responses here, it will also be a helpful resource for other Checkpoint customers moving to R81).

Thanks,

D

[the_rock]

I did bunch of those and I find it always goes smoothly from web UI. You can also do it via smart dashboard, but its been a while since I did that. 

[G_W_Albrecht]

I would upgrade the SMS using GAiA WebGUI and then the GWs using Smart Dashboard.

[superd]

Thanks. Can i get your rational behind this? I thought using blink directly on GW GUI would be best method.

[G_W_Albrecht]

If you go for a fresh install using Blink, yes. In SmartDashboard, you can download the upgrade package once and then locally install it on several GWs.

[G_W_Albrecht]

I would suggest R81.10 instead of R81.

[the_rock]

TOTALLY!!!

[superd]

For sure, yes. I assume R81.10 SMS can happily co-exist and manage R80 GWs?

[Piet_vd_Maas]

 I don't know how many gateways you're talking about and how many customization you have, but it can be a good moment to do a clean install and review your config why system variables are set etc.

[superd]

Thanks Piet, noted. And its generally a cluster and SMS.

[Ruan_Kotze]

Configure and test your Lights Out Management before you start (especially if you are not doing the upgrades on-site). 

Example: I recently ran into a a very annoying bug at a couple of deployments where if you had IOC feeds configured the gateway would reboot with the initial policy and having access to the console allowed me to do a "fw fetch".  Allowed me to finish the upgrades on schedule and saved me from a several hours round-trip.

[spottex]

I concur on checking lights-out first. Out of 5 clusters on 69000 appliance only one cluster was working. The other 4 clusters (2 nodes each) connectivity failed. Reboot of 3 clusters fixed connectivity. The last one needed a datacentre visit to pull the power cable to reset both nodes.

[_Daniel_]

Keep a close eye on the important files -in case you'd some custom config, mainly trac_client_1.ttm, etc. these will be over-written so make sure to copy them prior to upgrade. It's clearly mentioned in the upgrade guide.

Also I noticed on a VSX cluster the MAC address for the bond has changed post upgrade, impacting the proxy arp config, we ended up updating the local.arp.

Had a problem with one VS post upgrade member 1, I was able to push policies to every single VS but was complaining about one VS not having SIC with it! While waiting over 40 minutes for a TAC engineer to join the call, I rebooted the appliance which fixed the issue.

[spottex]

Oh yes we had the VS issue as well. One of the cluster members SIC was showing as initiating.
reboot didn't help us. sk97833 did though
Pulling the cert to the gateway. I think I needed to delete the current initiating cert but can't remember.

[Expert@HostName]# vsenv <relevant VSID>
[Expert@HostName]#cp_pull_cert -d -h <MGMT_IP> -n <VSX_Name_VS Name>

[superd · Accepted solution]

If possible - reboot server to give a clean system going into upgrade - (a personal thing).

Backup OS level and VM level (if applicable)

Check system for manually edited files, and copy of manually.
• SMS check /conf/user.def etc
• GW check trac_client_1.ttm, etc.
• On newly installed version, edit the mentioned files, do not copy in old one.

Check disk space from cli with df -k, and remove large files if disk space is low

• find / -type f -size +100000 -exec ls -lh {} \; 2> /dev/null | awk '{ print $NF ": " $5 }' | sort -nk 2,2

Remove old snapshots

Upgrade DA agent

Install latest upgrade tools

Use blink preferably, contains latest HFA, less reboots.

In CPUSE Right Click blink image to be installed and Verify

Start the upgrade, and monitor Blink log - /var/log/blink/<filename>/main_log.elg

[superd]

I ran into a recent issue with an R81 upgrade, where the user.def file had changed between R80.40 and R81. It caused some major issues with VPN users. It had to be manaully copied into R81.

Just an FYI in case this benefits someone else.

https://sc1.checkpoint.com/documents/R81/WebAdminGuides/EN/CP_R81_SecurityManagement_AdminGuide/Topi...

I guess this was the nature of my initial post. Could there be any other such .def or .conf files which require consideration between versions?

[the_rock]

Very good point, something to keep in mind, for sure!

[Abi]

After the upgrade and the first policy is pushed to upgraded gateways, you might not be able to login into the SmartConsole .If this occurs, check your implied rules.

[the_rock]

I believe thats more related to CPM process sometimes taking a bit of time, specially after upgrade and reboot. You can simply check it by running watch $FWDIR/scripts/./cpm_status.sh from expert mode and when it shows up and ready, that means console will work.

Andy

[superd]

Guys, Ive updated the solution here with some information which Im hoping might help with CP upgrades.. which is based on my recent upgrade experiences and challenges.