Central Deployment Question

[RemoteUser]

Hi everyone,

While doing a lab training to install a Jumbo Hotfix on a ClusterXL (HA) via SmartConsole Central Deployment (Standby member only, with "Download package from" set to Automatic), I noticed that the deployment skipped the Download phase and went straight to the Installation phase.

My assumption is that the package had already been downloaded from the Cloud to the Management Server's Package Repository and was then transferred directly to the gateway.

So, is my understanding correct? Once the package has already been downloaded to the Management Server, does skip the Download phase and proceed directly with the installation on the gateway?

Can anyone confirm this behavior?

Thanks in advance!

[Bob_Zimmerman]

When the management has a package and the firewall doesn't, there's a step where the management copies it to the firewall. From your description, it's more likely the firewall had already downloaded the package on its own.

It's also possible somebody had run a verification with the "Download/Deliver package to Security Gateways as part of verification" box checked at some point in the past. I do that with all upgrades and updates to make the actual upgrade window more predictable.

[RemoteUser]

The installation on all cluster members is a great feature, but is there any way to pause the process or take some time to verify that everything is OK before the failover happens?

Or is there anything planned for the future to support this kind of control?

[Duane_Toler]

No, that's the purpose of CDT (or using the SmartConsole method).  It runs the whole process end-to-end so it ensures both gateways of the cluster are updated equally.

[emmap]

When you have the install window open it should offer an option to do one member only. I don't have it open in front of me to confirm the exact wording, but there's something there for that.

[RemoteUser]

I understand that there is an option to install the Jumbo Hotfix on a single gateway, but my question is different.

When installing a Jumbo Hotfix automatically on both cluster members, the failover and installation process starts immediately. There doesn't seem to be an option to pause and verify the health of the upgraded gateway—for example, checking that all daemons are running correctly, traffic is flowing as expected, and the gateway is fully operational—before triggering the failover.

Is this something that might be considered for implementation in a future release?

In my opinion, the automated cluster update feature is great, but it is missing a health verification step before the failover

[emmap]

As a way to do these checks manually? You do it using the 'Install on non-active members' option, then when it's finished you do all the checks you want to do, do the fail over (unless you selected the option to fail over automatically) and kick off the other one when you're happy with the first.

When we're doing the full thing automatically the first gateway has to get back to a Standby state before the second gateway is upgraded - and it won't get to a Standby state until all monitored processes are running and happy, connections are sync'd and the gateway is otherwise operational. 

Note that the 'install on non-active members only' option isn't available for VSX clusters. I don't know what the roadmap is there.

[RemoteUser]

Thanks for the clarification.

I did some testing in my lab to better understand the behavior. On the standby member, I manually stopped the CPD daemon using:

cpwd_admin stop -name CPD -path "$CPDIR/bin/cpd_admin" -command "cpd_admin stop"

Even with the CPD process stopped, the cluster member still appeared as Standby and did not seem to be considered unhealthy by the cluster.

This made me wonder: during an automated upgrade, is the health validation based only on the ClusterXL state (Active/Standby) and monitored cluster mechanisms, or does it also verify that all critical Check Point daemons are actually running and healthy before proceeding with the failover?

That's the part I'm trying to better understand.

[emmap]

It relys on cluster state only.

CPD doesn't have a pnote associated with it, just the CPWD watchdog to monitor it and restart it as required, so it restarting won't take the cluster member down. It not starting after the reboot however will still have an effect on the cluster member state though as it handles the policy install that is required before the cluster member can be ready. Once the cluster member is up and running though, CPD doesn't have any immediate effect on the operational state, so there's no direct pnote for it. So it's still a reliable metric to look at cluster state after the patch install and reboot.