Upgrades of 3100 Gateway from R77.30 to 80.40 or 81.20 just reboot back in to R77.30

[basinUnaltered]

In preparation to update our fleet of gateways I'm doing some labbing. We have a few stragglers still on R77.30, so in my lab I'm testing a 3100 gateway running that version. Using SmartConsole from my lab Management server running R81.20 I tried using Central Deployment but it would fail with, "The package is not valid for installation on the relevant Security Gateways".

Directly from the device's web UI I've tried downloading and installing the following CPUSE packages:
File Name: Blink_image_1.1_Check_Point_R81.20_T631_JHF_T41_SecurityGateway.tgz
File Name: Blink_image_1.1_Check_Point_R80.40_T294_JHF_T206_SecurityGateway.tgz

Both will download correctly and the verifier tool reports that Clean Install is allowed and Upgrade is allowed for both packages. When I run Upgrade they both appear to install without errors and the gateway reboots, however once it comes back up it is still running R77.30. I've tried this on two separate 3100 gateways and the problem occurs on both of them. I've confirmed that the new version doesn't show in the boot menu either.

The unit is a 3100 running R77.30 Build 165 with BUNDLE_R77_30_JUMBO_HF Take: 351 installed. It is running in 64-bit mode.

df -h:

Filesystem	Size	Used	Avail	Use%	Mounted on
/dev/mapper/vg_splat-lv_current	32G	5.0G	25G	17%	/
/dev/sda1	289M	50M	224M	19%	/boot
tmpfs	3.9G	0	3.9G	0%	/dev/shm
/dev/mapper/vg_splat-lv_log	59G	25G	31G	45%	/var/log

If at all possible I need to be able to do this as an upgrade rather than a clean install, any ideas what the issue might be?

[basinUnaltered]

Checking through CLI confirms that the upgrade should have been installed:

FWLAB101> show installer package 11
Display name: R81.20 Security Gateway + JHF T41 for Appliances and Open Servers
Description: Blink Image for R81.20 Take 631 including Take 41 of  R81.20 Jumbo Hotfix Accumulator - Security Gateway only
Size: 8.09 GB
Type: Major Version
Status: Available for Install
Requires reboot: Yes
Recommended: No
Contains: None
Contained-in: None
Downloaded on: Wed Mar 20 11:14:22 2024
Imported on: N/A
Installed on: Wed Mar 20 14:34:48 2024
Installation log: /opt/CPInstLog//install_Major_BLINK_R81_20_T631_JHF_T41_GW_2.log

[Expert@FWLAB101:0]#
1_GW_2.logLAB101:0]# cat /opt/CPInstLog//install_Major_BLINK_R81_20_T631_JHF_T41
[03/20/24 - 14:34:23]: ------ Installing: ------
[03/20/24 - 14:34:31]: ------ Validating Install: ------
[03/20/24 - 14:34:42]: Validating CRs for product SecurePlatform (API: SecurePlatform)
[03/20/24 - 14:34:42]: Validating CRs for product CPinfo (API: cpinfo)
[03/20/24 - 14:34:42]: Validating CRs for product CVPN (API: cvpn)
[03/20/24 - 14:34:42]: Validating CRs for product DIAG (API: diag)
[03/20/24 - 14:34:42]: Validating CRs for product FW1 (API: fw1)
[03/20/24 - 14:34:42]: Validating CRs for product PPACK (API: sim)
[03/20/24 - 14:34:42]: ------ Installing: ------
[03/20/24 - 14:34:42]: About to execute command: /bin/gtar -zxvf /var/log/CPda/metadata/CheckPoint#Major#All#6.0#5#4#BLINK_R81_20_T631_JHF_T41_GW/Blink_image_1.1_Check_Point_R81.20_T631_JHF_T41_SecurityGateway_METADATA.tgz -C /var/log/CPda/metadata/CheckPoint#Major#All#6.0#5#4#BLINK_R81_20_T631_JHF_T41_GW/tmp/
[03/20/24 - 14:34:48]: /bin/gtar -zxvf /var/log/CPda/metadata/CheckPoint#Major#All#6.0#5#4#BLINK_R81_20_T631_JHF_T41_GW/Blink_image_1.1_Check_Point_R81.20_T631_JHF_T41_SecurityGateway_METADATA.tgz -C /var/log/CPda/metadata/CheckPoint#Major#All#6.0#5#4#BLINK_R81_20_T631_JHF_T41_GW/tmp/ command summary:
Return code = 0
Output = BlinkInstaller
BlinkInstaller.config
BlinkInstaller.sha256
CheckPoint_Gaia_fd.tgz.sha256
DDR-00-00.i386.rpm
DDR-00-00.i386.rpm.sha256
DDRpackage.tar
DeploymentConditions.xml
Upgrade_Validator.xml
blades_updates/
conditions_set.json
conditions_set_is_ngm_upgrade.json
conditions_set_upgrade.json
conditions_set_warning_install.json
conditions_set_warning_upgrade.json
hf.config
installation_logic/
installation_logic/fd_wizard_gateway.sh
installation_logic/post.sh
installation_logic/answers.xml
installation_logic/post.sh.sha256
installation_logic/fd_wizard_gateway.sh.sha256
major.conf
manifest.xml
manifest.xml.sha256
user_updates/

[Expert@FWLAB101:0]# clish -c 'show version all'
Product version Check Point Gaia R77.30
OS build 10
OS kernel version 2.6.18-92cpx86_64
OS edition 64-bit

[the_rock]

If version shows R77.30, then nothing new was installed/upgraded. What do you see in web UI?

Andy

[basinUnaltered]

Web UI shows the same. That's the problem, the upgrade installations appear to run successfuly, logs show that they were installed, but the upgrade isn't actually happening and I've yet to find any indications as to what's going wrong.

[the_rock]

Since upgrade wizard tool appears to have been deprecated, I cant sadly check whats the version you can upgrade directly to when going from R77.30. You say it tells you can upgrade directly to R80.40?

Best,

Andy

[basinUnaltered]

Thanks Andy, documentation says that I should be able to go straight to R81.20 so long as the current OS is in 64-bit mode:



I did try upgrading to R80.40 as well but the same problem occurred. It's looking like ultimately I may need to get someone technical on-site to do a clean install, but it would be nice if we could avoid that.

[the_rock]

I know documentation says that, but to me, thats a huge jump. I mean, to put it in context, it would be similar to if say windows xp could be upgraded directly to windows 11.

Andy

[PhoneBoy]

Not sure you can upgrade a gateway directly to R81.20 as upgrades from R77.30 generally require an upgrade through R80.40.
Of course, you may want to do a fresh install from ISO to get the newer filesystem in R81.20 (xfs), which can improve performance in some situations.

Meanwhile, I'd start by debugging the deployment agent: https://support.checkpoint.com/results/sk/sk92449 
TAC may be necessary to assist with this also: https://help.checkpoint.com 

[basinUnaltered]

Thank you, I'll try debugging the deployment agent and reach out to TAC if needed.

Documentation does seem to support direct upgrades from R77.30 to R81.20 for security gateways as long as the gateway's OS edition is set to 64-bit mode. I did try using the equivalent R80.40 image as well and the same problem occurs with that.

[the_rock]

I have a suggestion. Can you try say go to R80.10 first and see if that works?

Andy

[basinUnaltered]

Sure, I'll give that a try!

[the_rock]

Lets hope that works...fingers crossed.

Andy

[emmap]

Have you satisfied the other requirements in the documentation? 

Before you start the upgrade, you must make sure the GaiaClosed OS edition is 64-bit:

Get the current Gaia OS edition with this Gaia ClishClosed command:

show version all

If the Gaia OS edition is "32-bit", run these Gaia Clish commands:

set edition 64-bit

save config

reboot

I'd also recommend trying the main upgrade package instead of the blink package, after ensuring that the deployment agent is up to date.

[basinUnaltered]

The 64 bit requirement is met; I'll try the main package as you and _Val_ have mentioned!

[_Val_]

Don't use the BLINK image to upgrade, try the regular package. BLINK just copies over the file system on the pre-partitioned HDD, and R77.30 partitioning is different from what BLINK is expecting, so the process fails, and the system reverts to the original image.

[RamGuy239]

^ This.

When moving from R77.30, I would opt for regular CPUSE, not BLINK. Blink gives you the benefit of getting the latest recommended JHF as part of the process. But considering Blink was introduced with R80.XX, I wouldn't trust it to work when moving from R77.30.

As others have already mentioned, Check Point TAC, Check Point PS, and my local Check Point offices have always told me to upgrade from R77.XX -> R80.40 -> R81.XX. To get the full benefit of R80.40+, you must completely re-install using ISO/USB to get the XFS file system. On R81.20 there is also the 4K disk alignment, this also requires the hard drive to get formated.

This is more important in management installations. I would not move to R81.20 on the management without getting XFS and 4K disk alignment. It's still beneficial to do so on gateways. But it won't matter as much as you don't have nearly the same amount of writing to disk as you have on management installations.

[basinUnaltered]

I was hoping to put off the clean install until the R77.30 sites were due for a hardware refresh but if I want to get them on R81.20 it's looking like they're getting it early.

I was unable to upgrade using non-blink packages for any R80 version, 80.10, 80.20, 80.30, and 80.40 would all fail in the validation stage and the blink packages would indicate success but then reboot back in to R77.30.

Thank you all for your help!

[the_rock]

So do you plan to do fresh install on site?

Best,

Andy

[basinUnaltered]

Hi Andy,

Unless I can find a way to successfully upgrade hands-off I'm going to need somebody to be there, either to fresh install manually or to swap in an already up-to-date unit that I ship. Not the end of the world but it's an additional complication to schedule around.

[the_rock]

Understood. So you said even any flavor of R80.xx does not seem to work for the upgrade, always rolls back to R77.30 on its own?

Andy

[basinUnaltered]

That's correct. It gives no indication of a failure either; in the web GUI all dialogue boxes indicate that the upgrades are successful and then it reboots as normal, but then after reboot it comes back still in R77.30. I hadn't recorded exactly which packages this part happens with but sometimes it will even indicate that the package was installed while still running R77.30.

[PhoneBoy]

Before the upgrade starts, a snapshot is taken.
If there is a problem booting after the upgrade, the system will revert to the snapshot.
I would recommend actively reading the console output during the reboot AFTER the upgrade claims to have completed safely.
Perhaps there is a hint there?

Otherwise, TAC is probably your best bet: https://help.checkpoint.com

[Bob_Zimmerman]

What did the validation complain about?

[basinUnaltered]

Clean Install verifies successfully, but upgrade fails with, "could not read configuration file (major.conf)". This occurs with any of the non-blink packages. In CLISH I've run installer agent update which tells me that the deployment agent is up to date, and I've followed the instructions in sk102328 but the issue persists.