Improvement Suggestion for Gaia System Backup – Make Restore Dependencies Visible

[israelfds95]

I would like to suggest an improvement to the Gaia System Backup functionality. I have frequently faced problems when restoring System Backups because some required software dependencies are not visible before the backup is collected. Normally, before collecting a System Backup, I check the Gaia version, ISO build and installed Jumbo Hotfix Take, so I can prepare the same or another appliance with exactly the same software versions before restoring the backup. However, even when doing this, the restore can fail because of dependencies that were not visible in Gaia.

For example:

I recently collected a System Backup from a 6600 running R82 Build 777 + JHF Take 122. Before collecting the backup, Gaia just showed Take 122 as the installed Jumbo Hotfix.

I then formatted the target 9100 with the same R82 Build 777, installed JHF Take 122, and tried to restore the System Backup. However, the restore failed because the backup required BUNDLE_R82_JUMBO_HF_MAIN#91.

To complete the restore, I had to uninstall Take 122, install Take 91, install Take 122 again, and then restore the System Backup. For one appliance this is already inconvenient (and is a waste of time), but when working with tens or hundreds of firewalls, discovering these dependencies through trial and error can result in a significant waste of time.

My suggestion

It would be very useful if Gaia run a verification on System Backup and displayed the complete restore requirements (like a restore path) for each System Backup before downloading or after downloaded. For example:

Restore Requirements
Gaia: R82 Build 777
Current JHF: Take 122
Required dependencies: Take 91 → Take 122

Gaia already seems to know this information, since the restore process can identify the missing Take 91. My suggestion is simply to make these dependencies visible beforehand, allowing administrators to prepare the appliance correctly before attempting the restore. This would make System Backup much more predictable and useful, especially in large-scale deployments.

Has anyone else experienced this behavior with System Backup?

[David_C1]

Yes, and agree it is dumb. Not sure why it should matter what JHFA takes were installed previously if you are trying to restore to a machine with the same final JHFA installed. Perhaps there is a technical reason...?

Dave

[Bob_Zimmerman]

It's to let you uninstall the jumbo. If you don't go through the same path, then uninstalling stuff on the box restored from backup would give different results from uninstalling the same stuff on the original box.

[Bob_Zimmerman]

I mostly don't take backups of firewalls.

If certain changes go wrong, I use a snapshot to get back to the state before the change.

If a drive fails, the other drive in the RAID should keep the member up and running.

If a cluster member fails, I rebuild it from another member of the cluster. I find it faster than trying to reproduce the same version-and-hotfix installation path you noted as a problem, especially if you've installed jumbos freqently.

If a whole cluster fails, a sizable chunk of the datacenter is likely damaged, and the network around it will need to be rebuilt. I find backups not useful for recovering from this, because I have to make enough other changes I'm effectively building a new cluster anyway.

[israelfds95]

I have been using System Backup mainly when replacing firewalls that will remain on the same software version and JHF Take. The advantage is that it restores important files such as certificates, fwkern.conf, trac_client configurations, and other system files, making the replacement process much faster.

I also frequently prepare firewalls manually using scripts based on the show configuration output, and both approaches have worked very well for me. However, System Backup has a great advantage: in my most recent replacement, I didn't even need to reestablish SIC. I simply connected the cables to the new appliance, and communication with the SMS was established automatically. Unfortunately, issues like the one described in this post are becoming more frequent, making what should be a straightforward restore process unnecessarily complicated.

[Alex-]

ElasticXL largely solves this issue though. If you need to swap an appliance in an existing cluster, connect it, add it to the cluster, start Insights, grab a coffee, watch the show.

[israelfds95]

ElasticXL is a great step forward in simplifying cluster deployment and member replacement, bringing some of the Maestro concepts and improving cluster check point. However, it is still relatively new and has limitations, especially for more complex environments, which is why I'm sticking with traditional ClusterXL for now.

In any case, System Backup serves a different purpose. ElasticXL is certainly a welcome improvement, but it's a different matter altogether and doesn't solve the problem I'm raising in this post, which is the lack of visibility into the complete JHF installation path and dependencies required to restore a System Backup. Both solutions have their own place. 🙂

[Bob_Zimmerman]

Unless you run into sk184626, which has bitten me almost every time I've tried to add a member. As long as you know to expect it, it's not too bad. It took a while to figure out what was going wrong the first time, though, and it's easy to forget.

[Bob_Zimmerman]

One false start, and restoring the backup takes twice as long, or even longer if you find you need to install several jumbos and individual hotfixes. I find it consistently far faster to just rebuild the firewall.

[israelfds95]

Yes, I understand your point, and I also rebuild firewalls manually when necessary. However, if System Backup is an available feature, I believe it's worth improving it, including the suggestions others have shared here.

My intention with this post isn't to discuss whether System Backup is better or worse than other recovery methods. I'm simply highlighting a specific limitation of System Backup and suggesting an improvement. So I don't think comparing it with alternative methods really addresses the point I'm raising. The discussion here is specifically about improving System Backup itself.

[Tom_Hinoue]

I just stumbled across this issue while working with R82.10.
I raised a TAC case and am now waiting for a hotfix.

Restore from Gaia system backup fails with hotfix information does not match
https://support.checkpoint.com/results/sk/sk1000023

In my case, I started with a fresh R82 installation on a 9100/Smart-1 appliance, then upgraded to R82.10 (no JHF), applied JHF Take 40, and then applied JHF Take 44. The backup was taken at this point.

I then tried to restore the backup to an appliance that followed this upgrade path:

R82 fresh installation → R82.10 (no JHF) → JHF Take 44

The restore failed.

The strange part is that the backup/restore also failed even when I followed the exact same upgrade path (same version/build) as the appliance where the backup was originally taken.

I’m honestly wondering why this backup/restore scenario wasn’t fully tested by RnD before the software was released.
I agree that backup/restore should be a much more straightforward process.

I now have to wait for the hotfix before I can restore the backup and continue investigating on my customers original issue.
I honestly didn’t expect to get stuck at this point...

[israelfds95]

In my experience, restoring a System Backup after uninstalling a problematic JHF has worked well. Over the last few months, I've used this approach several times to recover from JHFs that caused serious issues, such as SMS, CPM, or API failures, avoiding the need to roll back to VM snapshots.

However, the problem described in this post is becoming increasingly frequent and frustrating. Your case is even more concerning because the restore failed despite following the exact same upgrade path. I believe Check Point needs to improve the reliability and testing of this process, as System Backup is a critical recovery mechanism we should be able to trust. Thanks for sharing your experience, and I hope the hotfix resolves your issue!