Checkpoint 4200 HA (stacked) vR77.20 is bricked with SIC issues after Gaia HOTFIX_R77_20

[not_a_net_admin]

Hello Checkmates!

Disclaimer - I'm not a network guy - not to the least - I'm only a GNU/Linux sysadm helping a friend. He has one quite old Datacenter with 2 CP4200 + Gaia VM running without licenses nor support contract.

After the CVE-2024-24919 he reached out to me to see what I could do to help him. First thing I did was to backup the configuration through SmartConsole, and I thought I was safe and that I could easily restore the situation whatever happens. Foolish.

After applying the GAIA hotfix mentioned in the subject, doing some changes (disabling Remote VPN, Mobile access, etc), verifying no local users were in use, changing the password of the AD service account system user used to sync the database, all was good until I could not apply changes any longer, due to the error: 

Installation failed. Reason: Peer SIC Certificate has been revoked try to reset SIC on the peer and re-establish the trust.   ( message from member GW-FW01 )

The above message repeats for the stacked twin with the message being identical other than mentioning the stacked GW-FW02.

So I ran cp_config to try to re-establish trust, but it asks for a Key issued at installation time. And we don't have that Key around anymore, it's been too long and too many different network admins later, there's no one with context to help.

So I'm trying to figure out if it's possible to un-brick his setup, to bring back Remote VPN at least after all. Any guides I should follow? Is there anything that could be done to this old pair of CP4200 on R77_20?

Best regards and keep filtering out the bad actors!

[PhoneBoy · Accepted solution]

You're referring to SmartConsole...Is your management R8x?
Otherwise, you're doing this in SmartDashboard.
Please clarify.

Actually, what you're setting with cpconfig is the initial key used by the management when SIC is (re)established.
In the relevant gateway objects, you reset SIC and supply the same password.

[not_a_net_admin]

Hello PhoneBoy, 

After resetting the SIC and re-establishing trust I was able to restore a backup from clish. 

And mostly everything is restored, but I'm now unable to open SmartDashboard. I've tried resetting the GUI clients and Admin users with cpconfig, but no luck so far. 

I'm now reading https://community.checkpoint.com/t5/Management/Cannot-connect-SmartDashboard-because-of-expired-cert... to see if I can manage to revoke the old Certificates to be able to login, or I was thinking that I could try to generate a certificate to login to SmartDashboard without user/pass, but I'm yet to find the relevant documentation regarding that. 

Thank you very much for pointing me to the general direction of my first issue!

[PhoneBoy]

The ICA is likely expired and needs to be regenerated with the command fwm sic_reset.
This will cause an outage with all managed gateways and will require SIC to be re-established.

[not_a_net_admin]

Got myself in a Chicken vs Egg issue, I can't get SmartDashboard to open because it has trust issues with Gaia, and Gaia won't allow me to reset SIC because of some IKE issue:

```

-sh-3.1# fwm sic_reset
***************** Warning: ****************
This operation will reset the Secure Internal Communication (SIC).
The internal Certificate Authority will be destroyed and ALL remote Check Point Components,
including VPN and Endpoint clients, will not be able to communicate.

In case of Endpoint & VPN clients, this action is not REVERSIBLE which means that clients
will lose connection with the Server and the only way to re-establish it can be done by
re-issuing all certificates (for VPN) or by the re-connect tool for Endpoint clients.

Server communication can be re-established if the following operations are implemented:
1. Re-initialize the Internal Certificate Authority (use cpconfig).
2. Restart Check Point Services (cpstart, cpridstart).
3. Reset SIC on each Station that is managed by this Security Management Server.
4. Re-establish Trust with each Station that is managed by
this Security Management Server.
*******************************************
This operation will stop all Check Point Services (cpstop)
Are you sure you want to reset? (y/n) [n] ? y

*** Checking IKE Certificates ***
There are IKE Certificates that were generated by the
internal Certificate Authority.
Please remove them (using the SmartDashboard) so that
the internal Certificate Authority can be destroyed.

SIC Reset operation could not be completed
-sh-3.1#

```

Am I screwed ? 

[not_a_net_admin]

I'm not sure it's relevant but to add a bit more context: 

```

-sh-3.1# cpca_client lscert -stat Valid -kind SIC
Operation succeeded. rc=0.
3 certs found.

Subject = CN=NET04-FW01,O=NET04-FW-GFW01..mi6waq
Status = Valid Kind = SIC Serial = 12291 DP = 0
Not_Before: Sun Jun 2 09:50:29 2024 Not_After: Sat Jun 2 09:50:29 2029

Subject = CN=NET04-FW-GFW01.corporate.net,O=NET04-FW-GFW01..mi6waq
Status = Expired Kind = SIC Serial = 84860 DP = 0
Not_Before: Sun Jul 26 19:24:01 2015 Not_After: Sat Jul 25 19:24:01 2020

Subject = CN=NET04-FW-GFW01,O=NET04-FW-GFW01..mi6waq
Status = Expired Kind = SIC Serial = 86175 DP = 0
Not_Before: Sun Jul 26 11:03:59 2015 Not_After: Sat Jul 25 11:03:59 2020
-sh-3.1#

```

[PhoneBoy]

Once you sort out the certificate issues, you can create an administrator user via cpconfig on the management station.
Meanwhile, back to the SIC issue.

Normally you'd need to go into SmartDashboard to remove the IKE certificates first: https://support.checkpoint.com/results/sk/sk14532 
Since you can't do that, you might be able to do it by removing the certificate line from the relevant objects in $FWDIR/conf/objects_5.0.C (it's a flat text file).
Then you can perform the SIC reset.

There's another matter to be concerned with: CVE-2024-24919.
While R77.x versions aren't mentioned, as they are largely out of support, the CVE exists there as well.
More information: https://support.checkpoint.com/results/sk/sk182336

Since this CVE is specifically related to Remote Access and that is your planned use case, the only valid mitigation is upgrading to a supported release, or at least a release with a patch.
All the various R80.x/R81.x releases have patches as of this writing (assuming you're on the latest JHF) as well as R77.20.xx for Quantum Spark (SMB) appliances.
The last release supported on the 4200 appliance (which is End of Life) is R80.40.
The necessary software to perform the upgrade(s) likely requires a support agreement to access.

[not_a_net_admin]

I'm afraid I can't touch IKE keys as I still have 2 site-to-sites being migrated out of the CP4200. I took the preventative measures to check about local users (had none with Remote VPN access) and Active Directory sync accounts.

The thing is now that after the password of this AD sync account was updated and the restore of the backup they're now both out of sync and I can't seem to find the old password to restore Auth for Remote VPN. As of now no-one that were able to connect remotely prior to the disclosure is able to authenticate due to this password change. 

Which is not a bad thing per-se, we were trying to decomission this piece of hardware since August 2023 but several factors and clients priorities didn't allow us to that earlier, we have other VPN solution in place but not everyone is happy with the change so I was thinking about re-establishing Remote Access until the final shutdown. 

New question: is it possible to edit the password of the ldap sync account using dbedit? 

I can print the object by running:

dbedit> print servers mydomain.local__AD

But if I try to modify the login_password for the configured user with:

dbedit> modify servers mydomain.local__AD ldap_servers.login_password My$ecureP4$$w0rd

It returns:

failed to get field ldap_servers.login_password

and also if I don't try to unnest ldap_servers.login_password, trying to update only login_password returns the equivalent error:

failed to get field login_password

I feel it should be possible to update that object, but I'm missing how to get the parameters correctly.

In any case, these stacked boxes will be decommissioned soon, so I'm now mostly trying things out without too much pressure. 

Best regards,

[PhoneBoy]

Unless you can confirm that Mobile Access Blade is disabled and the gateway is not in the RemoteAccess encryption domain (ie Remote Access VPN is completely disabled), the gateway is vulnerable to the CVE regardless of whether you use weak passwords or not.

While it may be possible to change the password with dbedit, it is not documented how to do so. 
Short of regenerating the ICA or backdating the systems as well as the clients used to access SmartDashboard (those certs a a hint at the date range you need to look at), I don’t see how you’re going to get into SmartDashboard as that is the only place you’re going to be able to make the necessary changes.
Even if you do all this, your gateway will still be vulnerable to the CVE unless you disable Remote Access VPN as per the SK.

[not_a_net_admin]

After I restored the backup Mobile Access came back but then I can't start SmartDashboard to be able to disable it.

Is there a way to do this without impacting site-to-site tunnels via command line interface ?

When I try to run `fwm load Standard MYNET-FW` it still complains about the mismatching SIC and won't install the policy. And I couldn't run `fwm sic_reset` as I don't want to disrupt traffic from site-to-site tunnels. 

Perhaps I should. I'll see with the people upstairs. 

Thank you! 

[not_a_net_admin]

I did run cvpnstop and confirm that no cvpnproc is running in neither of the blades of the stack. Now, am I correct to judge that removing the IKE from the objects_5_0.C would disrupt site-to-site connections ? 

[Duane_Toler]

Yep, any futzing with the certificates in objects_5_0.C will break VPNs, but only if you install policy.  Simply editing the file does nothing; this has to be compiled into a policy package and delivered to the gateways.  Until then, all management-side things are done there and have no bearing on gateway functionality.

After you get into SmartDashboard, you can renew gateway certificates (if required) and apply them without disruption.

If you need to renew your ICA, Check Point's article on that is here, with various scenarios.  Choose your own adventure:

https://support.checkpoint.com/results/sk/sk158096

There is a procedure to do a full SIC reset on the gateways WITHOUT having to restart services and break traffic flow.  I've done this numerous times as well:  https://support.checkpoint.com/results/sk/sk86521

If you do this procedure, you REALLY should still schedule a time as early as you can to do the reboot/restart.  However, this will get you going.

As for dbedit, please don't try to rely on that for management configuration changes.  As PhoneBoy said, that's a highly undocumented mechanism with an incredibly unstable command interface.  I've tried it myself over the years and was greeted with inconsistencies.  It's meant only for R&D access.

[PhoneBoy]

I don't believe cvpnstop is enough to mitigate the vulnernability.
sk158096 seems to be your best option here, as suggested by @Duane_Toler.

Even if your plan is to decommission this box, I strongly recommend upgrading to a release that has a patch for CVE-2024-24919 before doing so.
We now have patches in sk182336 for releases dating back to R77.30.

[not_a_net_admin]

Hello Phoneboy, thank you for your message.

Although I understand the issue, I'm afraid we can't upgrade this appliance unless we hire a support plan, in any ISO download page we're facing an alert stating we're "Missing software subscription to download this file". 

So unless there's a hassle-free way to download the correct ISO for R77.30 version to upgrade and install the Jumbo Hotfix for CVE-2024-24919 we will resort to mitigate the issue by not allowing local users nor AD users to authenticate against the disabled Remote VPN blade.

I'm not sure if we can upgrade this set of CP4200 with R77.30 or if Checkpoint would make EOL software/hardware Generally available for people to at least upgrade to their latest compatible software without forcing a subscription onto teams that are struggling to decommission hardware due to commercial constraints. 

I hope I'm being clear. I'll keep on looking for a download link which isn't blocked by a paywall. 

Regards!

[_Val_]

@not_a_net_admin I understand the concern. Upon checking, R77.30 should be available for download without a subscription. CVE hotfix for R77.30 still asks for a subscription, but I raised an issue internally, and I am pretty confident, this limitation will also be lifted. 

UPDATE: resolved see the next comment

[_Val_]

@not_a_net_admin Both installation/upgrade software and the CVE Hotfix for R77.30 are now available to all registered UserCenter users, even without a support subscription.

[not_a_net_admin]

Hello @_Val_ thank you for your answer! But when I go to https://support.checkpoint.com/results/download/41359  - Even though I'm registered and logged in I'm still facing the Note:

"Missing software subscription to download this file." 

Am I missing something? Is there a different download page/link that I'm unaware of ? 

Best regards!

[PhoneBoy]

This should not be necessary to do an in-place upgrade of the existing management.
That said, it would probably be useful to make this available without a support subscription.
Will check and revert back.

[not_a_net_admin]

I got to this https://support.checkpoint.com/results/download/53393 and managed to download the ISO, but I'm afraid the version of my appliance isn't listed CP4200, it does lists however 4000 series, should I assume 4200 is part of the 4000 series ?

[not_a_net_admin]

This one does still show the Note about software subscription: https://support.checkpoint.com/results/download/41317 

[PhoneBoy]

You need to be logged into SupportCenter to download this file.
However, it should work without a Software Subscription (I tested this on a private account).
Same with Deployment Agent, which may be necessary to perform the upgrade: https://support.checkpoint.com/results/download/80931 

[PhoneBoy]

The 4200 is part of the 4000 Series.
For the CPUSE package, I assume we can make this available as we did the ISO.

[_Val_]

Why are you trying to download a migration tool? Do in-place upgrade. Those packages are available without a subscription.

All links can be found here: https://support.checkpoint.com/results/sk/sk104859

[not_a_net_admin]

Hello @_Val_ I'm facing the alert saying "Note: Missing software subscription to download this file" when I try to download the relevant TGZ. I've managed to Download the ISO, though, with Take_5. But I'm afraid that I won't be able to download the JHF with the fix for CVE-2024-24919 after that, I might be mistaken tho and looking at the wrong location. 

Best regards

 

[_Val_]

You should be able to upgrade with the ISO image.

[the_rock]

Im fairly sure you would be able to.

Andy

[PhoneBoy]

Hi, I checked the file just now with a private account and it appears to be downloadable without a subscription.
https://support.checkpoint.com/results/download/41380 

[the_rock]

I can confirm as well, works 100%.

[Duane_Toler]

You may (almost certainly) need to revoke and recreate the cp_mgmt certificate.  Fear not, this isn't the same as SIC reset.  It's just the management certificate.

https://support.checkpoint.com/results/sk/sk137332

Do that and you will be able to open SmartDashboard.  Yes the article now discusses SmartConsole, but it's the same.  I've done this many many times.

After you get into SmartDashboard, you can do whatever else you need.

[the_rock]

SIC can literally be anything, as long as its at least 4 password character and you dont need to worry, its one time key thats encrypted. Can be 1234, earth, homersimpson, planet, you get the idea. As @PhoneBoy said, you do this via cpconfig menu on the gw, exit, will do cpstop/cpstart (meaning load initial policy), then you reset it from gateway object in smart console using same oassword you typed on the ssh when doing a reset.

Test, make sure its established, push policy, verify.

Andy