IKE certificate auto-renewal failure

[Duane_Toler]

Last night, I had a customer's gateway fail VPN authentication suddenly.  A quick VPN debug showed the IKE certificate was expired!  I checked SmartConsole and yep, the IKE certificate on the SmartCenter was expired!

(gateway is R77.30, mgmt R80.20; yes upgrades are scheduled, that's not the issue here)

IKE certificates are supposed to auto-renew by cpca at 75% expiry, yes? I haven't had issues with certificate auto-renewals in a very very long time, so this was a major surprise.  I found another gateway certificate that will expire in 5 days, so I manually renewed it (along with the problematic gateway), then pushed policy to all gateways.  I checked all other gateways and they are good into 2020 and 2021, so I have time to make any repairs if needed.

With R80.20 management, is there something new I missed or some behavior change?  The ICA was still valid (through year 2030), all gateways and management system times are current and valid (sync with known good NTP servers).  I checked all hosts date and time to be sure!

Management R80.20 was a migrate from R77.30, which has been working very well for 15+ years.  No corruption or strange issues over time.

I haven't found any smoking-gun SK articles about this (I have seen the SHA-1/SHA-256 articles, sk103840, but that doesn't seem relevant).  sk59510 does not apply because this is site-to-site VPN, not Remote Access.  Manually renewing in SmartConsole was error-free, as it should be, so other SKs regarding renewal errors don't apply.

This is an odd one... anyone seen this lately, or have insight?

[PhoneBoy]

Maybe $FWDIR/log/cpca.elg* on the management will have a clue?

[Douglas_Rich]

Hey Duane, you ever find a solution for this?
We're having the same issue on R80.20

[Duane_Toler]

Unfortunately, no  It hasn't come back up for other gateways (yet), but I'll be keeping an eye on it for this (and other) customers.

An obvious thing, perhaps, is making sure the gateway can reach the SmartCenter on port 18264 (ICA services) for auto-renewal and CRL fetching.  If your SmartCenter is behind NAT and via VPN, you'll have to modify the $FWDIR/lib/implied_rules.def to exclude FW1_ICA_SERVICES from the list at the top (comment out that pragma #define line), then push policy.

Other than that, I don't know what could be causing this.  If it comes up again, I'll go through the cpca.elg log as @PhoneBoy mentioned above.  At the the last incident, I don't recall anything helpful in the log.  I'll also find and run a cpca debug if necessary (there's a large SK on running debugs of various daemons, sk97638).

If you find anything yourself, please let us know. 🙂

[Douglas_Rich]

Thanks man, I'll let you know

[Douglas_Rich]

So, I don't think VPN Certs are auto-renewed. I can find zero documentation that says otherwise, but numerous comments that ICA is renewed at 75% and User Certs.. but that's it.. I'm concluding that IKE VPN certs are a manual process but typically we don't have to do it because a Firewall is replaced before 5 years.

[Thomas_Eichelbu]

Hello team, 

funny thing ... i saw the opposite happen, the MGMT has automatically renewed all IKE certificates ... and it worked.
yes i hade some VPN outage ... 2h for a couple of remote GW´s. but this was the first time i have encountered a working certificate renewal of IKE certificates ... environment is on R81.10 Take 87 ...
i was thinking this is working only for R81.10 Take 95 ...  because the release notes state:

[cpca 13238 4116002688]@XXXXXX-ZZZZ[9 Jun 22:01:21] fwCA::CreateIkeCert: IKE cert created with dn "CN=XXXXFW01 VPN Certificate,O=XXXXXX-ZZZZ..bn78tq"
[cpca 13238 4116002688]@XXXXXX-ZZZZ[9 Jun 22:05:53] fwCA::CreateIkeCert: IKE cert created with dn "CN=YYYYYFWCL VPN Certificate,O=XXXXXX-ZZZZ..bn78tq"
[cpca 13238 4116002688]@XXXXXX-ZZZZ[9 Jun 22:13:20] fwCA::CreateIkeCert: IKE cert created with dn "CN=AAAAAFWCL VPN Certificate,O=XXXXXX-ZZZZ..bn78tq"
[cpca 13238 4116002688]@XXXXXX-ZZZZ[9 Jun 22:20:04] fwCA::CreateIkeCert: IKE cert created with dn "CN=BBBSFW01 VPN Certificate,O=XXXXXX-ZZZZ..bn78tq"
[cpca 13238 4116002688]@XXXXXX-ZZZZ[9 Jun 22:20:50] fwCA::CreateIkeCert: IKE cert created with dn "CN=CCCCCW01 VPN Certificate,O=XXXXXX-ZZZZ..bn78tq"
[cpca 13238 4116002688]@XXXXXX-ZZZZ[9 Jun 22:21:45] fwCA::CreateIkeCert: IKE cert created with dn "CN=JJJJJFW01 VPN Certificate,O=XXXXXX-ZZZZ..bn78tq"
[cpca 13238 4116002688]@XXXXXX-ZZZZ[9 Jun 22:26:17] fwCA::CreateIkeCert: IKE cert created with dn "CN=AAAAFWCL1 VPN Certificate,O=XXXXXX-ZZZZ..bn78tq"

interesting thing 🙂

best regards