Debugging IKE/Tunnel establishment on VSX R81.20.

[PetterD]

We have a VSX R81.20 system with multiple VS`es.
Some virtual systems was created many years ago on earlier versions and some on R81.10 and R81.20.
All virtual systems are running on the same VSX-nodes, R81.20.

We have an issue with establishing a VPN-tunnel from one of the newer virtual systems (VSID 16) to a third party and tried running a regular IKE-debug but no files were being created. On another virtual system (VSID 3) (created a long time ago) ike.elg-files are being created when running ike debug.

Checking "vpn iked status" i can see that for these 3 different systems, created on different versions iked is only running on the "oldest":

VSID3 (initially created on older R7x version - have several S2S tunnels working)

[Expert@fw-vsxnode-1:3]# vpn iked status
vpn: 'iked' is enabled.
vpn: 'iked' is configured for 2 instances.
vpn: The 'iked0' process is currently running.
vpn: The 'iked1' process is currently running.
[Expert@fw-vsxnode-1:3]#

VSID13 (initially created on R81.10 - have several S2S tunnels working)

[Expert@fw-vsxnode-1:13]# vpn iked status
vpn: 'iked' is disabled.
[Expert@fw-vsxnode-1:13]#

VSID16 (initially created on R81.20 version - Have one vpn-tunnel that doesnt work)

[Expert@fw-vsxnode-1:16]# vpn iked status
vpn: 'iked' is disabled.
[Expert@fw-vsxnode-1:16]# vsenv 3

According to R81.20 CLI reference guide there was a change in R81.10 on which daemon handles S2S traffic:
https://sc1.checkpoint.com/documents/R81.20/WebAdminGuides/EN/CP_R81.20_CLI_ReferenceGuide/Content/T...

However according to https://support.checkpoint.com/results/sk/sk180488 , in R81.20 and higher, S2S is handled by iked ?
(which it cant be since we have working VPN-tunnels on VSID13, but iked is disabled)

According to sk180488 it now looks like we have to run a full vpnd debug to be able to investigate IKE/tunnel establishment and
get a maintenance window to perform this debug ?

Does this mean that iked is not disabled on all virtual systems on R81.20, only systems initially created on R81.10/R81.20 and higher and that we now need maintenance window to look at S2S tunnel establishment ?

[the_rock]

Can you run this and see what it shows? Say peer IP is 7.8.9.9, run vpn iked calculate 7.8.9.9

It should show which iked process is handling debugs.

Andy

[PetterD]

Hi,

Thanks for the reply!

I ran this command for the peer-address we are haveing troubles establishing a tunnel for (we dont have a valid SA for it):

[Expert@ffw-vsxnode-1:16]# vpn iked calculate 1.2.3.4
vpn: Address '1.2.3.4' is not handled by any IKED daemon
[Expert@fw-vsxnode-1:16]#

Looks like this, and the other VS (that have a tunnel for the same peer) doesnt have any iked-processes running.

Very confusing as the documentation for R81.20 doesnt mention anything about cores, but that S2S is not being handled by iked in R81.20 (while other documentation says it is). 

[JozkoMrkvicka · Accepted solution]

If VS has assigned only 1 IPv4 core, then iked is not enabled. If VS has more than 1 IPv4 core assigned, iked is automatically enabled. Try to check coreXL assignment for all mentioned VSs.

By default, all VSs have only 1 IPv4 and 1 IPv6 core assigned. It is up to administrator to increase number of cores within VS object in SmartConsole.

Please be aware that changing coreXL settings for specific VS (doesnt matter if IPv4 or IPv6) will result in total outage within affected VS for couple of minutes.

[PetterD]

Hi,

Thanks for the reply!

Does this mean that this is not related to R81.20 where the documentation states that its not handled by iked ?
Couldt find any information about this being related to number of virtual cores but this could also match (as VS 13 & 16 have 1 Core).

On VS13 we do have working Ipsec VPN tunnels and one core is enough to handle all the limited traffic we do have on these VS`es.
Cant find any limitations that we cant run Ipsec VPN with 1 core only, so does this mean its handled by "vpnd" and that we have to perform
a full vpnd-debug (maintenance window) to get the ike-debugs ?

[JozkoMrkvicka]

This finding was discovered just recently. Check Point just wrote it is like it is even despite the fact there is no sk article nor written anything related to this issue in any admin guide / ATRG.

We never modified default number of CoreXL for any VS, since we didnt run to any issue with performance.

[PetterD]

Just an update,

We were able to get a quick maintenance window and increased the number of virtual cores from 1->2 on this VS and now we do have "iked" running so you were correct 🙂

Expert@fw-vsxnode-1:16]# fw ctl multik stat
ID | Active | CPU | Connections | Peak
-----------------------------------------------
0 | Yes | 2-15+ | 341 | 353
1 | Yes | 2-15+ | 187 | 189
[Expert@fw-vsxnode-1:16]# vpn iked status
vpn: 'iked' is enabled.
vpn: 'iked' is configured for 1 instances.
vpn: The 'iked0' process is currently running.
[Expert@fw-vsxnode-1:16]#

[Alex-]

I remember a discussion with PS a while back that since R80+, the minimum amount of cores for any system is 2 to meet architectural requirements. For instance, a VM with just one core will fail at the FTS indicating at least 2 cores are needed. 

In the case of VSX, they advised 2 virtual cores even for small VS, to ensures all OS functions work optimally since there it allows you indeed to create a VS with the default of 1 core.

[JozkoMrkvicka]

In case of VSX, changing CoreXL for specific VS will cause outage for that VS. This "feature" is no go in production. If Check Point is able to change this "feature", I am OK changing coreXL to higher numbers.

[Alex-]

Correct.

I was pointing the fact that any new VS we create has a minimum of two virtual cores so processes of that VS work in an environment with supported core allocation.

[JozkoMrkvicka]

Good approach 🙂 

I wish I had crystal ball back in old times where VSX was created (on R77.30), so I knew Check Point is changing VPN architecture starting from R81.10 and more cores will be needed 😛

In old R77.30 times, everything related to VPN was handled by vpnd only (and even within CoreXL FW instance #0).

[the_rock]

Good ol' R77.30 🙂

For some reason, I have fond memories of R75.47 version, always treted me well lol

Andy