Debugging IKE/Tunnel establishment on VSX R81.20.

[PetterD]

We have a VSX R81.20 system with multiple VS`es.
Some virtual systems was created many years ago on earlier versions and some on R81.10 and R81.20.
All virtual systems are running on the same VSX-nodes, R81.20.

We have an issue with establishing a VPN-tunnel from one of the newer virtual systems (VSID 16) to a third party and tried running a regular IKE-debug but no files were being created. On another virtual system (VSID 3) (created a long time ago) ike.elg-files are being created when running ike debug.

Checking "vpn iked status" i can see that for these 3 different systems, created on different versions iked is only running on the "oldest":

VSID3 (initially created on older R7x version - have several S2S tunnels working)

[Expert@fw-vsxnode-1:3]# vpn iked status
vpn: 'iked' is enabled.
vpn: 'iked' is configured for 2 instances.
vpn: The 'iked0' process is currently running.
vpn: The 'iked1' process is currently running.
[Expert@fw-vsxnode-1:3]#

VSID13 (initially created on R81.10 - have several S2S tunnels working)

[Expert@fw-vsxnode-1:13]# vpn iked status
vpn: 'iked' is disabled.
[Expert@fw-vsxnode-1:13]#

VSID16 (initially created on R81.20 version - Have one vpn-tunnel that doesnt work)

[Expert@fw-vsxnode-1:16]# vpn iked status
vpn: 'iked' is disabled.
[Expert@fw-vsxnode-1:16]# vsenv 3

According to R81.20 CLI reference guide there was a change in R81.10 on which daemon handles S2S traffic:
https://sc1.checkpoint.com/documents/R81.20/WebAdminGuides/EN/CP_R81.20_CLI_ReferenceGuide/Content/T...

However according to https://support.checkpoint.com/results/sk/sk180488 , in R81.20 and higher, S2S is handled by iked ?
(which it cant be since we have working VPN-tunnels on VSID13, but iked is disabled)

According to sk180488 it now looks like we have to run a full vpnd debug to be able to investigate IKE/tunnel establishment and
get a maintenance window to perform this debug ?

Does this mean that iked is not disabled on all virtual systems on R81.20, only systems initially created on R81.10/R81.20 and higher and that we now need maintenance window to look at S2S tunnel establishment ?

[the_rock]

Can you run this and see what it shows? Say peer IP is 7.8.9.9, run vpn iked calculate 7.8.9.9

It should show which iked process is handling debugs.

Andy

[PetterD]

Hi,

Thanks for the reply!

I ran this command for the peer-address we are haveing troubles establishing a tunnel for (we dont have a valid SA for it):

[Expert@ffw-vsxnode-1:16]# vpn iked calculate 1.2.3.4
vpn: Address '1.2.3.4' is not handled by any IKED daemon
[Expert@fw-vsxnode-1:16]#

Looks like this, and the other VS (that have a tunnel for the same peer) doesnt have any iked-processes running.

Very confusing as the documentation for R81.20 doesnt mention anything about cores, but that S2S is not being handled by iked in R81.20 (while other documentation says it is). 

[JozkoMrkvicka · Accepted solution]

If VS has assigned only 1 IPv4 core, then iked is not enabled. If VS has more than 1 IPv4 core assigned, iked is automatically enabled. Try to check coreXL assignment for all mentioned VSs.

By default, all VSs have only 1 IPv4 and 1 IPv6 core assigned. It is up to administrator to increase number of cores within VS object in SmartConsole.

Please be aware that changing coreXL settings for specific VS (doesnt matter if IPv4 or IPv6) will result in total outage within affected VS for couple of minutes.

[PetterD]

Hi,

Thanks for the reply!

Does this mean that this is not related to R81.20 where the documentation states that its not handled by iked ?
Couldt find any information about this being related to number of virtual cores but this could also match (as VS 13 & 16 have 1 Core).

On VS13 we do have working Ipsec VPN tunnels and one core is enough to handle all the limited traffic we do have on these VS`es.
Cant find any limitations that we cant run Ipsec VPN with 1 core only, so does this mean its handled by "vpnd" and that we have to perform
a full vpnd-debug (maintenance window) to get the ike-debugs ?

[JozkoMrkvicka]

This finding was discovered just recently. Check Point just wrote it is like it is even despite the fact there is no sk article nor written anything related to this issue in any admin guide / ATRG.

We never modified default number of CoreXL for any VS, since we didnt run to any issue with performance.

[PetterD]

Just an update,

We were able to get a quick maintenance window and increased the number of virtual cores from 1->2 on this VS and now we do have "iked" running so you were correct 🙂

Expert@fw-vsxnode-1:16]# fw ctl multik stat
ID | Active | CPU | Connections | Peak
-----------------------------------------------
0 | Yes | 2-15+ | 341 | 353
1 | Yes | 2-15+ | 187 | 189
[Expert@fw-vsxnode-1:16]# vpn iked status
vpn: 'iked' is enabled.
vpn: 'iked' is configured for 1 instances.
vpn: The 'iked0' process is currently running.
[Expert@fw-vsxnode-1:16]#

[Alex-]

I remember a discussion with PS a while back that since R80+, the minimum amount of cores for any system is 2 to meet architectural requirements. For instance, a VM with just one core will fail at the FTS indicating at least 2 cores are needed. 

In the case of VSX, they advised 2 virtual cores even for small VS, to ensures all OS functions work optimally since there it allows you indeed to create a VS with the default of 1 core.

[JozkoMrkvicka]

In case of VSX, changing CoreXL for specific VS will cause outage for that VS. This "feature" is no go in production. If Check Point is able to change this "feature", I am OK changing coreXL to higher numbers.

[Alex-]

Correct.

I was pointing the fact that any new VS we create has a minimum of two virtual cores so processes of that VS work in an environment with supported core allocation.

[JozkoMrkvicka]

Good approach 🙂 

I wish I had crystal ball back in old times where VSX was created (on R77.30), so I knew Check Point is changing VPN architecture starting from R81.10 and more cores will be needed 😛

In old R77.30 times, everything related to VPN was handled by vpnd only (and even within CoreXL FW instance #0).

[the_rock]

Good ol' R77.30 🙂

For some reason, I have fond memories of R75.47 version, always treted me well lol

Andy

[Matlu]

Hi

I have a VS that is doing a S2S VPN using Ikev2

I would like to evaluate with a debug everything that is exchanged on P1 and P2 for my VPN

What command can help me with this goal?

Which files should I extract from my VSX box in order to evaluate them with the IKEView tool

My VS is ID 5 and my peer has the IP 181.90.100.50

Thanks for your comments.

[the_rock]

vpn debug trunc

vpn debug ikeon

-generate traffic

vpn debug ikeoff

Look for vpnd and ike* files in $FWDIR/log dir

Andy

[Matlu]

Bro

Is there any particular extension I need to look for for IKEv2?

I have problems with traffic exchange on P2, and I want to ‘detect’ what is being exchanged at this stage by both peers.

Cheers

[the_rock]

You mean file extension?

Andy

[Matlu]

Yep 🥸

[the_rock]

Either .trace or .elg

Andy

[Matlu]

Hi, Bro.

What file extension do files that need to be opened in IKEVIEW usually have?
Because I'm trying to open files with .elg extensions, but it doesn't seem to read the file.
I'm using IKEv2.

Cheers.

[the_rock]

Seems file could be empty. Anything with ext .trace?

Andy

[Matlu]

I have obtained the following results after debugging.

Which file can be read in IKEVIEW?
Can only .elg extensions be read?

[the_rock]

Try ones ending in trace.

[Matlu]

Hello,
In VSX environments.
Is it “objective” to use the “tcpdump” command to capture traffic?
I have a problem with a particular selector.
Source: 10.100.0.0/24
Destination: 172.20.10.200
Port: 8080

My VPN is up on P1 and P2, but when the source launches connection tests to the destination behind our VS, it doesn't work.

Is it useful to perform tshoot tests with tcpdump in these scenarios?
Is there a command that can be useful to capture whether the traffic is actually reaching our VS?

Best regards.

[Alex-]

You should use cppcap, not tcpdump which can impact your system.

The -v flag allows to capture traffic on a specific VS.

Check out the documentation and the SK.

https://sc1.checkpoint.com/documents/R82/WebAdminGuides/EN/CP_R82_CLI_ReferenceGuide/Content/Topics-...

[Matlu]

Hi.

Is there any CLI command able to show me in real time what my VS ‘observes’ in the P2 negotiation?

My VPN works fine but the problem is with a new flow that was added, and we want to be able to know if the CP is interpreting well this new flow added.