fw monitor inspection point e or E

[Tom_Coussement]

I know the inspection points "i", "I", "o" & "O"

Now I see for example eth2:e and eth2:E in fw monitor output

Is this voor incomming and outgoing encrypted traffic?

Can't find it in documentation...

[Timothy_Hall]

Please attach a screenshot of what you are seeing (with IP addresses sanitized of course if needed).  Also what level of gateway code are you running?

--
My book "Max Power: Check Point Firewall Performance Optimization"
now available via http://maxpowerfirewalls.com.

[Tom_Coussement]

Firewall is R80.10

[vs_0][fw_1] eth0:i[306]: 192.168.210.11 -> 192.168.197.142 (UDP) len=306 id=29530
UDP: 53 -> 61872
[vs_0][fw_1] eth0:I[306]: 192.168.210.11 -> 192.168.197.142 (UDP) len=306 id=29530
UDP: 53 -> 61872
[vs_0][fw_1] eth2:o[306]: 192.168.210.11 -> 192.168.197.142 (UDP) len=306 id=29530
UDP: 53 -> 61872
[vs_0][fw_1] eth2:O[306]: 192.168.210.11 -> 192.168.197.142 (UDP) len=306 id=29530
UDP: 53 -> 61872
[vs_0][fw_0] eth2:e[306]: 192.168.210.11 -> 192.168.197.142 (UDP) len=306 id=29530
UDP: 53 -> 61872
[vs_0][fw_0] eth2:E[306]: 192.168.210.11 -> 192.168.197.142 (UDP) len=306 id=29530
UDP: 53 -> 61872

I think it is encrypt and decrypt as it is vpn traffic but I haven't seen it before and couldn't find it in documentation.

eth2 is the WAN interface

[Timothy_Hall]

Hmm interesting, notice that the iIoO packets were handled on FW Worker core instance #1, yet the same packets when shown with eE are indicated on Firewall Worker core instance #0.  In R77.30 and earlier, all IPSec VPN traffic could be handled only by the lowest-numbered Firewall Worker core (#0).  This limitation was lifted in R80.10 gateway with the introduction of multicore IPSec which is enabled by default. 

So my guess is that these new eE capture points have something to do with VPN handling by the new multicore IPsec feature.  You can query the status of this feature in R80.10 with the new vpn tu mstats and vpn tu tlist commands.  I bet these eE capture points would no longer appear if multicore IPSec was disabled, however doing so is not supported.

--
My book "Max Power: Check Point Firewall Performance Optimization"
now available via http://maxpowerfirewalls.com.

[Gaurav_Pandya]

Hi,

Are you doing in R80? I think there is new parameters involved in R80 but I am not sure about meaning of :e & :E

[Vladimir]

I wonder if the wireshark filters for Checkpoint will work on the fw monitor output with eE...

[Tom_Coussement]

Not yet it seems.

[RickHoppe]

My colleague https://community.checkpoint.com/people/jelle257658b7-4137-449e-8a0a-0baf97f9f08c‌ also ran into these eE inspection points on R80.10.

I've started a chat with TAC and asked them to point us to documentation or an SK-article regarding those eE inspection points. Unfortunatly there is no official documentation yet. I'm also quite surprised no Check Point employee responded to this thread either.

This is the answer I've received:

"e is before the encryption and E is after the encryption. It's a new feature in R80.10."

I asked if there would be a dD for decryption.

"No, you won't see the d D, as the e stands for the status when it is not encrypted yet, and E is when it is encrypted. So for outbound traffic, you will see e E, and for inbound, you will see E e."

The very helpful TAC engineer seemed to be surprised too as he also wondered if dD inspection points would exist and asked his Technical lead. So kudo's to TAC. I told them I hope this would be documented in the near future.

[Timothy_Hall]

Sounds like two new default capture points, "e" at the entrance to the vpn_encrypt chain module (between o and O) and "E" at the exit of that same module as shown in output of fw ctl chain, yet only displayed if that module actually needs to encrypt.

--
My book "Max Power: Check Point Firewall Performance Optimization"
now available via http://maxpowerfirewalls.com

[RickHoppe · Accepted solution]

If it's in that order I would expect it to be displayed as iIoeEO but it is displayed as iIoOeE. When looking at the output of fw ctl chain the fw VM outbound chain comes before the vpn encrypt chain. That's where the inspection points are, right?

Out chain:

o  fw VM outbound  O - - e  vpn encrypt E

I guess sk30583  should be updated something like this:

There are six inspection points when a packet passes through a R80.10 Security Gateway:

#	Traffic direction (*)	Relation to FireWall Virtual Machine	Name of inspection point	Notion of inspection point
1	Inbound	Before the inbound FW VM	Pre-Inbound	"i"
2	Inbound	After the inbound FW VM	Post-Inbound	"I"
3	Outbound	Before the outbound FW VM	Pre-Outbound	"o"
4	Outbound	After the outbound FW VM	Post-Outbound	"O"
5	Outbound	Before the vpn encrypt	Pre-Encrypt	"e"
6	Outbound	After the vpn encrypt	Post-Encrypt	"E"

^(*) The traffic direction (inbound/outbound) relates to each specific packet, and not to the connection.

[_Val_]

Just for your information, the mentioned SK is already updated with relevant notes. Mind, e and E points are related to the cases where IPSec VPN is active and are not available otherwise.

[Gaurav_Pandya]

Correct

[Vladimir]

Unless there are dD values, the eE will probably show up as Ee on the decrypting side.

[Jelle_Hazenberg]

Actually i did test this in my lab. To test this I created a Star Community and initiated some HTTP traffic from

Site A(10.1.1.0/24) to Site B(10.2.2.0/24).

This were the results:

Site A

HTTP Request:

[vs_0][fw_2] eth1:i[52]: 10.1.1.50 -> 10.2.2.50 (TCP) len=52 id=8800
TCP: 51289 -> 80 .S.... seq=6cbefb2f ack=00000000
[vs_0][fw_2] eth1:I[52]: 10.1.1.50 -> 10.2.2.50 (TCP) len=52 id=8800

TCP: 51289 -> 80 .S.... seq=6cbefb2f ack=00000000

-------------- OS IP Stack -------------

[vs_0][fw_2] eth0:o[52]: 10.1.1.50 -> 10.2.2.50 (TCP) len=52 id=8800
TCP: 51289 -> 80 .S.... seq=6cbefb2f ack=00000000
[vs_0][fw_2] eth0:O[52]: 10.1.1.50 -> 10.2.2.50 (TCP) len=52 id=8800
TCP: 51289 -> 80 .S.... seq=6cbefb2f ack=00000000
[vs_0][fw_2] eth0:e[52]: 10.1.1.50 -> 10.2.2.50 (TCP) len=52 id=8800
TCP: 51289 -> 80 .S.... seq=6cbefb2f ack=00000000

Site B

HTTP Request:

[vs_0][fw_0] eth0:i[52]: 10.1.1.50 -> 10.2.2.50 (TCP) len=52 id=8800
TCP: 51289 -> 80 .S.... seq=6cbefb2f ack=00000000
[vs_0][fw_0] eth0:I[52]: 10.1.1.50 -> 10.2.2.50 (TCP) len=52 id=8800
TCP: 51289 -> 80 .S.... seq=6cbefb2f ack=00000000

-------------- OS IP Stack -------------

[vs_0][fw_0] eth1:o[52]: 10.1.1.50 -> 10.2.2.50 (TCP) len=52 id=8800
TCP: 51289 -> 80 .S.... seq=6cbefb2f ack=00000000
[vs_0][fw_0] eth1:O[52]: 10.1.1.50 -> 10.2.2.50 (TCP) len=52 id=8800
TCP: 51289 -> 80 .S.... seq=6cbefb2f ack=00000000

--- Webserver - 10.2.2.50:80 ---

Site B

HTTP Reply:

[vs_0][fw_0] eth1:i[52]: 10.2.2.50 -> 10.1.1.50 (TCP) len=52 id=0
TCP: 80 -> 51289 .S..A. seq=94647385 ack=6cbefb30
[vs_0][fw_0] eth1:I[52]: 10.2.2.50 -> 10.1.1.50 (TCP) len=52 id=0
TCP: 80 -> 51289 .S..A. seq=94647385 ack=6cbefb30

-------------- OS IP Stack -------------

[vs_0][fw_0] eth0:o[52]: 10.2.2.50 -> 10.1.1.50 (TCP) len=52 id=0
TCP: 80 -> 51289 .S..A. seq=94647385 ack=6cbefb30
[vs_0][fw_0] eth0:O[52]: 10.2.2.50 -> 10.1.1.50 (TCP) len=52 id=0
TCP: 80 -> 51289 .S..A. seq=94647385 ack=6cbefb30
[vs_0][fw_1] eth0:e[52]: 10.2.2.50 -> 10.1.1.50 (TCP) len=52 id=0
TCP: 80 -> 51289 .S..A. seq=94647385 ack=6cbefb30

Site A

HTTP Reply:

[vs_0][fw_2] eth0:I[52]: 10.2.2.50 -> 10.1.1.50 (TCP) len=52 id=0
TCP: 80 -> 51289 .S..A. seq=94647385 ack=6cbefb30

-------------- OS IP Stack -------------

[vs_0][fw_2] eth1:o[52]: 10.2.2.50 -> 10.1.1.50 (TCP) len=52 id=0
TCP: 80 -> 51289 .S..A. seq=94647385 ack=6cbefb30
[vs_0][fw_2] eth1:O[52]: 10.2.2.50 -> 10.1.1.50 (TCP) len=52 id=0
TCP: 80 -> 51289 .S..A. seq=94647385 ack=6cbefb30

Interesting stuff here if you ask me. I am not seeing the "E" (Post-Encrypt) in my fw monitor.

CoreXL is enabled on Site A:

[Expert@GW1:0]# fw ctl affinity -l -a
eth0: CPU 0
eth1: CPU 0
eth2: CPU 0
eth3: CPU 0
Kernel fw_0: CPU 3
Kernel fw_1: CPU 2
Kernel fw_2: CPU 1
Daemon in.asessiond: CPU 1 2 3
Daemon fwd: CPU 1 2 3
Daemon mpdaemon: CPU 1 2 3
Daemon cprid: CPU 1 2 3
Daemon cpd: CPU 1 2 3

Maybe it has something to do with the same instance handling the traffic?

[Expert@GW1:0]# vpn tu mstats

   Instance#     # of inSPIs    # of outSPIs
           0               0               0
           1               0               0
           2               2               2
   -----------------------------------------
   Summary:                2               2

CoreXL is disabled on Site B, but we do see the "e" (Pre-Encrypt)

Also in the previous post by Rick Hoppe the following was stated by TAC:

So for outbound traffic, you will see e E, and for inbound, you will see E e."

I am not seeing this behavior in my fw monitor.

[Vladimir]

Did you have a SecureXL disabled while running fw monitor?

[Jelle_Hazenberg]

Yes, to get this results i disabled SecureXL

[KennyManrique]

Perhaps "E" inspection point is for already encripted traffic destined to the peer?

Did you make a fw monitor capture between peer addresses (those who negotiate IPSec tunnel) for IKE and ESP traffic to verify if "E" inspection point is visible ??

[Jelle_Hazenberg]

Kenny Manrique‌, I don't think you would see the inspection points if you capture "outer layer" tunnel traffic (if i understand you correct), based on earlier outputs from the topic starter we are seeing the following "inner layer" behavior:

[vs_0][fw_1] eth0:i[306]: 192.168.210.11 -> 192.168.197.142 (UDP) len=306 id=29530
UDP: 53 -> 61872
[vs_0][fw_1] eth0:I[306]: 192.168.210.11 -> 192.168.197.142 (UDP) len=306 id=29530
UDP: 53 -> 61872
[vs_0][fw_1] eth2:o[306]: 192.168.210.11 -> 192.168.197.142 (UDP) len=306 id=29530
UDP: 53 -> 61872
[vs_0][fw_1] eth2:O[306]: 192.168.210.11 -> 192.168.197.142 (UDP) len=306 id=29530
UDP: 53 -> 61872
[vs_0][fw_0] eth2:e[306]: 192.168.210.11 -> 192.168.197.142 (UDP) len=306 id=29530
UDP: 53 -> 61872
[vs_0][fw_0] eth2:E[306]: 192.168.210.11 -> 192.168.197.142 (UDP) len=306 id=29530
UDP: 53 -> 61872

[KennyManrique]

Hi Jelle Hazenberg‌

Yesterday I deployed an R80.10 Cluster for a customer and established some Site to Site VPN.

For not staying with the doubt, I did a fw capture of internal hosts (those networks who communicate through VPN) and external interfaces (those who negotiate the tunnel); to me the inspection point E (Post-encrypt) is visible on external side.

I had to do two separate fw monitor's due the expression filter can only be applied once (make the capture simultaneously on two ssh sessions break my original filters defined).

To test the tunnel I did a ping from 192.168.42.13 (R80.10 GW LAN) to 192.168.2.25 (External GW LAN) with 800 bytes for discriminate from other traffic.

INTERNAL CAPTURE

[Expert@FIREWALL:0]# fw monitor -e "host(192.168.42.13) and host(192.168.2.25), accept;"
monitor: getting filter (from command line)
monitor: compiling
monitorfilter:
Compiled OK.
monitor: loading
monitor: monitoring (control-C to stop)
[vs_0][fw_1] bond1:i[828]: 192.168.42.13 -> 192.168.2.25 (ICMP) len=828 id=17837
ICMP: type=8 code=0 echo request id=1 seq=22445
[vs_0][fw_1] bond1:I[828]: 192.168.42.13 -> 192.168.2.25 (ICMP) len=828 id=17837
ICMP: type=8 code=0 echo request id=1 seq=22445
[vs_0][fw_1] eth4:o[828]: 192.168.42.13 -> 192.168.2.25 (ICMP) len=828 id=17837
ICMP: type=8 code=0 echo request id=1 seq=22445
[vs_0][fw_1] eth4:O[828]: 192.168.42.13 -> 192.168.2.25 (ICMP) len=828 id=17837
ICMP: type=8 code=0 echo request id=1 seq=22445
[vs_0][fw_1] eth4:e[828]: 192.168.42.13 -> 192.168.2.25 (ICMP) len=828 id=17837
ICMP: type=8 code=0 echo request id=1 seq=22445 --> PRE ENCRYPT PING (ECHO-REQUEST)
[vs_0][fw_1] eth3:I[828]: 192.168.2.25 -> 192.168.42.13 (ICMP) len=828 id=12130
ICMP: type=0 code=0 echo reply id=1 seq=22445 --> DECRYPTED PING (ECHO-REPLY)
[vs_0][fw_1] bond1:o[828]: 192.168.2.25 -> 192.168.42.13 (ICMP) len=828 id=12130
ICMP: type=0 code=0 echo reply id=1 seq=22445
[vs_0][fw_1] bond1:O[828]: 192.168.2.25 -> 192.168.42.13 (ICMP) len=828 id=12130
ICMP: type=0 code=0 echo reply id=1 seq=22445

monitor: caught sig 2
monitor: unloading

EXTERNAL CAPTURE

For security reasons, external addresses were changed to X.X.X.X (R80.10 GW) and Y.Y.Y.Y (External GW)

[Expert@FIREWALL:0]# fw monitor -e "host(X.X.X.X) and host(Y.Y.Y.Y), accept;"
 monitor: getting filter (from command line)
 monitor: compiling
monitorfilter:
Compiled OK.
 monitor: loading
 monitor: monitoring (control-C to stop)

[vs_0][fw_1] eth3:E[880]: X.X.X.X -> Y.Y.Y.Y (50)  len=880 id=11150 --> ENCRYPTED OUTGOING PING (ECHO-REQUEST)

[vs_0][fw_1] eth3:i[880]: Y.Y.Y.Y -> X.X.X.X (50)  len=880 id=35654 --> ENCRYPTED  INCOMING PING (ECHO-REPLY)

monitor: caught sig 2
 monitor: unloading

ICMP Header adds 8 bytes to packet while IP header adds 20 bytes so this way the original packet of 800 bytes is seen as one of 828 bytes on internal interface.

The encryption adds 52 bytes to original packet (new IP header, ESP data), resulting on 880 bytes packet leaving the gateway.

As you can view, on my scenario, "E" inspection point is visible on external addresses while "e" inspection point is visible on internal addresses.

Regards.

[Jelle_Hazenberg]

Well Kenny, it seems like in our case you're correct. But that this doesn't make any sense to me. Why are we seeing other behavior in the capture from the topic starter?

[vs_0][fw_1] eth0:i[306]: 192.168.210.11 -> 192.168.197.142 (UDP) len=306 id=29530
UDP: 53 -> 61872
[vs_0][fw_1] eth0:I[306]: 192.168.210.11 -> 192.168.197.142 (UDP) len=306 id=29530
UDP: 53 -> 61872
[vs_0][fw_1] eth2:o[306]: 192.168.210.11 -> 192.168.197.142 (UDP) len=306 id=29530
UDP: 53 -> 61872
[vs_0][fw_1] eth2:O[306]: 192.168.210.11 -> 192.168.197.142 (UDP) len=306 id=29530
UDP: 53 -> 61872
[vs_0][fw_0] eth2:e[306]: 192.168.210.11 -> 192.168.197.142 (UDP) len=306 id=29530
UDP: 53 -> 61872
[vs_0][fw_0] eth2:E[306]: 192.168.210.11 -> 192.168.197.142 (UDP) len=306 id=29530
UDP: 53 -> 61872

This is also "inner layer" traffic right?

[KennyManrique]

You're right, it seems inner layer traffic. I'm not sure why "E" inspection point is visible, the only reason I can think is fwaccel was enabled on the capture maybe?

[Timothy_Hall]

Looks like sk104760: ATRG: VPN Core has been updated with the following to describe the encrypt operation (oOeE):

1. In R77.30 and lower, a packet enters the Security Gateway on CoreXL FW instance #0 (at Pre-Inbound chain "i").
   In R80.10 and above, a packet enters the Security Gateway on the connection CoreXL FW instance.
2. The packet is inspected by the Firewall and sent to the OS Kernel (at Post-Inbound chain "I").
3. The OS routes the packet, using the destination IP address of the original packet.
4. The outgoing packet is inspected by the FireWall (at Pre-Outbound chain "o").
5. Encryption information is prepared in vpnk module (at Post-Outbound chain "O").
6. In R80.10 and above, the vpnk module on the tunnel CoreXL FW instance gets the packet before encryption (at chain "e").
7. In R80.10 and above, packet is encrypted by vpnk module (at chain "E").
8. IPsec packet is sent out.

Note the distinction between "connection" CoreXL instance and "tunnel" CoreXL instance, which does not exist for the decrypt operation at iI.  So it looks like on the encrypt side (oOeE) a different CoreXL FW instance can perform the encrypt operation (the tunnel instance) vs. execute the regular firewall inspection for rest of the connection (the connection instance).   eE was added to help see which CoreXL instance actually did the encrypt operation as it can be different from the CoreXL instance that did the rest of the inspection.  I was puzzled by this core number difference in my Nov 17th reply to Tom Coussement and this explains it.

--
My Book "Max Power: Check Point Firewall Performance Optimization"
Second Edition Coming Soon

[HeikoAnkenbrand]

See this articel R80.x Security Gateway Architecture (Logical Packet Flow).

It contains all inspection points in a overview.

Regards,

Heiko

[HeikoAnkenbrand]

SecureXL "fwaccel off" does not have to be disabled on R80.20 to run "fw monitor". This is good for performance, so "fw monitor" does not affect performance any more.

Furthermore there are new fw monitor inspection points (iq, iQ, id, iD, oq, oQ) available in R80.20.

More see here:

R80.x Performance Tuning and Debug Tips – fw monitor or

https://community.checkpoint.com/docs/DOC-3041-r80x-security-gateway-architecture-logical-packet-flo... 

Regards

Heiko