topic Re: GRE flapping -- three vendors pointing fingers in Firewall and Security Management

GRE flapping -- three vendors pointing fingers

Garrett_DirSec — Wed, 03 Feb 2021 18:57:53 GMT

Hello --

Cisco interior router trying to maintain a GRE tunnel with regional zScaler cloud through a checkpoint R80.20-based cluster (latest GA HFA applied).

The GRE tunnel is flapping sporadically and everyone pointing fingers at each other.

CP TAC involved and simply did a "fw montor ..." traffic capture of proto 47 (GRE) to assert "we're passing the traffic".

However, the devil is in the details and wondering if anyone else encountered such an issue and have recommendations?

update #1: if GRE keepalives are turned OFF on Cisco router, the GRE tunnel stays up.

ideally, we would like to do "fw monitor ..." specifically for GRE keepalives and validate NAT being properly applied on CP cluster. This is current area of focus.

reference:

How GRE Keepalives works HERE.

Thanks -GA

Re: GRE flapping -- three vendors pointing fingers

Timothy_Hall — Wed, 03 Feb 2021 22:28:07 GMT

This filter should focus on all traffic between the two routers with a GRE-tunneled Protocol Type of zero (which is indicative a GRE keepalive) and should show you the packet arriving and leaving so you can verify the firewall is not somehow mishandling it:

cppcap -f "proto gre and ip[33:2]=0x00 and host 1.2.3.4 and host 5.6.7.8"

The ip[33:2] is an offset to where I believe the tunneled IP Protocol Type is located, with 0x00 matching a keepalive. I think I calculated that offset correctly but don't have any live GRE traffic to test with it.

As I mentioned in my Max Capture course, NAT can impact this matching so you will probably need to construct a slightly more complex filter to take into account pre-NAT and post-NAT matching IP combinations. I only had a few examples of offsets such as these in Max Capture because normally there is some kind of predefined macro available to match the header fields you want, but matching protocol GRE/17 in the outer header (proto gre) was all that was available.

Re: GRE flapping -- three vendors pointing fingers

Garrett_DirSec — Wed, 03 Feb 2021 22:43:25 GMT

Hey @Timothy_Hall - sincere thanks for msg! I hope you and family are good.

A couple questions:

Honestly, I'm not familiar with cppcap. I understand it's a less resource intensive version of standard tcpdump on GAIA. Why use cppcap (or tcpdump) over standard "fw monitor..." with relevant options? (REF HERE).
while digesting the cisco article on GRE Keepalives , the following diagram makes me question specifically which Protocol Type we should be looking for. In the case of diagram below, the outter header is Protocol Type = IP, while the inner header is Protocol Type = 0. Do I understand correctly that cppcap will "see" the outter header (PT=IP) but not interior header (PT=0)?

Re: GRE flapping -- three vendors pointing fingers

the_rock — Wed, 03 Feb 2021 23:51:56 GMT

You can try something like this on CP -> fw monitor -e "accept port 1723 and proto 47;"

See what you get...thats GRE port and protocol, so should give you something. BUT, Im wondering though if it will, since you said CP is just a passthrough?

Andy

Re: GRE flapping -- three vendors pointing fingers

Timothy_Hall — Wed, 03 Feb 2021 23:56:09 GMT

These days cppcap is my preferred tool unless a special situation is present where you must use something else, but since GRE traffic is not accelerated by SecureXL and always goes F2F (that may have changed in R80.20 - can't remember), you could certainly use fw monitor -e if you want.

The ip[33:2] offset I calculated is going after the rightmost tunneled GRE Protocol Type of 0, the proto gre is going for the outer IP Protocol header Protocol Type which is not shown in that diagram. Unlike IPSec traffic, GRE is not encrypted so cppcap or fw monitor can see into the tunneled GRE header just fine. See below, I'm sure if I screwed up someone will be happy to correct me:

Re: GRE flapping -- three vendors pointing fingers

the_rock — Thu, 04 Feb 2021 00:07:12 GMT

Im sure you are correct, Im just an old school guy, thats all : ))

Re: GRE flapping -- three vendors pointing fingers

Garrett_DirSec — Thu, 04 Feb 2021 15:28:38 GMT

Hey Tim -- sincere thanks. Hand to forehead moment -- and why Checkpoint recommends IPSEC over GRE -- that the tunneled GRE traffic is cleartext.

thanks for taking the time on diagram with annotation. this great.