topic Re: BGP advice in Cloud Firewall

BGP advice

Sandgirl — Wed, 08 Mar 2023 13:44:24 GMT

Hi guys,

Let me start with saying I have very little experience with Checkpoints - only few weeks, so please bear that in mind 🙂

I'm trying to establish a BGP connectivity between our two Checkpoints in Azure (CloudGuard) and the Azure Route Server (ARS).

I have two setups:

One is the Checkpoint Server Manager and ARS.

The other one is Checkpoint Firewall and ARS.

In the first instance, I can see on Checkpoint (CLI) that there is a two way communication: SYN, SYN ACK, ACK (but also F, P and R). But the neighbourhood is not estaliblished - the peers show on Checkpoint SM as either 'active' or 'idle'.

With the second setup, I can only see traffic (in CLI) coming from ARS. Checkpoint does not respond at all. I have set up ASN in the GUI and peers, but there is absolutely no response. Is there any other setting somewhere I need to enable/setup?

Finally, the above setups are just in my lab. When we deploy the solution, the Checkpoints will be behind Azure Load Balancer. Is this supported? I have read somewhere on here that it might not be?

Any help would be greatly appreciated!

Re: BGP advice

the_rock — Wed, 08 Mar 2023 15:21:38 GMT

Hey @Sandgirl ,

No need to apologize, we are here to help. So, below is good article explaining BGP different statuses:

https://www.ciscopress.com/articles/article.asp?p=2756480&seqNum=4

Now, I dont know for certain if BGP is supported when CPs are behind Azure load balancer, so maybe someone else can confirm that for you.

Now, here are few thing to check if show bgp summary command from clish does not show you established.

-verify BGP settings match on both sides

-do fw ctl zdebug | grep x.x.x.x (other peer BGP ip address) and see why its getting dropped (if it is)

-do tcpdump or fw monitor as per below port (BGP)

tcpdump -nni any host x.x.x.x and port 179 (again, replace with other side peer IP)

fw monitor -e "accept host(x.x.x.x) and port(179)

Alternatively, you can also do new version of fw monitor flag

Idea is this -> fw monitor -F "src ip, src port, dst ip, dst port, protocol" -F "srcip, src port, dst ip, dst port, protocl"

So, lets say your IP is 1.1.1.1 and dst IP is 2.2.2.2 and port is BGP (179) and protocol can be anything, you can do this

fw monitor -F "1.1.1.1,0,2.2.2.2,179,0" -F "2.2.2.2,0,1.1.1.1,179,0"

We dont really care about source, only destination port.

Keep in mind all these commands are done in expert mode.

Hope that helps.

Andy

Re: BGP advice

Sandgirl — Wed, 08 Mar 2023 17:06:02 GMT

Thank you so much Andy!

So I run the
fw ctl zdebug drop

and I'm getting the following message:

fw_log_drop_ex: Packet proto =6 x.x.x.x:y -> z.z.z.z:179 dropped by fw_send_log_drop Reason: Rulebase drop - DEFAULT POLICY

Where would I find this policy so I can override it?

Re: BGP advice

the_rock — Thu, 09 Mar 2023 00:27:36 GMT

Sorry for the late reply, not sure how I missed the email notification, apologies. So to verify right policy, can you run fw stat command on the CP fw? If default policy or default filter is on, that would block everything.

Re: BGP advice

Blason_R — Thu, 09 Mar 2023 03:08:30 GMT

That means - You need to open the port TCP/179 destined to your firewall. You need to add explicit rule for the same above stealth if you have so. Since activating BGP does not add any implicit rule ensure you add the explicit rule as I said above.

Re: BGP advice

_Val_ — Thu, 09 Mar 2023 08:20:23 GMT

DEFAULT POLICY is the name of the policy package applied on your security gateway. It this specific case, it is out-of-the-box policy, which means you did not apply any policy to SG.

The default policy only allows internal communication with the other parts of your security system, and nothing else.

Please make sure you initiated SIC between your gateway and management server, and installed a new policy allowing essential communications and BGP to your gateway.

Re: BGP advice

Sandgirl — Fri, 10 Mar 2023 13:49:40 GMT

Thank you so much everyone!

After a lot of looking around, I have managed to unblock TCP 179 (as well as SSH and ICMP) after implementing the required rules and also turning off address spoofing.

I can now see the traffic being accepted by the firewall, and I can see SYN, ACK, Push, Finish and Reset. Tcpdump capture shows a flow - but only in one direction, from the ARS to the checkpoint, again I can see SYN, ACK, BGP Open... but then, I can see BGP Notification (which looks like the timeout waiting for the response) and FIN retransmission of FIN and Reset.

So the traffic seems to be getting there, but Checkpoint is still not willing to establish the BGP connection.

As mentioned before, I am trying to set it up with ARS, which only asks for IP and ASN.

Am I missing some setting on the Checkpoint that I need to enable to get it to work?

Re: BGP advice

the_rock — Fri, 10 Mar 2023 14:10:46 GMT

K, so something is not matching with the other side for sure. What do they see on Cisco side?

Can you send us output of below commands from clish? Please blur out any sensitive info.

Andy

From expert mode, just type clich and then run following commands (one by one)

show bgp errors

show bgp groups

show bgp memory

show bgp paths

show bgp peer

show bgp peers

show bgp routemap

show bgp stats

show bgp summary

Re: BGP advice

Sandgirl — Fri, 10 Mar 2023 14:29:23 GMT

Unfortunately I'm not using Cisco at the other end, it's ARS - Azure Route Server, which provides very little troubleshooting options.
I can see the BGP traffic from ARS arriving at Checkpoint, but I cannot see Checkpoint trying to communicate with ARS. So that's why I'm thinking I missed something out on Checkpoint.

Re: BGP advice

the_rock — Fri, 10 Mar 2023 14:36:22 GMT

Sorry @Sandgirl , I guess I cant spell properly...lol. I read ASR, so I assumed it was Cisco ASR, my bad, apologies. Ok, so if thats the case, yea, I know, Azure in general does not sadly provide many troubleshooting options at all. Can you actually run fw monitor -F flag I gave in my first response? Also, run ip r g command and then simply put IP address you are trying to reach, so we can confirm the routing part (example ip r g 8.8.8.8)

Andy

Re: BGP advice

Sandgirl — Fri, 10 Mar 2023 16:12:32 GMT

The fw -F command didn't return anything.

ip r g command helped - the traffic is going via the other interface, so once I change the settings on both ends, I can now see the traffic going both ways... although it's still failing... the connection is timing out on Checkpoint end it seems, the ARS sends a message that its Hold time has expired... Or am I reading this wrong?

Re: BGP advice

the_rock — Fri, 10 Mar 2023 16:16:41 GMT

If you check my very first response on your post, I gave an example of fw monitor -F command.

idea is this:

fw monitor -F "srcip,srcport,dstip,dstport,protocol" -F "srcip,srcport,dstip,dstport,protocol"

(just an example -> fw monitor -F "1.1.1.1,0,2.2.2.2,179,0" -F "2.2.2.2,0,1.1.1.1,179,0"

I really think that capture would help us here.

Andy

Re: BGP advice

Sandgirl — Mon, 13 Mar 2023 10:13:01 GMT

Hi Andy,

I run the fw monitor command for few minutes and this is what it returned:

[vs_0][ppak_0] eth1:i[40]: 10.0.4.4 -> 10.6.1.4 (TCP) len=40 id=27946
TCP: 64818 -> 179 ..R.A. seq=a532c7d3 ack=2ef94736
[vs_0][fw_0] eth1:i[40]: 10.0.4.4 -> 10.6.1.4 (TCP) len=40 id=27946
TCP: 64818 -> 179 ..R.A. seq=a532c7d3 ack=2ef94736
[vs_0][fw_0] eth1:I[40]: 10.0.4.4 -> 10.6.1.4 (TCP) len=40 id=27946
TCP: 64818 -> 179 ..R.A. seq=a532c7d3 ack=2ef94736
[vs_0][ppak_0] eth1:i[44]: 10.0.4.4 -> 10.6.1.4 (TCP) len=61 id=27947
TCP: 64938 -> 179 ...PA. seq=cce9b89b ack=80f42139
[vs_0][fw_1] eth1:i[44]: 10.0.4.4 -> 10.6.1.4 (TCP) len=61 id=27947
TCP: 64938 -> 179 ...PA. seq=cce9b89b ack=80f42139
[vs_0][fw_1] eth1:I[44]: 10.0.4.4 -> 10.6.1.4 (TCP) len=61 id=27947
TCP: 64938 -> 179 ...PA. seq=cce9b89b ack=80f42139
[vs_0][ppak_0] eth1:i[40]: 10.0.4.4 -> 10.6.1.4 (TCP) len=40 id=27948
TCP: 64938 -> 179 F...A. seq=cce9b8b0 ack=80f42139
[vs_0][fw_1] eth1:i[40]: 10.0.4.4 -> 10.6.1.4 (TCP) len=40 id=27948
TCP: 64938 -> 179 F...A. seq=cce9b8b0 ack=80f42139
[vs_0][fw_1] eth1:I[40]: 10.0.4.4 -> 10.6.1.4 (TCP) len=40 id=27948
TCP: 64938 -> 179 F...A. seq=cce9b8b0 ack=80f42139
[vs_0][ppak_0] eth1:i[40]: 10.0.4.4 -> 10.6.1.4 (TCP) len=40 id=27949
TCP: 64938 -> 179 F...A. seq=cce9b8b0 ack=80f42139
[vs_0][fw_1] eth1:i[40]: 10.0.4.4 -> 10.6.1.4 (TCP) len=40 id=27949
TCP: 64938 -> 179 F...A. seq=cce9b8b0 ack=80f42139
[vs_0][fw_1] eth1:I[40]: 10.0.4.4 -> 10.6.1.4 (TCP) len=40 id=27949
TCP: 64938 -> 179 F...A. seq=cce9b8b0 ack=80f42139
[vs_0][ppak_0] eth1:i[44]: 10.0.4.4 -> 10.6.1.4 (TCP) len=52 id=27950
TCP: 65059 -> 179 .S.... seq=f24889d5 ack=00000000
[vs_0][fw_0] eth1:i[44]: 10.0.4.4 -> 10.6.1.4 (TCP) len=52 id=27950
TCP: 65059 -> 179 .S.... seq=f24889d5 ack=00000000
[vs_0][fw_0] eth1:I[44]: 10.0.4.4 -> 10.6.1.4 (TCP) len=52 id=27950
TCP: 65059 -> 179 .S.... seq=f24889d5 ack=00000000
[vs_0][ppak_0] eth1:i[40]: 10.0.4.4 -> 10.6.1.4 (TCP) len=40 id=27951
TCP: 65059 -> 179 ....A. seq=f24889d6 ack=6ecba4fd
[vs_0][fw_0] eth1:i[40]: 10.0.4.4 -> 10.6.1.4 (TCP) len=40 id=27951
TCP: 65059 -> 179 ....A. seq=f24889d6 ack=6ecba4fd
[vs_0][fw_0] eth1:I[40]: 10.0.4.4 -> 10.6.1.4 (TCP) len=40 id=27951
TCP: 65059 -> 179 ....A. seq=f24889d6 ack=6ecba4fd
[vs_0][ppak_0] eth1:i[44]: 10.0.4.4 -> 10.6.1.4 (TCP) len=91 id=27952
TCP: 65059 -> 179 ...PA. seq=f24889d6 ack=6ecba4fd
[vs_0][fw_0] eth1:i[44]: 10.0.4.4 -> 10.6.1.4 (TCP) len=91 id=27952
TCP: 65059 -> 179 ...PA. seq=f24889d6 ack=6ecba4fd
[vs_0][fw_0] eth1:I[44]: 10.0.4.4 -> 10.6.1.4 (TCP) len=91 id=27952
TCP: 65059 -> 179 ...PA. seq=f24889d6 ack=6ecba4fd
[vs_0][ppak_0] eth1:i[40]: 10.0.4.4 -> 10.6.1.4 (TCP) len=40 id=27953
TCP: 64938 -> 179 ..R.A. seq=cce9b8b1 ack=80f42139
[vs_0][fw_1] eth1:i[40]: 10.0.4.4 -> 10.6.1.4 (TCP) len=40 id=27953
TCP: 64938 -> 179 ..R.A. seq=cce9b8b1 ack=80f42139
[vs_0][fw_1] eth1:I[40]: 10.0.4.4 -> 10.6.1.4 (TCP) len=40 id=27953
TCP: 64938 -> 179 ..R.A. seq=cce9b8b1 ack=80f42139

Re: BGP advice

Sandgirl — Mon, 13 Mar 2023 10:31:39 GMT

For some reason I cannot reply to this comment with the output from the fw monitor...

Re: BGP advice

Sandgirl — Mon, 13 Mar 2023 10:33:30 GMT

Hi Andy,

I've attached the output from fwmonitor as the text file - hopefully it will update this time.

Re: BGP advice

Sandgirl — Mon, 13 Mar 2023 10:55:04 GMT

When I've done the capture not limiting port to 179, I got another capture.

I've uploaded the second capture.

Re: BGP advice

Sandgirl — Mon, 13 Mar 2023 11:29:50 GMT

Thanks! That helped a lot! Although I'm still not able to establish BGP session between peers 😞

Re: BGP advice

the_rock — Mon, 13 Mar 2023 13:23:18 GMT

Small favor @Sandgirl ...I need to know your local CP ip address, as well as remote side. Also, can you send the actual pcap file, rather than txt, as I would like to examine it in wireshark. Please send me offline message or blur out any sensitive info.

Cheers.

Re: BGP advice

Sandgirl — Mon, 13 Mar 2023 14:16:31 GMT

I've attached the capture. It's my lab environment.

Re: BGP advice

the_rock — Mon, 13 Mar 2023 14:17:51 GMT

Thanks! Can you also please tell us the IP addresses involved (CP and other side)?

Cheers,

Andy