topic Re: R81.10 and BGP in Firewall and Security Management

R81.10 and BGP

pce17 — Sun, 13 Mar 2022 17:07:48 GMT

I have upgraded from R80.20 to R81.10 . I currently have 2 eBGP peers and 1 iBGP peer.

When switching from active to standby the (old active) now standby cluster member goes into down status briefly. ROUTED on the now standby member uses high (CPU 65% one cpu) for over 60 minutes.

Status so far,

- lots of debugs and cpinfo

- Checkpoint TAC's (ticket open 2 weeks) solution was to remove graceful restart which on causes all connections to be dropped and high CPU. I will continue to work with TAC .

FYI (In R80.20 the cluster lost all connections for 30 seconds when going from active to standby. Checkpoint said the solution was to turn on graceful restart. I turned on graceful restart and it resolved the dropping of all connections for 30 seconds in R80.20.)

But now Checkpoint TAC claims removing graceful restart will fix the issue.

Is anyone else using iBGP and R81.10? DO you have any ideas

Leo

Re: R81.10 and BGP

Chris_Atkinson — Sun, 13 Mar 2022 22:51:22 GMT

How many routes are in the BGP table and do the adjacent peer/s have GR configured on their side?

Which JHF take is used on this gateway/cluster?

Re: R81.10 and BGP

pce17 — Mon, 14 Mar 2022 00:38:00 GMT

400,000+ routes, GR is on both sides (see below) Members at JHF 30

PeerID AS Routes ActRts State InUpds OutUpds Uptime
12.122.NNN.NNN 7018 46809 40356 Established 11888 3 06:57:37
50.220.NNN.NNN 7922 7222 5110 Established 1936 3 06:57:01
4.53.NNN.NNN 21NNN 408564 392414 Established 126974 2 06:56:33

----- Peer 12.122
State Established (Uptime: 07:00:38)
Peer Type eBGP Peer
Remote AS 7018
Peer Capabilities IPv4 Unicast,Route Refresh,Cisco Route Refresh,Graceful Restart,4-Byte AS Extension
Our Capabilities IPv4 Unicast,Route Refresh,Graceful Restart,4-Byte AS Extension,Enhanced Route Refresh

----- Peer 50.220
State Established (Uptime: 07:00:02)
Peer Type eBGP Peer
Remote AS 7922
Peer Capabilities IPv4 Unicast,Route Refresh,Cisco Route Refresh,Graceful Restart,4-Byte AS Extension
Our Capabilities IPv4 Unicast,Route Refresh,Graceful Restart,4-Byte AS Extension,Enhanced Route Refresh

----- Peer 4.53
State Established (Uptime: 06:59:40)
Peer Type iBGP Peer
Remote AS 21NNN
Peer Capabilities IPv4 Unicast,Route Refresh,Cisco Route Refresh,Graceful Restart,4-Byte AS Extension,Enhanced Route Refresh
Our Capabilities IPv4 Unicast,Route Refresh,Graceful Restart,4-Byte AS Extension,Enhanced Route Refresh

Re: R81.10 and BGP

Chris_Atkinson — Mon, 14 Mar 2022 01:02:46 GMT

From an external view point 400,000 in iBGP seems high for most environments.

Has TAC provide guidance on if the situation would be improved by reducing this with employing route optimization strategies downstream?

Which model gateway appliances are used here out of interest?

Re: R81.10 and BGP

pce17 — Mon, 14 Mar 2022 01:13:56 GMT

In R80.20 I demonstrated to TAC that the issue went away when I filtered the iBGP routes. I mentioned the iBGP route size to TAC but TAC did not seem interested. I think TAC thinks it is a configuration issue. In R80.20 a custom ROUTED was created to fix the iBGP route issue. We are using open hardware.

Re: R81.10 and BGP

pce17 — Mon, 14 Mar 2022 01:27:29 GMT

My issue has been open with Sirius since February and TAC for two weeks. You have been asking some very good questions. I can try to adding the route filtering tomorrow and 6 - 7pm ET . That is our slow time during the week. I have assumed it is an iBGP and the number of routes from the beginning. TAC keeps on saying that was fixed in R80

Re: R81.10 and BGP

Chris_Atkinson — Mon, 14 Mar 2022 01:28:07 GMT

If you have the SR number for the same issue under R80.20 you should be able to request a portfix via TAC if a hotfix was provided.

Where possible I would suggest both strategies are employed to ensure stability.

Re: R81.10 and BGP

pce17 — Mon, 21 Mar 2022 17:52:29 GMT

Checkpoint R&D now claims that the standby cluster member in high CPU (ROUTED) for hours is caused by having ONLY a 1gig heartbeat interface. They said I need to upgrade to a 10 GIG heartbeat connection..... Very interesting CISCO says "Cisco typically recommends a minimum of 512 MB of RAM in the router to store a complete global BGP routing table from one BGP peer" 512MG needs a 10GIG connection?

Re: R81.10 and BGP

pce17 — Mon, 21 Mar 2022 17:57:04 GMT

My switch says the heartbeat interface max'ed out at 141Mbps ? 10 GIG?

Re: R81.10 and BGP

Chris_Atkinson — Wed, 23 Mar 2022 01:59:13 GMT

Can you please share your SR number for the TAC case with me in private?

(P.S. How did you go with the route filtering / summarization?)

Re: R81.10 and BGP

John_Fleming — Tue, 22 Mar 2022 04:46:08 GMT

holy zoinks bat scoob!

That is an impressive amount of routes. I'm assuming those aren't all 1918 prefixes?

Re: R81.10 and BGP

pce17 — Tue, 22 Mar 2022 13:06:43 GMT

They are all Internet routes. I have been filtering out the RFC1918 routes out since R77.30 (the good old days)

Re: R81.10 and BGP

pce17 — Tue, 22 Mar 2022 13:14:37 GMT

Todays update is that ROUTED crashed on the active cluster member (HA1) and the (now standby member HA1) CUL'ed non-stop for 4 hours and 20 minutes.

Re: R81.10 and BGP

the_rock — Tue, 22 Mar 2022 14:20:02 GMT

Message me privately if you need more help with this...I have BGP running in my lab with R81.10 and I had not seen these issues at all. Personally, I dont see logic in why you were asked to remove graceful restart option, that can only help in situation like yours.

Re: R81.10 and BGP

pce17 — Tue, 22 Mar 2022 16:20:14 GMT

Can you setup your lab to have one Peer configured as iBGP and then send in the full BGP route table? In R80.20 if I restricted the routes (only default) from my iBGP peer the issue went away. Never had any issues with R77.30.

Re: R81.10 and BGP

the_rock — Tue, 22 Mar 2022 17:19:40 GMT

I can, but might take some time, since I gave lab access to lots of my colleagues, as its very good setup. I will try do it some time this week. In the meantime, be free to message me privately and we can do remote tomorrow if you have time. Im in EST time zone (GMT -4 currently).

Re: R81.10 and BGP

Chris_Atkinson — Wed, 23 Mar 2022 02:35:16 GMT

Despite this occurrence, I want to come back to your original statement briefly.

"When switching from active to standby the (old active) now standby cluster member goes into down status briefly. ROUTED on the now standby member uses high (CPU 65% one cpu) for over 60 minutes."

What operational problem is this creating for you, how many cores/CPUs are assigned to the machine?

Re: R81.10 and BGP

the_rock — Wed, 23 Mar 2022 01:35:34 GMT

I see what you are saying...tested it in R81.10, same issue. I wonder if its some kind of bug...

Re: R81.10 and BGP

Chris_Atkinson — Wed, 23 Mar 2022 01:52:15 GMT

Out of interest what about your internal topology needs the full routing table versus fewer summarized routes?

Perhaps cBit is an alternate to GR that may assist per sk175923.

Re: R81.10 and BGP

pce17 — Thu, 31 Mar 2022 12:33:13 GMT

we need the full touting table or else the data is not routed correctly . Do to the usage of BGP the same subnets are used by many carriers