CheckPoint 6500 channel-group to Cisco Nexus 9Ks VPC

Quentin_Antrim — Wed, 22 May 2019 23:39:48 GMT

I have a pair of CheckPoint 6500 appliances, GAIA R80.10. Each appliance is connected to a pair of Cisco Nexus 9k switches using a VPC port-channel. Thus I have a channel-group on each firewall consisting of two slave interfaces with the IP address on the bond interface. I am using 802.3ad (LACP) and jumbo frames (9216).

Essentially, the bond interface is DOWN on the appliances, but UP on the Cisco N9ks.

Cisco:

chw_srvrm_dcswt1# sh vp brief
Legend:
(*) - local vPC is down, forwarding via vPC peer-link

vPC domain id : 1
Peer status : peer adjacency formed ok
vPC keep-alive status : peer is alive
Configuration consistency status : success
Per-vlan consistency status : success
Type-2 consistency status : success
vPC role : primary
Number of vPCs configured : 4
Peer Gateway : Enabled
Dual-active excluded VLANs : -
Graceful Consistency Check : Enabled
Auto-recovery status : Disabled
Delay-restore status : Timer is off.(timeout = 30s)
Delay-restore SVI status : Timer is off.(timeout = 10s)
Operational Layer3 Peer-router : Enabled

vPC status
----------------------------------------------------------------------------
Id Port Status Consistency Reason Active vlans
-- ------------ ------ ----------- ------ ---------------
41 Po41 up success success 503

chw_srvrm_dcswt1# sh port-channel summ
Flags: D - Down P - Up in port-channel (members)
S - Switched R - Routed
U - Up (port-channel)
--------------------------------------------------------------------------------
Group Port- Type Protocol Member Ports
Channel
--------------------------------------------------------------------------------
41 Po41(SU) Eth LACP Eth1/17(P)

chw_srvrm_dcswt1# sh run int e1/17

interface Ethernet1/17
description OOB FW
switchport access vlan 503
mtu 9216
channel-group 41 mode active

chw_srvrm_dcswt1# sh run int port-channel 41

interface port-channel41
description OOB Port-Channel 41
switchport access vlan 503
mtu 9216
vpc 41

This is the same on both Cisco Nexus 9k switches.

CheckPoint

[Expert@chw_pbx_bbfw1:0]# cat /proc/net/bonding/bond41
Ethernet Channel Bonding Driver: v3.2.4 (January 28, 2008)

Bonding Mode: IEEE 802.3ad Dynamic link aggregation
Transmit Hash Policy: layer3+4 (1)
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 200
Down Delay (ms): 200

802.3ad info
LACP rate: slow
Active Aggregator Info:
Aggregator ID: 2
Number of ports: 2
Actor Key: 17
Partner Key: 32809
Partner Mac Address: 00:23:04:ee:be:01

Slave Interface: eth1-01
MII Status: up
Link Failure Count: 1
Permanent HW addr: 00:1c:7f:67:2e:5c
Aggregator ID: 2

Slave Interface: eth1-02
MII Status: up
Link Failure Count: 5
Permanent HW addr: 00:1c:7f:67:2e:5d
Aggregator ID: 2

[Expert@chw_pbx_bbfw1:0]# cphaconf show_bond bond41

Bond name: bond41
Bond mode: Load Sharing
Bond status: DOWN
Balancing mode: 802.3ad Layer3+4 Load Balancing
Configured slave interfaces: 2
In use slave interfaces: 2
Required slave interfaces: 1

As you can see, the Cisco port-channel is UP, but the CheckPoint bond interface is DOWN. I have tried both the L2 and L3+4 transmit hash policy setting, MTU 1500, and both LACP rate of fast and slow with no difference.

This should really be very simple and there should be no reason why this bond interface should be down, but I'm looking for any suggestions on what the problem could be and what I'm missing.

By the way, I do have a ticket open with CheckPoint tech support, but with no solution so far.

Thanks.

Quentin

Re: CheckPoint 6500 channel-group to Cisco Nexus 9Ks VPC

Peter_Lyndley — Sun, 07 Jul 2019 08:16:22 GMT

I'd be curious if you ever got a reply to this. We are seeing something similar in another environment

Re: CheckPoint 6500 channel-group to Cisco Nexus 9Ks VPC

Matt_Killeen — Mon, 08 Jul 2019 13:57:24 GMT

We had a similar issue after upgrading CP 15600 two member cluster running r77.30 to r80.20

In our case the bond interface was flapping due to CCP packets not being received on the bond interfaces.

The issue could be resolved by changing the cluster from multicast mode to unicast mode.

This was not an option for our three member clusters where multicast was required so, with no resolution to the issue, we elected to break the bond and use a single interface.

Re: CheckPoint 6500 channel-group to Cisco Nexus 9Ks VPC

Daniel_Taney — Mon, 08 Jul 2019 16:16:36 GMT

Did you see this sk? Seems like there may be a known Cisco bug.

It seems the workaround is to change the cluster mode to broadcast.

Re: CheckPoint 6500 channel-group to Cisco Nexus 9Ks VPC

Matt_Killeen — Mon, 08 Jul 2019 16:22:08 GMT

@Daniel_Taney - yes, thanks but seen that one. Unfortunately, we're running Cisco OTV across geographical sites so broadcast mode can't be implemented in our infrastructure.

Re: CheckPoint 6500 channel-group to Cisco Nexus 9Ks VPC

Martin_Raska — Tue, 09 Jul 2019 08:00:48 GMT

Try to upgrade to R80.20 and set cluster mode to unicast

topic Re: CheckPoint 6500 channel-group to Cisco Nexus 9Ks VPC in Firewall and Security Management

CheckPoint 6500 channel-group to Cisco Nexus 9Ks VPC

Re: CheckPoint 6500 channel-group to Cisco Nexus 9Ks VPC

Re: CheckPoint 6500 channel-group to Cisco Nexus 9Ks VPC

Re: CheckPoint 6500 channel-group to Cisco Nexus 9Ks VPC

Re: CheckPoint 6500 channel-group to Cisco Nexus 9Ks VPC

Re: CheckPoint 6500 channel-group to Cisco Nexus 9Ks VPC