Cluster Interface for VLAN

[gurowar]

Good day all,

Hope everyone is ready for Thanksgiving!! I have a pair of 16200 FWs running R81.10 Jumbo Hotfix Take 156 in a HA configuration.

I have 2 sub-interfaces off of eth3-02:

eth3-02.1925

eth3-02.301

both are connected to a trunk port to my layer 3 switch which in turn has a vlan1925 that connects to the Metro-E.  What I am trying to do is vlan1925 is connected to a metro-E to 2 other locations and we are in the process of decomming one site called 1ND. So I thought all I would need to do is at the 1ND location I can just disconnect the cable of the metro-E and be done with it.  But when I did that I received an alert on the firewall

Alert: mail; OriginSicName: CN=FireWall01,O=CheckPointMgmt.omeda.local.oy6o8p; cluster_info: (ClusterXL) member 1 (192.168.255.253) is down (Interface Active Check on member 1 (192.168.255.253) detected a problem (eth3-02.1925 interface is down, 9 interfaces required, only 8 up).).; ProductName: VPN-1 & FireWall-1; ProductFamily: Network

 HeaderDateHour: 26Nov2024  9:35:34; ContentVersion: 5; HighLevelLogKey: N/A; Uuid: {0x0,0x0,0x0,0x0}; SequenceNum: 98; Action: ctl; Origin: FireWall01; IfDir: >; IfName: N/A; Alert: mail; OriginSicName: CN=FireWall01,O=CheckPointMgmt.omeda.local.oy6o8p; cluster_info: (ClusterXL) member 2 (192.168.255.254) is down.; ProductName: VPN-1 & FireWall-1; ProductFamily: Network

 HeaderDateHour: 26Nov2024  9:35:34; ContentVersion: 5; HighLevelLogKey: N/A; Uuid: {0x0,0x0,0x0,0x0}; SequenceNum: 1; Action: ctl; Origin: FireWall02; IfDir: >; IfName: N/A; Alert: mail; OriginSicName: CN=FireWall02,O=CheckPointMgmt.omeda.local.oy6o8p; cluster_info: (ClusterXL) member 2 (192.168.255.254) is down (Interface Active Check on member 2 (192.168.255.254) detected a problem (eth3-02.1925 interface is down, 9 interfaces required, only 8 up).).; ProductName: VPN-1 & FireWall-1; ProductFamily: Network

why would it say that the interface 1925 is down on the firewall when I disconnected the cable from 1ND which is 40 miles west? When I plugged it back in everything cleared.  Should I have disabled and remove the VLAN/IP address first from the 1ND location instead of just unplugging it?

diagram

FW ----eth3-02.1925  => Trunk port to layer 3 SW => Trunk port to Metro-E => Trunk Port 1ND VLAN1925

              eth3-02.301  

Thank you in advance!!!!

[PhoneBoy]

ClusterXL monitors the state of all interfaces on cluster members.
If one of the members loses access to one of their monitored interfaces...you'll get that message.
See also: https://support.checkpoint.com/results/sk/sk61323 

[gurowar]

So according to the SK61323 my set up is that it is monitoring both the high and low vlans.  I only have 2 vlans:

vlan302

vlan1925

[Expert@Firewall01:0]# fw ctl get int fwha_monitor_low_high_vlans
fwha_monitor_low_high_vlans = 1

so if I change it to 0 then only the lowest vlan will be monitored, in my case only vlan302.  So then I should be able to go to 1ND and unplug the cable and we should be good to go or am I reading this incorrectly?

[the_rock]

I believe if its set to 1, ONLY lowest and highest vlans are monitored. If its 0, then most likely just lowest. To answer your question, yes, thats my understanding as well, you should be good.

Andy

[emmap]

If the VLAN and switching layer is operating properly then the two cluster members should always see each other on the VLAN and that will satisfy the monitoring requirements. If the VLAN itself is disappearing from the trunk then yes, either change the monitoring or remove the interface (or set it to private/non-monitored in the topology section in the cluster object in smartconsole)

[gurowar]

All just wanted to update you all on this:

One more time 

diagram

FW ----eth3-02.1925  => Trunk port to layer 3 SW => Trunk port to Metro-E => Trunk Port 1ND VLAN1925

              eth3-02.301  

So my original issue was when I went to the 1ND location and I just removed the ethernet cable from the switch, we started to get alerts on the firewall saying that the interface, eth3-02.1925 went down so I immediately plugged the cable back in.  So taking a look at this closer each interface is running OSPF on it so the VLAN1925 in 1ND is running OSPF on it as well.  So I went in during the Thanksgiving weekend and I unplugged it, received the same alerts but this time I waited for OSPF to converge, probably overkill I left the office to get a cup of coffee, took all about 15 mins and check firewall and no longer seeing any alerts,  They all cleared and firewall was happpy.  So now the topology looks like this

diagram

FW ----eth3-02.1925  => Trunk port to layer 3 SW => Trunk port to Metro-E 

              eth3-02.301  

I have been monitoring this for a day or two and so far so go.  So I believe when I originally unplugged it and saw alerts panicked and plugged it back in, I didn't wait long enough for OSPF to converge.   Thank you al for all your help and probably I wasn't clear on what I was trying to do but this seems to be the resolution.

Thank you all!!!!

[Duane_Toler · Accepted solution]

ROUTED is a monitored PNOTE for ClusterXL.  When OSPF lost its DR (likely the peer on the other end of the link), OSPF registered the fault to ROUTED thus to ClusterXL.  You must not have had a BDR for some reason (ospf link-type point-to-point?).  When OSPF had a new DR, the PNOTE cleared with ROUTED.

[gurowar]

Hi Duane,

Between the sites there was a DR and BDR but you know come to think of it I did notice something weird about the BDR and told myself I would look into it later but later never came cause we ended up decomming the site.  But I agree with you about OSPF and gave it as much time to reconverge.  So I believe your right about that. 

Thank you!! 

[Bob_Zimmerman]

Are member 1 and member 2 on the same VLAN 1925? That is, are they able to talk to each other over that network?

Is VLAN 1925 the highest VLAN ID on the interface? You mentioned you have two subinterfaces, but you didn't say you have only two subinterfaces.

Are there any other devices with IP addresses on VLAN 1925?

[gurowar]

HI Bob,

Yes member 1 and 2 are the same VLAN1925, they are set up in a HA configuration 

member 1 - eth3-02.1925  192.168.5.2   ------Trunk----------- SW1 gi0/2 ---gi0/10 Trunk Metro-E ------------------ Chicago

   Firewall                     HA - 192.168.5.1       Stack SW                                                                           NYC

member 2 - eth3-02.1925 192.168.5.3  ------Trunk------------SW1  gi12

I hope that makes sense; this is a metro-E that connect 3 sites 

DataCeter

Chicago 

NYC

The firewall sits in the DC, each firewall has a physical connection the Switch Stack, from the same SW stack there is an interface that is a trunk port that connects to the Metro-E and via the Metro-E  Chi and NYC are connected.  So what happened is the other day I was in the Chi office and since we are decomming this site got lazy and figure I would just remove the cable from the switch that connects Chi office to the Metro-E.  When I did that I received alarms that I was expecting but the one the caught my attention was the alert on received from the Firewall saying that interface