HA Cluster VLAN Interface Remove/Delete

[Martin_S_1]

Hi Experts, 

I am planning to delete an interface from a HA Cluster setup (R81.10) and I have come up with the following steps.. Do these steps look correct to you? It's very hard for me to know the correct way to do this having never done it before and I'm praying this is correct..

Brief action plan for removing an interface from cluster topology (R80.10 and above)

• Remove the Virtual IP address and Change the Interface to 'Private' in SmartConsole and push policy.
• check chaprob -a if for the change on both firewall gateway members.
• Disable clustering on standby gateway.
• delete the interface from standby gateway.
• delete the interface from active gateway.

• Delete the interface  from SmartConsole and push policy.
• Restart clustering on standby gateway.

Detailed action plan for removing an interface from cluster topology (R80.10 and above)

1. Perform these steps in SmartConsole (before removing an interface from Cluster object topology, set it to 'Private'):

Open the cluster object properties.

1. Go to the 'ClusterXL and VRRP' pane.
2. Under the 'Upon cluster member recovery' section, make sure the 'Maintain current active Cluster Member' option is selected.
3. Go to the 'Network Management' pane.
4. Highlight the interface by clicking on it once and then click on 'Edit' button.
5. Remove the Virtual IP address from the pair of the interfaces that should be removed from Cluster object topology.
6. In the 'Network Type' dropdown, select 'Private'
7. Click on 'OK' to apply the changes.
8. Proceed with installing the relevant policy to that cluster

1. On each cluster member:
1. Connect to the command line (over SSH, or console).
2. Log in to the Expert mode.
3. Run the cphaprob -a if command.
4. Check the 'Required number of interfaces' - the total number has to decrease by the number of interfaces that were configured as 'Private'.
   
   Example:
   If there were 11 interfaces
   And 1 interface was configured as 'Non-Monitored Private'
   Then now 'Required number of interfaces' should show 10 interfaces.
   Note: If the 'Required number of interfaces' did not decrease, then reboot the problematic cluster member.

1. Perform these steps on the Standby member:
1. Either stop the Clustering by running the 'cphastop' command, or bring this member administratively down by running the 'clusterXL_admin down' command.
2. Delete the interface via one of the following two ways:
• Gaia Web Portal:

Step	Description
1	In the navigation tree, click Network Management > Network Interfaces.
2	Select the correct Interface from the list and Click the 'Delete' button.

• Gaia Clish Mode:
  delete interface eth1 vlan 172

          save config

1. Perform these steps on the Active member:
1. Do NOT disable clustering. Check clustering is active using the 'cphaprob state' command.
1. Delete the interface via one of the following two ways:
• Gaia Web Portal:

Step	Description
1	In the navigation tree, click Network Management > Network Interfaces.
2	Select the correct Interface from the list and Click the 'Delete' button.

• Gaia Clish Mode:
  delete interface eth1 vlan 172

           save config

1. Perform these steps in SmartConsole:
1. Open Cluster object properties.
2. Go to the 'Network Management' and then highlight the interface by clicking on it once and then click on 'Edit' button.
3. Remove the interface from the Topology table from the cluster object.
4. Click on 'OK' to apply the changes.
5. Install the relevant policy onto the cluster object.

1. Perform these steps on Standby member:
1. Connect to the command line (over SSH, or console).
2. Log in to the Expert mode.
3. Either start the Clustering by running the 'cphastart' command, or bring this member administratively up by running the 'clusterXL_admin up' command.

1. Verify that the new interface was deleted from cluster topology - run this command on each cluster member:
   [Expert@HostName]# cphaprob -a if
   If the new interface was not deleted yet, then reboot each cluster member.

[Chris_Atkinson]

Hi Martin, sk57100 provides a reasonable reference for such activities.

As a side, Cluster XL monitors the highest and lowest active VLANs on an interface, is the VLAN ID in question either of those or somewhere in-between?

[Martin_S_1]

Hi Chris, 
I'm really glad you mentioned "Cluster XL monitors the highest and lowest active VLANs on an interface", because I didn't know this. So am I right in saying that ClusterXL will monitor itself through all physical interfaces, but if an interface is trunked with VLANs, then it elects to monitor itself through one particular VLAN on that interface? I just checked now and I can see the VLAN in question is indeed the lowest numbered VLAN on it's particular interface that it's currently being trunked on, yes. What are the implications and what steps must I follow? 

[Chris_Atkinson]

The ClusterXL admin guide and serval SK articles describe VLAN monitoring in detail.

The highest and lowest VLAN IDs on a trunk are both monitored by default (configurable).

In your situation it's important to not take shortcuts since the lowest VLAN ID is one that will directly trigger the interface active check/pnote of ClusterXL. 

[Martin_S_1]

The ClusterXL admin guide says the following:
"ClusterXL (including VSX) supports the Synchronization Network (CCP packets that carry Delta Sync information) only on the lowest VLAN ID (VLAN tag). For example, if three VLANs with IDs 10, 20 and 30 are configured on interface eth1, then you can use only the VLAN interface eth1.10 for the State Synchronization."

This leaves me with more questions than answers. Okay so here are my questions:

Is the state synchronization mentioned here the same type of synchronization offered by a 'Sync' interface?
If I have a dedicated 'Sync' interface, is the above statement about 'the lowest VLAN ID' irrelevant?
If the state synchronization mentioned here is a different type of synchronization offered by a 'Sync' interface, then what exactly is that difference, where I can learn more information about this difference, and what are the mitigation steps to avoid any issues if this 'lowest VLAN ID' were to be deleted?
Where exactly does it mention that the lowest VLAN ID is one that will directly trigger the interface active check/pnote of ClusterXL?

Any information you can shed is very  gratefully appreciated. 

[Chris_Atkinson]

For a given cluster there is typically only a single sync interface defined in the cluster topology either physical or VLAN.

Again, both the highest and lowest VLAN are monitored on a trunk port used as a data interface by default.

[the_rock]

Just to tell you something from my own experience...when you add clans in web UI, and you go to dashboard, do NOT click "get interface with topology", as that can mess up everything. Just do get interfaces without topology and I would also recommend to set topology as "network defined by routes", as that calculates topology behind the interface.

[Martin_S_1]

@the_rock - thank you for this note about your experience with adding VLANs. I did actually see this in another post, that adding "WITH topology" will cause problems. I'll make sure I add new interfaces WITHOUT topology, yes. Many thanks for this. 

[Alan_S]

Hi - could you not simply delete the interface in Cluster XL and then delete the interfaces on the gateway?

[Chris_Atkinson]

In very simple terms you have described what sk57100 documents as the removal process.

[Alan_S]

Thanks Chris. What I was wondering was, could you delete the cluster interface, and remove/disable the interface on the gateways, without entering "cphastop" and then "cphastart" on the standby gateway?

[Martin_S_1]

This was the exact process I used. I did not need to use cpstop/cpstart.

1. Backup both gateways

Take backups and snapshots. Save to external location.

2. Disable the interfaces from Solarwinds Monitoring.

3. Edit the Cluster object in SMS

Go into Cluster member tab, change the IP addresses for both cluster members to the new IP addresses.

4. Perform a SIC test in SMS to ensure comms to/from both gateways function as expected.
SIC was working fine.

5. Update the Alias URL (Platform Portal URL) within Smart Centre.
1. Place the new URLs in manually for each gateway.

6. Push an blank/empty Policy change to the firewall cluster in SMS

7. Change the Management IP address of the gateway in GAIA

Update each gateway to the new management interface in CLISH

Update the DNS host file entry for the firewall hostname/IP mapping:

8. Check ID Awareness PDP to ensure the firewalls are still connected to ID sharing peers:

pdp connections pep

pep show pdp all

Perform validation testing

[Alan_S]

Thanks - I was just wondering. I am due to remove a cluster object and reinstate it on a different interface on our firewall gateways. Would you recommend following that process?

[Martin_S_1]

Sorry, no. I didn't read the title of this thread properly. That process I just gave you was for changing a Management interface to a new interface on the same firewall. Do not follow it for what you are doing, no. 

[Alan_S]

Thanks Martin. So, would you recommend following the process outlined earlier in this post, or just delete the cluster interface, reconfigure the physical interfaces on the gateways, and then add the cluster interface referencing the new physical interfaces?

[Martin_S_1]

yes, follow it. You will need to cpstop / cpstart. 

[Alan_S]

Thanks - so that is "cphastop/cphastart" on the standby gateway? Just making sure to cover all bases.

[Martin_S_1]

yes, exactly as I have written it in bold in the original post. 

[Alan_S]

Thank you Martin. Much appreciated.