Can't delete interfaces: 'This interface is used by the Dynamic Routing Protocol VRRP'

[Tim_Bernat]

Hi All,

I have recently started using two preconfigured Gateways  (Cluster of 2 4800 appliances, HA) with a new Management Server. All went well with config, new policy and upgrade from 77.30 to 80.10 (both SMS and GWs). However, right from the start I had a problem removing some interfaces:

 

I have removed all old VRRP interfaces in Gaia webuUI/High Availability/VRRP. Though these are still showing in the menu Gaia webuUI/High Availability/Advanced VRRP; with the message 'Simplified VRRP configuration is present', so I can't remove them there.

Doing >show vrrp interfaces does show the old unwanted interfaces as monitored circuits. 

Do you know how to remove the 'stuck' interfaces and advanced vrrp settings? I know it would have been easy with he old SMS, but that was/is not possible. 

All functions correctly; failover, policy etc. Interestingly one GW is stuck on a low effective priority (with no obvious problems/reasons for this) and I had to manually lower the other one for the HA to work as expected.

Thank you for any comments. 

[PhoneBoy]

You may be able to remove the offending entries with dbset (expert mode command).

But first we need to figure out what entries to whack.

We can do that with dbget.

From export mode, do something like (replace eth0 with the affected interface): dbget -r interface:eth0

Paste the output here, obscuring sensitive information as appropriate.

From this I should be able to help you with the correct dbset commands. 

Disclaimer: dbset directly interacts with the Gaia OS configuration database and does not do any sort of error checking.

Exercise appropriate caution when using this tool.

[Tim_Bernat]

Thanks Dameon,

since they are subinterfaces, there are a couple to list. Here is the sanitised output of dbget -r

GW1 and GW2 (IPs are 1 higher on the GW2)

interface:eth1.111:comments
interface:eth1.111:depend_on
interface:eth1.111:ipaddr:172.31.0.1
interface:eth1.111:ipaddr:172.31.0.1:mask
interface:eth1.111:label
interface:eth1.111:state

interface:eth1.112:comments
interface:eth1.112:depend_on
interface:eth1.112:ipaddr:150.0.0.1
interface:eth1.112:ipaddr:150.0.0.1:mask
interface:eth1.112:label
interface:eth1.112:state

interface:eth1.113:comments
interface:eth1.113:depend_on
interface:eth1.113:ipaddr:10.10.10.1
interface:eth1.113:ipaddr:10.10.10.1:mask
interface:eth1.113:label
interface:eth1.113:state

interface:eth2.221:comments
interface:eth2.221:depend_on
interface:eth2.221:ipaddr:10.10.20.2
interface:eth2.221:ipaddr:10.10.20.2:mask
interface:eth2.221:label
interface:eth2.221:state

interface:eth2.222:comments
interface:eth2.222:depend_on
interface:eth2.222:ipaddr:10.10.30.3
interface:eth2.222:ipaddr:10.10.30.3:mask
interface:eth2.222:label
interface:eth2.222:state

interface:eth2.223:comments
interface:eth2.223:depend_on
interface:eth2.223:ipaddr:10.10.40.4
interface:eth2.223:ipaddr:10.10.40.4:mask
interface:eth2.223:label
interface:eth2.223:state

interface:eth2.224:comments
interface:eth2.224:depend_on
interface:eth2.224:ipaddr:10.10.50.5
interface:eth2.224:ipaddr:10.10.50.5:mask
interface:eth2.224:label
interface:eth2.224:state

interface:eth2.225:comments
interface:eth2.225:depend_on
interface:eth2.225:ipaddr:10.10.60.6
interface:eth2.225:ipaddr:10.10.60.6:mask
interface:eth2.225:label
interface:eth2.225:state

None of these addresses are in use by static routes, etc.; the physical int eth1 and 2 are disabled (for VRRP to work correctly).

Also, show mcvr vrids shows only wanted VRIDs and IPs (3).

Thanks.

[PhoneBoy]

I thought that VRRP was attached to the interface.

Been a while, so I guess I got that wrong.

Maybe do a grep through /config/active and see if you can find the lines that reference the bogus VRRP info.

[Maarten_Sjouw]

In clish do:

show mcvr vrids and see what it shows, most probably there is still a backup address that is linked to the IP on the interface.

Other command will be show configuration mcvr  which will show you the VRRP backup address commands, when you see the offending IP, just remove it by using delete mcvr vrid # backup-address a.b.c.d  where # is the VRID number and a.b.c.d is the offending IP.

[Tim_Bernat]

Thanks Maarten,

I did check that, all looks good there, show mcvr vrids shows only wanted VRIDs and IPs (3).

Cheers.

[Maarten_Sjouw]

What are you actually trying to delete?

did you check the show configuration mcvr?

[Tim_Bernat]

I am trying to delete the actual subinterfaces (in Gaia webUI/Network Management/Network Interfaces). Please see the list in my reply to Dameon. These are all disabled. 

Yes, this was checked. Both commands,  show mcvr vrids and show configuration mcvrs, list expected IPs and VRIDs.

[Maarten_Sjouw]

And you are trying to delete the subinterface by:

delete interface eth1 vlan 111

that should remove the VLAN interface and all it's config.

[Tim_Bernat]

Thanks Dameon and Maarten,

yes Dameon , less /config/active | grep vrrp does show all the stuff I don't need. For example, for eth1.111:

routed:instance:default:vrrp:interface:eth1.111:mode monitoredcircuit
routed:instance:default:vrrp:interface:eth1.111:virtualrouter:13 t
routed:instance:default:vrrp:interface:eth1.111:virtualrouter:13:priority-delta 10
routed:instance:default:vrrp:interface:eth1.111:virtualrouter:13:priority 100
routed:instance:default:vrrp:interface:eth1.111:virtualrouter:13:advertiseinterval 1
routed:instance:default:vrrp:interface:eth1.111:virtualrouter:13:address:addr:172.31.0.1 t
routed:instance:default:vrrp:interface:eth1.111:virtualrouter:13:monitor:monif:eth2.224 t
routed:instance:default:vrrp:interface:eth1.111:virtualrouter:13:monitor:monif:eth2.224:priority 10
routed:instance:default:vrrp:interface:eth1.111:virtualrouter:13:monitor:monif:eth3 t
routed:instance:default:vrrp:interface:eth1.111:virtualrouter:13:monitor:monif:eth3:priority 10
routed:instance:default:vrrp:interface:eth1.111:virtualrouter:13:monitor:monif:eth1.112 t
routed:instance:default:vrrp:interface:eth1.111:virtualrouter:13:monitor:monif:eth1.112:priority 10
routed:instance:default:vrrp:interface:eth1.111:virtualrouter:13:monitor:monif:eth2.221 t
routed:instance:default:vrrp:interface:eth1.111:virtualrouter:13:monitor:monif:eth2.221:priority 10
routed:instance:default:vrrp:interface:eth1.111:virtualrouter:13:monitor:monif:eth1.113 t
routed:instance:default:vrrp:interface:eth1.111:virtualrouter:13:monitor:monif:eth1.113:priority 10
routed:instance:default:vrrp:interface:eth1.111:virtualrouter:13:monitor:monif:eth2.223 t
routed:instance:default:vrrp:interface:eth1.111:virtualrouter:13:monitor:monif:eth2.223:priority 10
routed:instance:default:vrrp:interface:eth1.111:virtualrouter:13:monitor:monif:eth2.222 t
routed:instance:default:vrrp:interface:eth1.111:virtualrouter:13:monitor:monif:eth2.222:priority 10 

In reply to Maarten, yes, I do just want the subinterfaces gone. Have not tried doing that from the CLI, just webUI (where I am getting that annoying message). I'm new to CP and, as with anything, it's a bit of a process. I'll try that (delete interface eth1 vlan 111)and let you know.

Also, what would be really helpful, would be to know if there is a way to 'no' (in a Cisco style) commands in CP. So I would want to be able do something like '#sh run' and then just 'no' what I don't need. Am I right in thinking that there is more than one running conifg file? In that case, am I right in thinking that I could not download the config file, stick it in an editor, remove all unwanted parts, and put it back on the box?

I guess my question is, 'how can I quickly get rid of all the lines?'.  configuration‌ 

Thanks

[Maarten_Sjouw]

show run ==> show configuration

No there is no 'no command' You have to either turn the command off or delete the part that is in your way, like the delete interface eth1 vlan 111 which will delete all related config from eth1.111 

You cannot however clear dependent items, like a VRRP backup-address, you will need to remove the backup address before you can delete the member address. 

[PhoneBoy]

I think you can disable VRRP on that interface with the expert command:

dbset routed:instance:default:vrrp:interface:eth1.111:virtualrouter:13

(Or at least disable that particular virtual router)

Note while there is a text copy of the configuration database in /config/active, the real database is somewhere else.

[Tim_Bernat]

Thanks, will try that later. 

[Tim_Bernat]

Hi Daemon,

tried that now, not sure it made any difference. The command is accepted, but the output of less /config/active | grep vrrp (and other show vrrp/mcvr commands is the same). I tried the command with made up ints/vlans and it accepted that too.

I can still see all those annoying ints/vlans being monitored:

MyGW> show vrrp interfaces

VRRP Interfaces
Interface eth3
    Number of virtual routers: 1
    Flags: MonitoredCircuitMode
    Authentication: NoAuthentication
    VRID 13
            State:                    Master               Time since transition:    21190
            BasePriority:             100                  Effective Priority:       50
            Master transitions:       1                    Flags:
            Advertisement interval:   1                    Router Dead Interval:     3
            VMAC Mode:                VRRP                 VMAC:                     00:00:xx:00:xx:xx
        Primary address: xx.xx.xx.196
        Number of Addresses: 1
            xx.xx.xx.195
        Monitored circuits
            eth1.111 (priority 10)
            eth1.900 (priority 10)
            eth1.902 (priority 10)
            eth2.612 (priority 10)
            eth2.614 (priority 10)
            eth2.616 (priority 10)
            eth2.620 (priority 10)

I would like to get rid of the highlighted part :  )

Cheers.

[PhoneBoy]

For each of the interfaces, you should be able to do something like:

dbset routed:instance:default:vrrp:interface:eth1.111:virtualrouter:13:monitor:monif:eth2.224

Note that when you're done hacking with dbset, you should issue a dbset :save to commit the changes.

[Tim_Bernat]

Cheers,

as you say, I can't delete the vlans: 

>

 delete interface eth2 vlan 612
NMSETH0059  VLAN eth2.612 cannot be deleted, it is in use by protocol VRRP 

>

I can delete old backup addresses with:

>

delete mcvr vrid 13 backup-address 172.31.0.1
WARNING this may take a while; please be patient

>

Once the addresses are removed, I still can't delete the vlans (same message); I think it's because VRRP has these monitored. How do I remove the lines that tell the VRID to monitor all these ints/vlans?
How to get rid of lines like these:

routed:instance:default:vrrp:interface:eth1.111:virtualrouter:13:monitor:monif:eth2.224 t
routed:instance:default:vrrp:interface:eth1.111:virtualrouter:13:monitor:monif:eth2.224:priority 10
routed:instance:default:vrrp:interface:eth1.111:virtualrouter:13:monitor:monif:eth3 t
routed:instance:default:vrrp:interface:eth1.111:virtualrouter:13:monitor:monif:eth3:priority 10

Can I just  show configuration, copy  it to notepad++, remove the lines and put the config back in (say using WinSCP)?

Thanks.

[Hugo_vd_Kooij]

This is so IPSO like 😉

Maybe the fastest trick to to enable the interface again first and then get it out of VRRP before you remove the interface.

But I recall more then one issue where I had to export the config, clean it up and then reload it on the gateway resulting in a short outage.

At least that is what worked on IPSO. Never done VRRP on GAIA but the issue looks very much the same.

[Tim_Bernat]

Hi Hugo,

cheers, yeah, I thought about it... But then I thought there's got to be an easier way ]:-> Also, I thought it would be interesting to 'find a way'...

When you say get the config out and back in, do you mean something like:
show configuration, copy the content to notepad++, remove the unwanted lines and put the config back in (say using WinSCP)?

Thanks

[PhoneBoy]

xpand in IPSO became confd in Gaia, so no shock or surprise

[Petr_Hantak]

I faced completely the same issue on R77.20 or R77.30 versions time to time on VRRP clusters. Even I removed subinterface from VRRP, disable it, etc. I was not able to delete it "Because it is under VRRP". Sometime it happens on the one cluster node only. 

Successful solution for me was alway reboot of affected gateway and after that I was able to remove it.

[Tim_Bernat]

Hi Petr,

thanks, rebooting has not worked for me, on either of the GWs. Thanks. 

[Petr_Hantak]

I'm so sorry that didn't help you with your issue. I look forward to the right solution for your case. Because I still take care about some VRRP clusters it could be helpful for me in future as well.

[Tim_Bernat]

I have exported the config using migrate export ...

I now want to extract the .tar from the .tgz, and the config from the .tar, and then put it back in using migrate import again. Is that a good idea?

Thanks

[Alexander_Kim]

Tim, 

Try to remove each line using dbset binary in expert mode. Just remove the value at the end of the binding :

#dbset routed:instance:default:vrrp:interface:eth1.111:virtualrouter:13:monitor:monif:eth2.224
#dbset routed:instance:default:vrrp:interface:eth1.111:virtualrouter:13:monitor:monif:eth2.224:priority
#dbset routed:instance:default:vrrp:interface:eth1.111:virtualrouter:13:monitor:monif:eth3
#dbset routed:instance:default:vrrp:interface:eth1.111:virtualrouter:13:monitor:monif:eth3:priority

#dbset save

Sundeep Mudgal‌

BR,

Kim

[Tim_Bernat]

Hi Alexander,

thank you for your reply. Which part of the command needs 'clipping'? I have tried removing 't', '10', and whole sections (cirtual router/monitor/etc.) + dbset save. It does not seem to be making any difference to the active config. What am I missing?

Thanks, Tim

[PhoneBoy]

Removing the 't' should "unset" the section. 

In any case, this should probably go through the TAC so we can look at it in more detail.

[Balamurugan_M]

Friends,

Can you please check the sk106396 if you have access.

Thanks, Bala

[Tim_Bernat]

Hi Bala,

will do, thank you. 

[Balamurugan_M · Accepted solution]

Hi Tim,

Good news!! 

Actually it is very simple. when i am playing around in my lab, i have found this.

"delete interface eth1 vlan 662 force"

Perform this change on both of your gateways if it is a cluster setup.

My test run command output below:

fw-test-01>delete interface eth1 vlan 10 force
Removed configurations for protocol(s) using eth1.10
VRRP

Thanks,

Bala

[Petr_Hantak]

I applied it today on one "corrupted" cluster member where I was not avble to remove it via WebUI and it works fine with force.