OSPF on NSX

[Scott_Paisley]

We have OSPF running successfully on a number of hardware clusters connected to hardware switches. We are trying to run OSPF now on a cluster of virtual gateways in a VMware environment running NSX-T. As soon as we add an OSPF interface on the cluster Routed on the standby gateway fails and ClusterXL marks the member as down.

Anybody else seen or fixed this?

[Chris_Atkinson]

Are you saying the issue doesn't resolve when the configuration is set consistently on both cluster members and with which Version/Jumbo?

[Scott_Paisley]

R81.10 Jumbo 109

The gateways are in a cloning group so the configuration is consistemt across the gateways. Enabling OSPF on both gateways instantly disables ROUTED on the standby.

[PhoneBoy]

What does /var/log/messages and /var/log/routed* have to say when this occurs?

[Scott_Paisley]

messages

Sep 9 15:01:44 2023 EU-AZ-EDC-WAN-CKP-02 xpand[10207]: instance name is [default]
Sep 9 15:01:44 2023 EU-AZ-EDC-WAN-CKP-02 xpand[10207]: Configuration changed from localhost by user admin by the service rmbserver
Sep 9 15:01:44 2023 EU-AZ-EDC-WAN-CKP-02 xpand[10207]: finalize: routed conf file is [/etc/routed0.conf]
Sep 9 15:01:44 2023 EU-AZ-EDC-WAN-CKP-02 xpand[10207]: finalize: routed instance is [default]
Sep 9 15:01:44 2023 EU-AZ-EDC-WAN-CKP-02 xpand[10207]: moving /etc/cprd_syntax_test_default to /etc/routed0.conf
Sep 9 15:01:44 2023 EU-AZ-EDC-WAN-CKP-02 xpand[10207]: Using routed pid 15436 for 'default'
Sep 9 15:01:44 2023 EU-AZ-EDC-WAN-CKP-02 routed[13470]: [routed] NOTICE: task_reconfigure re-initializing from /etc/routed.conf
Sep 9 15:01:44 2023 EU-AZ-EDC-WAN-CKP-02 routed[13470]: [routed] NOTICE: parse_instance_only: my_instance_id -1 parsing instance default
Sep 9 15:01:44 2023 EU-AZ-EDC-WAN-CKP-02 routed[13470]: [routed] NOTICE: task_reconfigure reinitializing done
Sep 9 15:01:44 2023 EU-AZ-EDC-WAN-CKP-02 xpand[10207]: admin localhost t +routed:instance:default:ospf2:instance:default:area:0.0.0.0:interface:eth5 t
Sep 9 15:01:44 2023 EU-AZ-EDC-WAN-CKP-02 xpand[10207]: admin localhost t +routed:instance:default:ospf2:instance:default:interface:eth5:area 0.0.0.0
Sep 9 15:01:44 2023 EU-AZ-EDC-WAN-CKP-02 xpand[10207]: admin localhost t +routed:instance:default:ospf2:instance:default:area:0.0.0.0:interface:eth5:priority 1
Sep 9 15:01:44 2023 EU-AZ-EDC-WAN-CKP-02 xpand[10207]: admin localhost t +routed:instance:default:ospf2:instance:default:area:0.0.0.0:interface:eth5:auth:null t
Sep 9 15:01:44 2023 EU-AZ-EDC-WAN-CKP-02 xpand[10207]: admin localhost t +routed:instance:default:ospf2:instance:default:area:0.0.0.0:interface:eth5:authtype null
Sep 9 15:01:44 2023 EU-AZ-EDC-WAN-CKP-02 fwk: CLUS-211700-1: Remote member 2 (state STANDBY -> DOWN) | Reason: ROUTED PNOTE

routed_messages

Sep 9 15:01:45.956161 [routed] ERROR: OSPF2 instance default OspfInterfaceUp(4656): not starting protocol on interface 172.25.48.35(eth5)

[Chris_Atkinson]

Is IPV6/RD enabled on this cluster (sk102369)?

[Scott_Paisley]

IPv6 is not enabled

Router Discovery is not enabled

[the_rock]

If I were you, I would call TAC and see if you can do remote session, or at least provide further files/debugs for investigation. I had never seen issue like this myself before, either with OSPF or BGP.

Andy

[Scott_Paisley · Accepted solution]

We got it working. Details here for reference.

To recap, 3 clusters running in an ESX environment using NSX-T for the networking.

Each cluster configured as a separate cloning group so the configurations match.

On 2 of the 3 clusters, when we enable OSPF on a single interface, the standby cluster member fails with a ROUTED PNOTE.

The 'fix' was to break the cloning groups, reboot each member, reconfigure OSPF on each individual box, then enable the cloning group again. Now all 3 clusters are happy, although somehow the OSPF interface moved on one of the clusters