Create a Post
cancel
Showing results for 
Search instead for 
Did you mean: 
Simon_Macpherso
Advisor

SGM Member 2 Down

Hi,

 

I'm in the process of deploying 2 x MHO-140 Orchestrators and 2 x 6200 plus gateways. 

All cabling has been completed.

Licenses have been manually applied to each gateway. 

Both gateways are showing in Orchestrators -> unassigned gateways

MHO-140 Orchestrators are running R80.20SP GA take 295

6200 Gateways are running R80.30SP GA take 49 (we were advised R80.30SP could be installed on the gateways)

Ive created one security group, assigned one gateway and assigned mgmt and uplink interfaces and vlans. Applied topology.

I added a second gateway and assigned mgmt and uplink interfaces and vlans. Applied topology.

 

Issue

Member 1_2 SGM is DOWN.

On member 1, the output of show smo verifiers report shows a chassis error.

On member 2, the output of show smo verifiers report shows a chassis error and a MAC address verification error.

Below I’ve included the output of  asg stat -v on member 1,  show smo verifiers report on member 1 and 2 and mac_verifier -v on member 2.

 

[Local SGM] fw-adi-maestro-ch01-01> asg stat -v

 

--------------------------------------------------------------------------------

| System Status - Maestro                                                      |

--------------------------------------------------------------------------------

| Up time                     | 22:47:19 hours                                 |

| SGMs                        | 1 / 2 !                                        |

| Version                     | R80.30SP (Build Number 47)                     |

--------------------------------------------------------------------------------

| SGM ID                              Chassis 1                                |

|                                     ACTIVE                                   |

--------------------------------------------------------------------------------

|  1                                    ACTIVE(!)                              |

|  2                                    DOWN                                   |

--------------------------------------------------------------------------------

| Chassis Parameters                                                           |

--------------------------------------------------------------------------------

| Unit        |                      Chassis 1                        | Weight |

--------------------------------------------------------------------------------

| SGMs        |                        1 / 2   !                      |   6    |

| Ports       |                                                       |        |

|   Standard  |                        0 / 0                          |  11    |

|   Bond      |                        0 / 0                          |  11    |

|   Other     |                        0 / 0                          |   6    |

| Sensors     |                                                       |        |

|   SSMs      |                        2 / 2                          |  11    |

|             |                                                       |        |

| Grade       |                       28 / 34  !                      |   -    |

--------------------------------------------------------------------------------

| Synchronization                                                              |

|     Sync to Active chassis:    Enabled                                       |

--------------------------------------------------------------------------------

 

 

[Local SGM] fw-adi-maestro-ch01-01> show smo verifiers report

Duration of tests vary and may take a few minutes to complete

 

--------------------------------------------------------------------------------

| Tests Status                                                                 |

--------------------------------------------------------------------------------

| ID | Title              | Result     | Reason                                |

--------------------------------------------------------------------------------

| System Components                                                            |

--------------------------------------------------------------------------------

|  1 | System Health      | Failed (!) | (1)Chassis 1 error                    |

|  2 | Resources          | Passed     |                                       |

|  3 | Software Provision | Passed     |                                       |

|  4 | Media Details      | Passed     |                                       |

|  5 | SSD Health         | Passed     |                                       |

|  6 | Firmware Verifier  | Passed     |                                       |

--------------------------------------------------------------------------------

| Policy and Configuration                                                     |

--------------------------------------------------------------------------------

|  7 | Distribution Mode  | Passed     |                                       |

|  8 | DXL Balance        | Passed     |                                       |

|  9 | Policy             | Passed     |                                       |

| 10 | AMW Policy         | Passed     | (1)Not configured                     |

| 11 | SWB Updates        | Passed     | (1)Not configured                     |

| 12 | Installation       | Passed     |                                       |

| 13 | Security Group     | Passed     |                                       |

| 14 | Cores Distribution | Passed     |                                       |

| 15 | Clock              | Passed     |                                       |

| 16 | Licenses           | Passed     |                                       |

| 17 | IPS Enhancement    | Passed     |                                       |

| 18 | Configuration File | Passed     |                                       |

--------------------------------------------------------------------------------

| Networking                                                                   |

--------------------------------------------------------------------------------

| 19 | MAC Setting        | Passed     |                                       |

| 20 | ARP Consistency    | Failed (!) | (1)Verifier error - Check raw output  |

| 21 | Bond               | Failed (!) |                                       |

| 22 | IPv4 Route         | Passed     |                                       |

| 23 | IPv6 Route         | Passed     | (1)Not configured                     |

| 24 | Dynamic Routing    | Passed     | (1)Not configured                     |

| 25 | Local ARP          | Passed     | (1)Not configured                     |

| 26 | IGMP Consistency   | Passed     | (1)Not configured                     |

| 27 | PIM Neighbors      | Passed     | (1)Not configured                     |

--------------------------------------------------------------------------------

| Misc                                                                         |

--------------------------------------------------------------------------------

| 28 | Core Dumps         | Passed     |                                       |

| 29 | Processes          | Passed     |                                       |

| 30 | Performance hogs   | Failed (!) |                                       |

--------------------------------------------------------------------------------

| Tests Summary                                                                |

--------------------------------------------------------------------------------

| Passed: 26/30 tests                                                          |

| Run: "show smo verifiers list id 1,20,21,30" to view a complete list of fail |

| ed tests                                                                     |

| Output file: /var/log/verifier_sum.1-30.2020-11-04_10-10-30.txt              |

| Run "show smo verifiers last-run print" to display verbose output            |

--------------------------------------------------------------------------------

 

[Local SGM] fw-adi-maestro-ch01-02> show smo verifiers report

Duration of tests vary and may take a few minutes to complete

 

--------------------------------------------------------------------------------

| Tests Status                                                                 |

--------------------------------------------------------------------------------

| ID | Title              | Result     | Reason                                |

--------------------------------------------------------------------------------

| System Components                                                            |

--------------------------------------------------------------------------------

|  1 | System Health      | Failed (!) | (1)Chassis 1 error                    |

|  2 | Resources          | Passed     |                                       |

|  3 | Software Provision | Passed     |                                       |

|  4 | Media Details      | Passed     |                                       |

|  5 | SSD Health         | Passed     |                                       |

|  6 | Firmware Verifier  | Passed     |                                       |

--------------------------------------------------------------------------------

| Policy and Configuration                                                     |

--------------------------------------------------------------------------------

|  7 | Distribution Mode  | Passed     |                                       |

|  8 | DXL Balance        | Passed     |                                       |

|  9 | Policy             | Passed     |                                       |

| 10 | AMW Policy         | Passed     | (1)Not configured                     |

| 11 | SWB Updates        | Passed     | (1)Not configured                     |

| 12 | Installation       | Passed     |                                       |

| 13 | Security Group     | Passed     |                                       |

| 14 | Cores Distribution | Passed     |                                       |

| 15 | Clock              | Passed     |                                       |

| 16 | Licenses           | Passed     |                                       |

| 17 | IPS Enhancement    | Passed     |                                       |

| 18 | Configuration File | Passed     |                                       |

--------------------------------------------------------------------------------

| Networking                                                                   |

--------------------------------------------------------------------------------

| 19 | MAC Setting        |

Failed (!) | (1)Inconsistent Firewall value and MA |

|    |                    |            | C Address                             |

| 20 | ARP Consistency    | Failed (!) | (1)Verifier error - Check raw output  |

| 21 | Bond               | Passed     | (1)Not configured                     |

| 22 | IPv4 Route         | Passed     |                                       |

| 23 | IPv6 Route         | Passed     | (1)Not configured                     |

| 24 | Dynamic Routing    | Passed     | (1)Not configured                     |

| 25 | Local ARP          | Passed     | (1)Not configured                     |

| 26 | IGMP Consistency   | Passed     | (1)Not configured                     |

| 27 | PIM Neighbors      | Passed     | (1)Not configured                     |

--------------------------------------------------------------------------------

| Misc                                                                         |

--------------------------------------------------------------------------------

| 28 | Core Dumps         | Passed     |                                       |

| 29 | Processes          | Passed     |                                       |

| 30 | Performance hogs   | Failed (!) |                                       |

--------------------------------------------------------------------------------

| Tests Summary                                                                |

--------------------------------------------------------------------------------

| Passed: 26/30 tests                                                          |

| Run: "show smo verifiers list id 1,19,20,30" to view a complete list of fail |

| ed tests                                                                     |

| Output file: /var/log/verifier_sum.1-30.2020-11-04_05-08-30.txt              |

| Run "show smo verifiers last-run print" to display verbose output            |

--------------------------------------------------------------------------------

 

 

[Expert@fw-adi-maestro-ch01-02:0]# mac_verifier -v

--------------------------------------------------------------------------------

Collecting information from SGMs...

^C

[Expert@fw-adi-maestro-ch01-02:0]# mac_verifier -v

--------------------------------------------------------------------------------

Collecting information from SGMs...

--------------------------------------------------------------------------------

Verifying FW1 mac magic value on all SGMs...

FW1 mac magic value on all SGMs:

Command completed successfully

 

Success

--------------------------------------------------------------------------------

Verifying IPV4 and IPV6 kernel values...

IPV6 is not enabled

Success

--------------------------------------------------------------------------------

Verifying FW1 mac magic value in /etc/smodb.json...

FW1 mac magic value and /etc/smodb.json value are the same (217)

Success

--------------------------------------------------------------------------------

Verifying MAC address on local chassis (Chassis 1)...

-*- 1 blade: 1_01 -*-

BPEth0      MAC address of BPEth0 is correct

 

-*- 1 blade: 1_01 -*-

BPEth1      MAC address of BPEth1 is correct

 

MAC address inconsistency found on interface eth1-05 (FW1 value is different)

-*- 1 blade: 1_01 -*-

eth1-05 00:1c:7f:82:05:d9

0 Kudos
11 Replies
Maarten_Sjouw
Champion
Champion

Keep in mind that the up / down state will only change when you apply a policy, so first install the policy before you start checking the state. When the gateway is shown in the Maestro configuration page, you should be just fine.

Regards, Maarten
0 Kudos
Simon_Macpherso
Advisor

Hi,

My understanding (from the r80.30sp admin guide) is that the policy is first installed on the SMO master. The SMO master then copies the policy and security appliances configuration to all security appliances in the UP state. When the security appliance enters the UP state, it automatically gets the installed policy and configurations that are installed, from the SMO. 

As member 1_2 is currently in down state, will the master be able to copy the policy to the other member?

Regards,

Simon

0 Kudos
Chris_Atkinson
Employee Employee
Employee

To confirm all SGMs were present when the policy was installed?

CCSM R77/R80/ELITE
0 Kudos
Gomboragchaa
Advisor

Hi,

 

I've exact same appliances, versions and similar error. ARP Consistency error.

| 20 | ARP Consistency | Failed (!) | (1)MAC inconsistencies

==============================
ARP Consistency:
==============================

Address           HWtype           HWaddress                Flags Mask           Iface              SGMs
10.10.0.100       .                       (incomplete)                .                            bond5.5        1_02
10.10.0.100      ether               00:0c:29:80:57:77        C                          bond5.5         1_01

ARP is incomplete on SGM2 and some traffics dropping. Still can't solving this issue.... I guess there is something wrong sync between SGMs and it could be the kernel or os issue...

0 Kudos
Maarten_Sjouw
Champion
Champion

Did you make sure with lldpctl that all connections are up and running?

Regards, Maarten
0 Kudos
anikaralam
Participant

What is the solution, I running through similar issues with MHO 140

show smo verifiers report
Duration of tests vary and may take a few minutes to complete

--------------------------------------------------------------------------------
| Tests Status |
--------------------------------------------------------------------------------
| ID | Title | Result | Reason |
--------------------------------------------------------------------------------
| System Components |
--------------------------------------------------------------------------------
| 1 | System Health | Failed (!) | (1)Chassis 1 error |
| 2 | Resources | Passed | |
| 3 | Software Provision | Passed | |
| 4 | Media Details | Passed | |
| 5 | SSD Health | Passed | |
| 6 | Firmware Verifier | Passed | |
--------------------------------------------------------------------------------
| Policy and Configuration |
--------------------------------------------------------------------------------
| 7 | Distribution Mode | Passed | |
| 8 | DXL Balance | Passed | |
| 9 | Policy | Failed (!) | |
| 10 | AMW Policy | Passed | (1)Not configured |
| 11 | SWB Updates | Passed | (1)Not configured |
| 12 | Installation | Passed | |
| 13 | Security Group | Passed | |
| 14 | Cores Distribution | Passed | |
| 15 | Clock | Passed | |
| 16 | Licenses | Failed (!) | (1)Licenses differ across members |
| 17 | Configuration File | Passed | |
--------------------------------------------------------------------------------
| Networking |
--------------------------------------------------------------------------------
| 18 | MAC Setting |
Failed (!) | (1)Inconsistent Firewall value and MA |
| | | | C Address |
| 19 | ARP Consistency | Failed (!) | (1)Verifier error - Check raw output |
| 20 | Bond | Failed (!) | |
| 21 | IPv4 Route | Passed | |
| 22 | IPv6 Route | Passed | (1)Not configured |
| 23 | Dynamic Routing | Passed | (1)Not configured |
| 24 | Local ARP | Passed | (1)Not configured |
| 25 | IGMP Consistency | Passed | (1)Not configured |
| 26 | PIM Neighbors | Passed | (1)Not configured |
--------------------------------------------------------------------------------
| Misc |
--------------------------------------------------------------------------------
| 27 | Core Dumps | Passed | |
| 28 | Performance hogs | Passed | |
--------------------------------------------------------------------------------
| Tests Summary |
--------------------------------------------------------------------------------
| Passed: 22/28 tests |

0 Kudos
emmap
Employee
Employee

Is this a new build? Is there a policy installed? What're the details for that failed check? If you check the details of the failed tests they will generally give you a good idea of what it's alerting on.

0 Kudos
anikaralam
Participant

This issue is resolved yesterday. Thanks Emmap for the response.

0 Kudos
khansaijaz
Explorer

Hello Anikarlam

May i know what was done to resolve the issue ? i am in similar situation

0 Kudos
Bilguunbileg
Explorer

Dear Anikaralam,

How to fix this issue? Please, i am in similar error.

0 Kudos
anikaralam
Participant

Issue resolved after installing policy