Maestro Security Group not accessible via https (MHO 140 with 5600 Appliances)

[Yasushi_Kono1]

Dear Check Point Experts,

I am wondering whether you have seen this kind of issue: You configure a single site/single orchestrator (show maestro configuration [orchestrator-amount | orchestrator-site-amount]) environment. As I have configured a Security Group with an IP address in the same IP subnet as my Windows host is located, the next step would be to launch that Security Group via https. But the TLS connection cannot be established. This is misbehaviour I have never seen before. 

Do you have any hints?

Thanks in advance!

[simonemantovani]

Hello

are you able to ping the Security Group IP? Did you try a tcpdump on the management interface configured with the Security Group IP? Could you also perform a netstat -tapn | grep 443 when connected to the Security Group via ssh?

[Martijn]

Hi,

What version are you on? What kind of setup? Gateway, VSX Gateway or VSNext Gateway?

I would work my way up the OSI layers to see what is wrong.

I assume cabling has been checked by you.
Did you configure a bonding group (MAGG0) for management? If so, make sure this is Active/Backup with a Primary Interface configured. The ports on the switch should be normal access ports.

Does your computer learn the MAC-address of the Security Group?
Packets captures with tcpdump would be the next to investigate.

Did you manage to create the SMO in SmartConsole and install policy?
If so, do you see drops in SmartLog? Perform a zdebug if needed.

Hope these tips point you in the right direction. Let us know how it goes.

[Yasushi_Kono1]

Thanks to Simone and Martijn, 

I could extract the SG configuration from the MHO CLI with 
show maestro security-group id 1
and tried to ping the ip of the sg. And I established a second SSH connection to the MHO and did a 
tcpdump -i Mgmt2 -n arp
and could see a series of "ARP: who-has" entries.
So, Layer 1 is fine, but Layer 2 not. I put the interfaces in question to the appropriate VLAN in Cisco Catalyst, but the behaviour remained the same. I am not yet in the phase of configuring the SMO object on the Management Server. 
Because of these issues, I am not able to SSH to the SG.

[Martijn]

Hi,

Interface Mgmt2 seems to me a management interface of the MHO. The management interface of a Security Group is eth1-mgmt or magg0 if you have created a bonding group.

Are you performing the tcpdump on the MHO or on a SGM within the Security Group?

Martijn

[simonemantovani]

I agree with @Martijn ... You should connect to the Security group (through the MHO) and then start TCPDUMP using the management interface of the Security Group to check if you received any traffic.

[Yasushi_Kono1]

I get an "ARP who-has" entry in a quite irregular basis. But, ifconfig eth1-Mgmt1 reveals the receipt of packets. Quite interesting behaviour!

[simonemantovani]

Well ... do you see any mac address using the command arp -an | grep eth1-Mgmt1?

Are you able to ping any host on the network connected to eth1-Mgmt1?

[Yasushi_Kono1]

No, what I noticed now is that when I ping the Security Group from the MHO, you can see increasing amount of RX bytes but no changes to TX bytes. So, it gets the ping packets but does not reply to them. Never noticed this behaviour before.

[simonemantovani]

What is the IP address of the MHO?

Could you provide the routing table (netstat -r) of the security group?

So using TCPDUMP on the Security Group, when pinging from the MHO you should see these icmp request arriving, right?

[Yasushi_Kono1]

The MHO's address is 172.30.47.251. This is accessible from a different subnet in which my Windows host is located. 

The routing table looks like this:
[Expert@Bravo_SG-ch01-01:0]# netstat -rn
Kernel IP routing table
Destination       Gateway                Genmask           Flags         MSS   Windows       irtt  Iface
0.0.0.0               172.30.47.1                    0.0.0.0               UG                0   0                     0      eth1-Mgmt1
172.30.47.0              0.0.0.0        255.255.255.0               U                   0   0                     0      eth1-Mgmt1
192.0.2.0                   0.0.0.0       255.255.255.0               U                   0   0                     0       Sync
198.51.101.0             0.0.0.0   255.255.255.128               U                   0   0                     0      eth1-CIN

The icmp packets does not make it to the destination interface in a regular basis. So, now it seems for me that Layer 1 is somewhat affected. But not sure.

The IP of the Security Group is 172.30.47.247.

[simonemantovani]

another question ... what is the ouput of ethtool eth1-Mgmt1?

[Yasushi_Kono1]

 

[Martijn]

Does your MHO has an ARP entry for 172.30.47.247? If not, I would check layer 1.

- Cables
- SFP
- Switch port configuration

You mention increasing amount of RX packets, but no TX packets. Can you verify with tcpdump? Can you share the output?

[Yasushi_Kono1]

What I have noticed thanks to your question is that the auto-speed setting does not work. It says that the link speed is 10000base/full, but the switch is a Cisco Catalyst 8350 and does only support 1G. So, I set it to auto, but still remains at 10G. And manually configuring to 1G lead to an error message saying that config is locked. Although I entered "lock database override" many times. 

[simonemantovani]

As I remember you can set it to 10 or 1G not auto.

[Martijn]

Can you perform the following in the MHO?

show maestro port 1/1/1 qsfp-mode
It is 10G by default.
 
You can set it to 1G if you have the right SFP installed.

set maestro port 1/1/1 qsfp-mode 1G

[Yasushi_Kono1]

It turned out that the transceiver used was defective. So, some time, you got the MAC address in the ARP cache and some time, the cache remained empty! Now, everything works as designed. One odd think was that SIC to a Security Group could not be established, so I had to decrease the MTU size to 1486 Bytes. How odd is that?
Anyway, thanks a lot to you guys for your assistance!