MHO-140 40/100Gbit Uplink to network devices

[MIwer]

Hello,

I have trouble to connect our MHO-140 to our switches with the 40/100G Ports.

We currently have several 1/10G connections running and are upgrading our backend.
To have some bandwidth flexibility we wanted to invest directly in 100G connections between our switches and our orchestrator, even if we won't increase throughput with current attached Security Gateways.
We have currently 2 MHOs which are distributed in our premise as single-site, single-orchestrator configuration.
Between those 2 the 100G interfaces make no problems but when we connect a switch with appropriate transcievers to the port it is not comming up. in the Gaia-Portal of the assigned SMO the Interface says "Link Speed: Not Supported" on the working Link to the other MHO the link speed is auto-negotiated to 10G/FullDuplex... which made me curious...

Is the Maestro Orchestrator even possible to utilize all of the 40/100G Ports? Or only 4 of them with the Beakout-Cables... I checked the Jumpstart videos to look for hints, but there are no examples mentioned exept breakouts...

Maybe someone might point me into the right direction...

Thanks in advance

Both MHO run following 
Software Version: R81.10 - Build 884
Kernel Version R81.10 - Build 794

Scheme (red= no link , green = link up):

SWITCH --100G-- MHO ----100G---- MHO --100G-- SWITCH
                                    I                                I
                                 10G                           10G
                                    I                                I
                              SWITCH                    SWITCH

[emmap]

You can use all 8 QSFP ports without breakout cables. What optics are you using? What does 'orch_stat -p' say for those ports from MHO expert mode?

It's normal that the SGMs report the speed as 10G on all interfaces.

[MIwer]

Hello @emmap ,

Thank you for those questions.

We are using Singe Mode Fiber Patch Cables. Our Transceivers are CPAC-TR-100LR-D and on the switches it is a corresponding 100G -LR4 Transceiver which is supported by the switch vendor. In my understanding this looks fine.

The "orch_stat -p" gave following:

Phys.Port - IF Name - Type - QSFP Mode - Admin State - Link State - Trans State - Op Speed - MTU

1/50/1 - eth1-51 - Uplink - 100G - UP - UP - PLUGGED - 100G_LR4_ER4 - 10240 (working to other orchestrator)
1/51/1 - eth1-53 - Uplink - 100G - DOWN - DOWN - PLUGGED - N/A - 10240 (obviously not working to switch)

EDIT:
with the idea 'the MTU setting might "confuse" the devices' we checked the max MTU of the switch (9216)
we tried to lower the MTU to the same value (9200) on both devices, but the MHO gave different vaule with 
"SHOW MAESTRO PORT MTU" - 9200 and "orch_stat -p" - 9204

and still no effect on the links...

[MIwer]

 

Found the issues:

two things came in on this problem...

1. I must have set the command 'set maestro port 1/51/1 admin-state down' somewhen in the past
 in result the port could never come up...
2. the used QSFPs are not compatible with each other, but if used on both ends, they work...

Info to the pictures:

Port 1/50/1 is Transceiver "CPAC-TR-100LR-D"
Port 1/51/1 is Transceiver "CISCO QSFP-100G-LR-S"

[emmap]

It looks like you changed the MTU back to default, so that's good. It's best not to change it on the MHOs. 

The Cisco QSFP is not supported to use in the MHO, would be best to stick to supported optics as if you have any issues with that port, TAC won't be able to offer much support with it. 

[MIwer]

Is there somewhere a list of compatible hardware to be connected to?
Like MHO140 with Transceiver QSFP LR needs this kind of Fiber quality and the other side can have one of these Transceivers (multiple vendor and quality types) because either our "CPAC-TR-100LR-D" are not supported on the switches or the "CISCO QSFP-100G-LR-S" are not supported one Orchestrator...

[emmap · Accepted solution]

Transceiver compatibility for CP stuff is here:

https://support.checkpoint.com/results/sk/sk92755

If you have a look in the accessories guide you'll find more information about connector types and supported cable specs:

https://www.checkpoint.com/downloads/products/check-point-appliance-accessory-guide.pdf

Between those you can see what we support, which will help with finding an equivalent optic that your other vendors support for their end of the wire.

[MIwer · Accepted solution]

I finally found it. I feel a little bit embarrassed now. 

CISCO uses 2 different LR transceivers "CISCO QSFP-100G-LR-S" and "CISCO QSFP-100G-LR4-S" which indicate different specs... Our company has the incompatible ones...

Thank you very much for your support.

[emmap]

No probs mate, thanks for the update and glad you found a resolution.

[Wolfgang]

@MIwer you wrote "e have currently 2 MHOs which are distributed in our premise as single-site, single-orchestrator configuration."

Are you running two different Maestro environments, each with only one MHO ? Or maybe you mean single site with dual-orchestrator ?

[MIwer]

Hello @Wolfgang ,

I meant what I wrote. Sadly my higher ups decided first for single site solution and puchased for this and later changed their mind to go dual site... so the initial planned single site dual orchestrator solution was spilt up... and budget for new hardware won't be in this year...

edit: grammar correction

[Wolfgang]

with two orchestrators you can run

- single site with dual orchestrator or

- dual site with single orchestrator per site

It's not supported to connect a single site / single orchestrator to another single site / single orchestrator

[MIwer]

FYI

We use 2 areas with directly connected fiber lines which travers "unsecure" premises in view of our IT-Security department. Thus they require definive encryption between those areas. So we deployed 2 separate single site single orchestrator arrangements, because they want to have hardware configuration mirrored in both areas... Layer 2 is transparrent and on layer 3 those arrangements have a "/30" subnet to "talk" to eachother securely.