Interface order on VMWare

[Andrew_Tindall]

I'm having a issue with building a pair of HA gateways (81.10) for a test lab on VMWare 7.  This environment only breaks out to the wider network via it's 'outside' interface with all of the other networks and interfaces existing solely within VMWare and not breaking out to any physical switches.  

When building the Gateways with multiple interfaces the order for the vmnic and ethx do not match up.

This seems some what random as one GW build will have the first 3 interfaces in order before seeing issues and the second will have issues from eth0/vmnic1 onwards.

I have seen sk69621 for re-naming the interfaces, but the issue is in identifying the 'incorrect' order in the first place. I've seen a couple of posts that reference this SK but non that seem to have the solution to identifying the order of the interfaces.

Of course the irony of all of this is that the test lab is there to enable us to test upgrades to the firewalls for any issues before rolling out to production, which are appliances so don't suffer from this issue.

[Timothy_Hall]

Yeah good luck with that, I've been setting up VMWare training environments for 20 years and still get burned by an unexpected mapping of interface name to NIC adapter in VMWare Workstation occasionally.  On a real server the assignment of interface names to physical cards seems to be based on the ordering/architecture of the system bus, but in VMWare it seems to be ordered by the hardware path/ID which is usually predictable but not always.  The best command I've found to look at the mapping is lshw -short -class network like this from expert mode:

Expert@gw-5ad2f3:0]# lshw -short -class network
H/W path Device Class Description
=====================================================
/0/100/15/0    eth0 network VMXNET3 Ethernet Controller
/0/100/15.1/0 eth1 network VMXNET3 Ethernet Controller
/0/100/16/0    eth2 network VMXNET3 Ethernet Controller
/0/100/17/0    eth3 network VMXNET3 Ethernet Controller
/0/100/18/0    eth4 network VMXNET3 Ethernet Controller
/2 gretap0 network Ethernet interface
[Expert@gw-5ad2f3:0]# 

Hopefully this helps.

[Andrew_Tindall]

Thanks for the reply, I actually figured out a solution (work around) when driving between offices shortly after I posted this.

I'm just in the process of writing it up for our internal wiki/knowledgebase. Once I've got it in some semblance of order and spellchecked I'll post it to this thread.

I'm sure someone else will come across the same  issue at some point, and it may save them a bit of head scratching.

[the_rock]

As soon as I read your post, it reminded me while back when I was in Maldives and I heard this dude on the beach talking to someone on his cell phone about this EXACT issue, so I introduced myself after he was done, told him I worked in IT and he actually ended up emailing me the process, worked fine few times, but when I tried it year later, it did not.

I would be interested to see the steps you share here.

Best,

Andy

[Timothy_Hall]

For a physical interface you can use the -p option to ethtool to constantly blink the LEDs on a certain interface name and then visually identify the physical card it corresponds to.  Obviously that doesn't work in VMWare, so in that case I would uncheck the "Connected" checkbox for a particular defined interface in VMWare, then from Gaia run ethtool on all interfaces to see which single named interface in Gaia was reporting Link Down, and then note the mapping.

[Andrew_Tindall · Accepted solution]

Ok not the quickest solution but here goes.

Initial build is with a Gateway install with two interfaces, one of which must be reachable by a device with a web browser for the first-time setup to be completed.

Once completed SSH into the device (or use the VMware console) and run the following command from expert mode:

See Checkpoint sk69621 for more detail.

[Expert@FW-B:0]# cat /etc/udev/rules.d/00-OS-XXrules

ID=="0000:0b:00.0", NAME="eth0"        

ID=="0000:13:00.0", NAME="eth1"        

This shows the currently assigned interface names and their associated PCI BUS ID’s.

We can now add the additional interfaces via VMWare, been careful to only have “Network adapter 1” connected to the device.

Now reboot the Gateway VM.

Log back into the Gateways Gaia interface and enable all of the newly created interfaces.

Now we need to enable the interfaces one at a time in VMWare to match them up against the Gateway interface showing up. Make a note of which Network adapter on VMware causes which ethx port on the Gateway to come up. You can do this either in Gaia or using the ethtool command.

Repeat the above steps for the remaining VMWare network adapters, at the end should have a table that looks something like this.

VMWare	Checkpoint Gateway
Network adapter  1	eth0
Network adapter  2	eth1
Network adapter  3	eth8
Network adapter  4	eth2
Network adapter  5	eth4
Network adapter  6	eth6
Network adapter  7	eth9
Network adapter  8	eth3
Network adapter  9	eth5
Network adapter  10	eth7

Now we need to ssh (VMWare console) back into the Gateway and edit the file we looked at earlier.

Before editing the the file looks like this.

ID=="0000:0b:00.0", NAME="eth0"

ID=="0000:13:00.0", NAME="eth1"

ID=="0000:04:00.0", NAME="eth2"

ID=="0000:05:00.0", NAME="eth3"

ID=="0000:0c:00.0", NAME="eth4"

ID=="0000:0d:00.0", NAME="eth5"

ID=="0000:14:00.0", NAME="eth6"

ID=="0000:15:00.0", NAME="eth7"

ID=="0000:1b:00.0", NAME="eth8"

ID=="0000:1c:00.0", NAME="eth9"

Once we have finished editing it.

ID=="0000:0b:00.0", NAME="eth0"
ID=="0000:13:00.0", NAME="eth1"
ID=="0000:04:00.0", NAME="eth3"
ID=="0000:05:00.0", NAME="eth7"
ID=="0000:0c:00.0", NAME="eth4"
ID=="0000:0d:00.0", NAME="eth8"
ID=="0000:14:00.0", NAME="eth5"
ID=="0000:15:00.0", NAME="eth9"
ID=="0000:1b:00.0", NAME="eth2"
ID=="0000:1c:00.0", NAME="eth6"

Now save the file and reboot the gateway.

For a final check once it has rebooted you can disconnect the interfaces one at a time in VMware and watch the state of the interface change via the Gaia web interface.

[the_rock]

Thank you, that definitely rings a bell.

Andy