Linux / FreeBSD keyboard.

VMware Networking

Ethernet and IP Networking in VMware

VMware bridged networking.

VMware Bridged networking behaves as if there were a hub-based Ethernet LAN inside the system case, and a 2-port Ethernet switch between that and the outside.

Within the system, all operating systems see all frames of all guests and the host. But only frames destined for exterior systems leave the box. The guests have routable IP addresses and can be reached from remote systems. Ethernet frames from them have unique MAC addresses, with the first 3 octets (the manufacturer code) being the 24-bit pattern assigned to VMware.

VMware NAT networking has the host OS doing Network Address Translation for the guests. Click here to learn what NAT is and how it works. It behaves as if there were a hub-based Ethernet LAN inside the system case, and the host has a virtual interface connected to that and its real network interface port exposed to the outside.

VMware NAT networking.

Within the system, all operating systems see all frames of all guests and the host. Each guests still has a unique MAC address with a VMware manufacturer code making up the first half. But only those frames destined for exterior systems leave the box, bearing the source IP and MAC address of the host OS.

The guest systems have private IP addresses which are not routable from exterior systems. They can connect out through the NAT router, but inbound connections are only possible if you have set up static tunnels on the NAT device. You usually do not do that, as one of the main points of doing NAT in the first place is hiding the interior machines behind the NAT router and just using that one exterior IP address!

Specifying (or Changing) the Physical NIC Used by the Virtual or Guest Machines

I wanted to do something a little different, and I found it rather difficult to find how to accomplish this. The VMware tools aren't concisely documented, and Google searches led me to a lot of outdated or just plain wrong suggestions.

I am using VMware Player, a free product that does most things that I need. VMware says that you need to buy VMware Workstation to create new machines. Not if you know how to use EasyVMX! Just specify all the details of what you want, and it creates a small zip file for you. Unzip that, and you have a new VMware virtual machine, ready to be started for the first time and its operating system installed.

That lets me easily create new virtual machines, and their archives of tutorials show you how to modify the guest.vmx file to make the changes that might seem to require the VMware Workstation product.

My problem was that my searches all led to incorrect claims that I should just use the script to step through a series of questions and make my needed changes. But is not included with VMware Player!

Here's what I wanted to accomplish: I would use bridged networking. My VMware host system would also be a gateway. One Ethernet interface is connected to a cable modem and connects toward the outside world. The other Ethernet interface connects to an internal Ethernet switch. The gateway runs Linux, does routing and NAT, and has an Iptables firewall rule set to protect itself and the inside hosts. That includes the VMware guests, as they should also have IP addresses on the internal network. They can reach the outside world, but only because the Linux host OS is acting as a NAT router.

Inside are other hosts, including a D-Link wireless router. Its "Internet" port is connected to the Ethernet LAN. An 802.11 host gets a address from that wireless router, which does NAT before forwarding its packet onto the network. If that wireless host is connecting to the outside world, then the Linux gateway does NAT a second time, masquerading the D-Link's address of to whatever DHCP assignment the gateway picked up on the outside world. So, wireless hosts go out through two layers of NAT.

VMware networking with a Linux NAT router and firewall as the VMware host and multiple Ethernet networks inside.

That can include my mobile telephone, if I have the battery-eating 802.11 mode switched on. And, by extension, the Багта-50 Soviet rotary-dial telephone from 1955 if I am using it as a handset.

The problem is that VMware will prefer to use the first Ethernet interface, eth0, and there is no explanation of how to change that. The packets from the guests are transmitted right out from eth0 to the cable modem with their 10.0.0.x addresses intact, and from there toward the core of the Internet.

VMware Player installs from a file that is a 584-line shell script with a gzip'ed tar archive appended. That combination installs components all over the place, and since they don't use any sort of packaging scheme, you're left hunting for the pieces. But those of us who know how to use cat and tail and tar can figure out that those pieces include:

Boot script to start and stop the VMware services.
Various programs started by the VMware service to provide the virtual environment.
The vmplayer program itself, used to start and run a virtual machine.
A directory containing about twenty subdirectories, each containing many subdirectories and/or files, containing shared libraries, data, and tools.
Device special file used for the virtual machine communication interface kernel service. Created dynamically when the kernel loads the vmci.ko kernel module.
Device special file used for the virtual machine monitoring service. Created dynamically when the kernel loads the vmmon.ko kernel module.
Device special files used for the virtual networking service — vmnet0 is used for the actual communicating by the virtual machines, while vmnet1 and vmnet8 are used for any internal host-only networking. They get IP addresses from a DHCP service run by VMware on the virtual network only. Created dynamically when the kernel loads the vmnet.ko kernel module.
Log file used by vmnet service, discovered only because I was poking around with lsof trying to figure out where VMware hides all the files!

The solution to my problem was to modify how the vmnet-bridge process was started, telling it to prefer eth1 and to avoid eth0.

Good luck figuring out how to do that! I could not find a shell script that started that process. It appears to be started from within some binary with an exec(); call.

I finally found a solution: modify the file /etc/vmware/networking and specify weights of 0 and -1 for eth1 and eth0, respectively. You do that by inserting two new lines. Below is my file, with the newly inserted lines highlighted:

add_bridge_mapping eth1 0
add_bridge_mapping eth0 -1
answer VNET_1_DHCP yes
answer VNET_1_DHCP_CFG_HASH 4A4AE55F8580E079D81A16FCF6BBBD7CC5C4039B
answer VNET_8_DHCP yes
answer VNET_8_DHCP_CFG_HASH 19236B469B0420644DAC5D6C9AF16A2E604C4CB9
answer VNET_8_NAT yes

Now my vmnet-bridge process is started correctly:

% ps axuww | egrep 'PID|vmnet-bridge' | cut -c 1-5,9-15,65-
root  17262  /usr/bin/vmnet-bridge -s 6 -d /var/run/ -n 0 -eeth0 

And now things finally work!

There was one remaining oddity that I noticed only because I was doing lots of testing in parallel to my many experiments with different configuration settings. It doesn't cause any problem, but it looks a little odd.

I was running two Wireshark packet sniffer processes simultaneously, one listening on eth0 and the other listening on eth1. Both were filtering to only capture frames from the one guest I was using for testing.

The one listening on eth0 finally saw no bogus packets leaking out onto the Internet — that's good!

The other was seeing all the frames between the host OS and the guest OS, even though they should not be transmitted onto the internal LAN (there would be no real harm if they were, but no benefit, either).

Then I realized my mistake — I was listening on that very host OS! Of course it is going to see those frames, because I was using bridged networking! Thanks to the VMware virtual network and that apparent internal hub and two-port switch, it is going to see all frames to and from guest systems as well as all frames to and from itself!