Virtualization Resources – VMware ESX and ESXi Server
Ok so your server probably has more than one physical NIC, by default most have two built in.
You may have been wondering what is the best network configuration for ESX/ESXi Servers.
How many physical NICs to use, which vSwitch to assigned them to and what vSwitch to use for certain traffic.
If you have more than one physical switch that supports link aggregation 802.3ad then you may want to take advantage of that too.
The number of network cards you have and what your requirements are may differ but I belive the 4 NIC setup is the best for performance, reliability and functionality. You can have more if you like but 4 will certainly suffice.
Why would you want to go to the trouble of setting this up and annoying the network team…
1. To present trunk ports to all ESX network interfaces. This is to segregate traffic to different VLANs for improved security (802.1q). Meaning you will be able to put virtual machines onto different VLANs (Maybe you have dev, test and production VLANs).
2. To load balance across NICs and physical switches using the IP hash (802.3ad). This will improve bandwidth usage and increase reliability.
Setup the network configuration as follows on each ESX/ESXi Server:
Virtual Switch Layout:
Virtual Switch Used For
vSwitch0 Virtual Machines
vSwitch1 Service Console and VMotion
vSwitch2 Internal
Virtual Switch to Physical NIC to Physical Switch Mapping:
Physical Network Interface Used by Virtual Switch Connected to Physical Switch Type
vmknic0 vSwitch0 Switch2 Trunk
vmknic1 vSwitch1 Switch2 Trunk
vmknic2 vSwitch0 Switch1 Trunk
vmknic3 vSwitch1 Switch1 Trunk
Virtual Networks:
Used For Port Group Type VLAN ID
Server Network Virtual Machines 11
VMotion Vmkernel 12
Service Console Service Console 12
Failover and Load Balancing Settings for vSwitchX:
Load Balancing: IP Hash
Network Failure Detection: Link Status Only
Notify Switches: Yes
Failback: Yes
Active Adaptors: vmnicX, vmnicX
Standby Adaptors: None
Unused Adaptors: None
NOTE: You cannot use VLAN 1 in virtual switches with trunking because the traffic is not tagged.
Switch Channel-Group Configuration for 802.3ad link aggregation
Create a port channel for each server.
interface Port-channel2
description ESX Server 1
switchport
switchport trunk encapsulation dot1q
switchport mode trunk
switchport nonegotiate
spanning-tree portfast trunk
end
Switch Port Configuration for 802.1q trunking
Make the port a trunk port and assign to that servers channel group.
interface GigabitEthernet2/10
description ESX Server 1
switchport trunk encapsulation dot1q
switchport mode trunk
switchport nonegotiate
spanning-tree portfast trunk
channel-group 2 mode on
end