This document describes the implementation of Microsoft Network Load Balancing (NLB) mode on Cisco Unified Computing System-B (UCS-B) series with Fabric Interconnect (FI) in End-Host mode. There are also a number of requirements on the upstream devices to facilitate the correct forwarding of NLB traffic which are described in this document. The configuration sample focuses on multicast Internet Group Management Protocol (IGMP) mode.
Cisco recommends that you have knowledge of these topics:
- Microsoft Network Load Balancing
- Cisco UCS B-Series Servers
- Cisco Catalyst and/or Nexus Switches
This document is not restricted to specific software and hardware versions.
The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, make sure that you understand the potential impact of any command.
Microsoft NLB functions in three different operational modes: unicast, multicast, and multicast IGMP. A group of NLB nodes is collectively known as an NLB cluster. An NLB cluster services one or more virtual IP (VIP) addresses. Nodes in the NLB cluster use their load balancing algorithm in order to agree on which individual node will service the particular traffic flow destined for the NLB VIP.
This document does not make specific deployment recommendations for Microsoft NLB on UCS. As described in this document, NLB relies on unconventional methods for delivery of cluster bound traffic. It has been observed that both multicast and multicast IGMP modes appear to have stable and consistent operation on UCS-B Series servers. While NLB sizing guidelines are beyond the scope of this document, it is a solution generally recommended for smaller deployments.
Microsoft NLB Modes
The NLB default setting is unicast mode. In unicast mode, NLB replaces the actual MAC address of each server in the cluster to a common NLB MAC address. Typically, something in the 02bf:xxxx:xxxx range. All the nodes in the NLB cluster understand what the NLB VIP and MAC address is. Traffic, which includes Address Resolution Protocol (ARP) replies from NLB nodes, is never sourced from the NLB MAC or IP address. Instead NLB nodes use an assigned MAC address based on the host ID of the member. The MAC address is usually in the 0201:xxxx:xxxx, 0202, 0203, and so on range, one for each node in the cluster. This is the source address in the Layer 2 (L2) header when an ARP request is answered. However, the ARP header information contains the NLB MAC address. Thus, hosts that wish to correspond to the NLB VIP address send traffic towards the NLB MAC address.
IEEE compliant switches (L2 devices) build their MAC address table based on the L2 source header and not the information contained in the ARP payload. This means that traffic forwarded to the NLB cluster is sent to the NLB MAC address, which is never the source of traffic. Therefore traffic destined for the NLB MAC address is flooded as unknown unicast.
Mulitcast/Multicast IGMP Mode
Multicast mode assigns the cluster unicast virtual IP address to a non-Internet Assigned Numbers Authority (IANA) multicast MAC address (03xx.xxxx.xxxx). IGMP snooping does not dynamically register this address, which results in flooding of the NLB traffic in the VLAN as unknown multicast.
Multicast IGMP mode assigns the cluster virtual IP address and a multicast MAC address within the IANA range (01:00:5E:XX:XX:XX). The clustered nodes send IGMP membership reports for the configured multicast group and thus the FI dynamically populates its IGMP snooping table to point towards the clustered servers.
There is a slight operational advantage to the use of multicast IGMP mode since state information (via IGMP membership reports and IGMP snooping) about the interested L2 ports can be maintained both upstream and downstream. Without the optimization of IGMP snooping, NLB relies on unknown multicast flooding into the NLB VLAN for delivery to the cluster via the UCS designated broadcast/multicast receiver. In releases later than UCS Release 2.0, the designated broadcast/multicast receiver is chosen on a per-VLAN basis.
A summary of the steps required to support NLB in multicast IGMP mode is shown here:
- Static ARP entries for Virtual NLB IP addresses are typically on the VLAN SVI. If you use Hot Standby Router Protocol (HSRP) or First Hop Redundancy Protocol (FHRP), be sure that both devices have the static ARP entry.
- An IGMP snooping querier in the NLB VLAN. In releases later than UCS Release 2.1, snooping querier functionality is supported in UCS Manager.
- IGMP snooping needs to be enabled on all switches, which includes UCS. Note that most platforms that include UCS have IGMP snooping enabled by default.
A basic setup of NLB, the nodes can be virtual machines (VMs) or bare-metal installation of Windows Server OS, is shown in this diagram.
NLB VLAN 10 that has IP subnet 10.1.1.0 /24. The MAC address is truncated for brevity.
NLB VIP (MAC = 01, IP = 10.1.1.1)
NODE-A (MAC = AA, IP = 10.1.1.10)
NODE-B (MAC = BB, IP = 10.1.1.11)
NODE-C (MAC = CC, IP = 10.1.1.12)
Microsoft NLB Data Flow
Static ARP entry on the upstream switch SVI points to VIP address 10.1.1.1 to MAC 01.
Microsoft NLB nodes send the IGMP membership report. Note that the IGMP snooping table can take 30-60 seconds to populate.
With IGMP snooping and the VLAN querier, the snooping table is populated with the NLB MAC address and groups that point to the correct L2 ports.
- Off-subnet clients send traffic to NLB VIP address 10.1.1.1.
- This traffic is routed into the VLAN 10 interface which uses a static ARP entry in order to resolve the MAC address (01) of the NLB VIP.
- Since this is a multicast frame destination, it is forwarded per the IGMP snooping table.
- The frame arrives at all NLB nodes (Node A,B,C).
- The NLB cluster uses its load balancing algorithm in order to determine which node will service the flow. Only one node responds.
See these documents for more information:
Special Consideration for Nexus 1000v
Nexus 1000v only supports unicast Microsoft NLB mode. So on deployments of Nexus 1000v with UCS, multicast IGMP mode will only work after you disable snooping on Nexus 1000v. When this is done, Microsoft NLB packets on that VLAN are flooded as unknown multicast.
In order to minimize the impact of flooding:
- Disable snooping only on that VLAN in the Nexus 1000v.
- Use a dedicated VLAN for Microsoft NLB traffic.
The verification procedures for the configuration examples described in this document are provided in the respective sections.
There is currently no specific troubleshooting information available for this configuration.