This document discusses quantized congestion notification (QCN), one of the methods proposed for end-to-end congestion notification in data center networks, and its limitations in today's Fibre Channel over Ethernet (FCoE) networks.
What Is QCN?
QCN is part of the IEEE Data Center Bridging (DCB) standard proposal aimed at enhancing existing IEEE 802.1 bridge specifications to satisfy the requirements for I/O consolidation. The goal of QCN is provide end-to-end congestion notification for Layer 2 networks. For QCN to be effective, it must be enabled on the entire data path throughout the network.
Switches that support QCN can notify end hosts (which must also support QCN) about the presence of congestion in the network. The end hosts can then respond by reducing their traffic transmission, helping alleviate the upstream congestion. This extension is defined in the IEEE 802.1Qau standard.
Figure 1 shows how QCN works in a Layer 2 domain.
Figure 1. Using QCN in a Layer 2 Domain
The figure shows hosts H1 and H2 concurrently sending traffic to host H3 while congestion is occurring on the path to H3. The upstream switch supporting QCN responds to the congestion by sending QCN notifications to hosts H1 and H2. This QCN message notifies the end hosts of the congestion in their data path, causing them to slow down or halt their packet transmission. QCN messages are destined to the MAC addresses of the end device(s) sending packets through the congested part of the network. The switch at the congestion point can send these QCN messages to any hosts in the Layer 2 domain as long as the entire data path, including the host, supports QCN.
Is QCN a Requirement for FCoE Networks?
FCoE is a method for transporting Fibre Channel traffic over lossless Ethernet networks. FCoE is defined in the Fibre Channel Backbone Generation 5 (FC-BB-5) standard, completed by the INCITS T11 Technical Committee in June 2009 and published by ANSI in May 2010. FC-BB-5 is part of the Fibre Channel family of standards and specifies how to transport Fibre Channel over other network technologies. FC-BB-5 defines the FCoE frame format, addressing structures, forwarding behavior, discovery protocols, and requirements for deployment. In this context, FC-BB-5 requires a lossless Ethernet network to enable FCoE to provide the same level of service that native Fibre Channel links provide to Fibre Channel.
The requirement for lossless Ethernet is explicitly defined in the FC-BB-5 standard; however, how this lossless behavior is achieved is not defined. The goal of QCN is to alleviate congestion within a Layer 2 network, but QCN does nothing to guarantee the lossless transport required for transmitting Fibre Channel traffic over an Ethernet network. This lossless guarantee must be provided by other means, such as priority flow control (PFC), which is another extension defined in the IEEE DCB standard proposal. Therefore, from a standards perspective, QCN is not a requirement for deploying an end-to-end FCoE network.
Does QCN Help in an FCoE Environment?
Although both Fibre Channel and Ethernet are Layer 2 protocols, FCoE acts more as a routed Layer 3 protocol from the perspective of Ethernet, and when placed on an FCoE fabric. FC-BB-5 requires all data being transmitted between FCoE hosts and targets to transverse a Fibre Channel forwarder (FCF) before reaching its final destination. When a host sends FCoE frames to the networks, these frames include the source MAC address of the host and the destination MAC address of the FCF. When this traffic reaches the FCF, the Ethernet headers, including both source and destination MAC addresses, are stripped from the frame, and the forwarding decision is made based on the Fibre Channel addressing information within the encapsulated Fibre Channel frame. After the switching decision has been made and the destination interface has been chosen by the FCF, the Fibre Channel frame is reencapsulated with new a source and destination MAC addresses; the source MAC address is the FCF-MAC address, and the destination MAC address is either the next-hop switch MAC address or the target MAC address. Figure 2 shows this MAC address translation.
Figure 2. MAC Address Translation
This MAC address translation occurs for all FCoE frames passing through an FCF. Because FCoE frames leaving the FCF no longer contain the source MAC address of the end hosts, a QCN message cannot be directed to an end host's MAC address after passing through an FCF.
The goal of QCN is to give switches the capability to notify end hosts of any congestion in the network so that the hosts can respond by decreasing the transmission of packets and therefore alleviate the congestion. QCN must be enabled throughout the entire Layer 2 fabric (including the hosts) to be effective. While it may work in Layer 2 Ethernet networks, FCoE networks require that FCoE traffic transverse an FCF, where source and destination MAC addresses are rewritten, making it impossible to send QCN messages to an end-host MAC address after traffic has passed through an FCF. Because of this limitation, QCN is ineffective in today's FCoE networks.