DCB and FCoE

Since its inception, Brocade has been a driving force behind open industry standards for data center networking technologies. Today, Brocade continues to lead the creation of open standards for converged data center technologies:

• Fibre Channel over Ethernet (FCoE)
• Data Center Bridging (DCB)
• Transparent Interconnection of Lots of Links (TRILL)

Brocade also leads the industry in high-performance solutions that streamline server I/O consolidation and enable converged data centers. Today, the unified, end-to-end Brocade DCB/FCoE solution integrates seamlessly into existing data center environments and includes:

• Brocade Converged Network Adapters (CNAs)
• Top-of-rack FCoE switches
• End-of-row FCoE blades for backbone platforms
• Data center management applications

Data Center Bridging (DCB) defines the standards development efforts underway by the IEEE 802.1 work group. The work is aimed at adding new extensions to bridging and Ethernet to turn it into a lossless transport suitable for transporting storage traffic without losing data. The ability to move various types of data flows over a single shared lossless Ethernet link creates a convergence-friendly transport and opens the door for supporting requirements of applications from different purpose-built networks. The new enhancements are expected to be ready in the second half of 2010. The following sections describe these DCB efforts.

• 802.1Qbb: Priority-based Flow Control (PFC)
  • PFC attempts to emulate Virtual Channel technology that is widely deployed in current Brocade Fibre Channel SANs. While borrowing the lossless aspect of Virtual Channels, PFC retains the option of being configured as lossy or lossless. PFC is an enhancement to the current link-level Ethernet flow control mechanism defined in IEEE 802.3X (PAUSE). Current Ethernet protocols support the capability to assign different priorities to different applications, but the existing standard PAUSE mechanism ignores the priority information in the Ethernet frame. Triggering the PAUSE command results in a shutdown of the entire link for all applications, even when a single application causes the congestion. The current PAUSE is not suitable for links where storage FCoE and networking applications share the same link, because it is important to ensure that congestion caused by one application does not disrupt the other application traffic.
  • IEEE 802.1Qbb is tasked with enhancing the existing PAUSE protocol to include the priority in the frames contributing to congestion. PFC establishes eight priorities using the priority code point field in the IEEE 802.1Q tags (see Figure 1) that enable the control of individual data flows, or flow control, based on the frame’s priority. Using the priority information, the peer (server or switch) stops sending any traffic for that specific application, or priority flow, while the other application data flows continue without disruption on the shared link.
  • The new PFC feature allows Fibre Channel storage traffic encapsulated in FCoE frames to receive lossless service from a link that is being shared with traditional LAN traffic, which is loss-tolerant. In other words, separate data flows can share a common lossless Ethernet while each is protected from flow control problems of the other flows. Note that LAN traffic priorities can be configured with PFC off, allowing for lossy or lossless LAN transmissions.

Figure 1. Priority Flow Control using tagged Ethernet frames.

• 802.1Qaz: Enhanced Transmission Selection (ETS)
  • ETS attempts to manage the traffic priorities between multiple applications by regulating data flows and by assigning preset amounts of link bandwidth and relative priority to each application. It creates the ability to group each type of data flow, such as storage or networking, in a traffic class group, and assigns a group identification number to each of the groups. For example, FCoE can use two different priorities, but both priorities are assigned one Group ID number 3 as shown in Table 1.
  • The value of this new feature lies in the ability to manage bandwidth on the Ethernet link by allocating portions (percentages) of the available bandwidth to each of the priority groups. The assigned percentage indicates the maximum bandwidth a priority group or traffic class group can use. Bandwidth allocation delivers quality of service to applications and data flows that need it, such as storage flows or media-rich transmissions. Figure 1 shows an example of possible distribution of bandwidth on a lossless Ethernet link. When a DCB link is configured as shown in Figure 1, all IPC traffic is transmitted first, and then the remaining bandwidth is equally shared between LAN and FCoE traffic. As a result, the IPC applications, which might be sensitive to latency, are never starved for bandwidth by the other applications.
    
    

    Priority Group ID	Bandwidth %	Application
    1	-	IPC (Clustering)
    2	50%	LAN (TCP/IP)
    3	50%	SAN (FCoE)

    Table 1. Priority Group ID (traffic class groups) and bandwidth allocation.
    
    
  • The IEEE ETS effort also incorporates Data Center Bridging Exchange (DCBX), a discovery and initialization protocol that discovers resources connected to the lossless Ethernet, or DCB cloud. DCBX distributes the local configuration and detects the misconfiguration of ETS and PFC between peers. It also provides the capability for configuring a remote peer with PFC, ETS, and application parameters. The application parameter is used for informing the end station which priority to use for a given application type (such as FCoE, iSCSI, and so on). In other words, DCBX allows the mutual discovery of DCB-capable hosts and switches then allows DCB-specific parameters, such as those of ETS and PFC, to be exchanged before Ethernet links are shared. The exchange of parameters defines the capabilities and the boundaries of the DCB cloud and the applicability of Ethernet DCB enhancements within the DCB cloud. DCBX leverages the capabilities of IEEE 802.1AB Link Layer Discovery Protocol (LLDP) and will enhance its capabilities with DCB-specific parameters.
• 802.1Qau: Congestion Notification (CN or QCN)
  • This end-to-end congestion management mechanism enables the throttling of traffic at the end stations in the network in the event of traffic congestion. When a device is congested, it sends a congestion notification message to the end station to reduce its transmission. End stations discover when congestion eases so that they can resume transmissions at higher rates. Note that PFC and ETS are independent protocols that do not require QCN to be implemented.