DCB and FCoE

Since its inception, Brocade has been a driving force behind open industry standards for data center networking technologies. Today, Brocade continues to lead the creation of open standards for converged data center technologies:

• Fibre Channel over Ethernet (FCoE)
• Data Center Bridging (DCB)
• Transparent Interconnection of Lots of Links (TRILL)

Brocade also leads the industry in high-performance solutions that streamline server I/O consolidation and enable converged data centers. Today, the unified, end-to-end Brocade DCB/FCoE solution integrates seamlessly into existing data center environments and includes:

• Brocade Converged Network Adapters (CNAs)
• Top-of-rack FCoE switches
• End-of-row FCoE blades for backbone platforms
• Data center management applications

The advent of Fibre Channel over Ethernet (FCoE) and Data Center Bridging (DCB) ushers the beginning of renewed efforts to converge data center Storage Area Networks (SANs) and Local Area Networks (LANs). The diverse nature of the technologies needed to enable convergence requires the development of new industry standards that cover Fibre Channel, Ethernet, and Link Layer (Layer 2) routing.

The FCoE protocol was developed by the INCITS Technical Committee T11 as part of the T11 FC-BB-5 project. The INCITS public review was completed with no comments, which means that the standard will soon be published by INCITS as an industry standard.

The FCoE and DCB protocols are being developed by three different industry standards bodies, each focusing on technology areas that fall under a specific domain of expertise:

• INCITS Technical Committee T11: FCoE
  • FCoE and FIP are defined in FC-BB-5, which describes how other protocols are transported and mapped over a Fibre Channel network. The T11 committee completed its technical work for FC-BB-5 in June 2009, and forwarded the draft standard to INCITS for approval and publishing. INCITS will likely publish the formal industry standard soon.
  • The new FCoE standard is an encapsulation protocol that wraps Fibre Channel storage data into Ethernet frames, enabling them to be transported over a new lossless Ethernet medium.
• IEEE: Data Center Bridging
  • The Data Center Bridging (DCB) effort undertaken by an IEEE 802.1 work group is aimed at adding new extensions to bridging and Ethernet so that it becomes capable of converging LAN and storage traffic on a single link. DCB is designed to make Ethernet more like Fibre Channel, because the new features being added to Ethernet are solving issues that Fibre Channel faced in the past and successfully resolved. IEEE is expected to complete its work on the various components of DCB in the second half of 2010. The new enhancements are:
  • 802.1Qbb: Priority-based Flow Control (PFC)
    • Establishes eight priorities for flow control based on the priority code point field in the IEEE 802.1Q tags. This enables control over individual data flows on shared lossless links. PFC allows Fibre Channel storage traffic encapsulated in FCoE frames to receive lossless service from a link that is being shared with traditional LAN traffic, which is loss-tolerant.
  • 802.1Qaz: Enhanced Transmission Selection (ETS)
    • ETS provides the capability to group each type of data flow, such as storage or networking, and assigns an identification number to each of the traffic class groups. The value of this new feature lies in the ability to manage bandwidth on the Ethernet link by allocating portions (percentages) of the available bandwidth to each of the groups. Bandwidth allocation allows traffic from the different groups to receive their target service rate (such as 8 Gbps for storage and 2 Gbps for LAN). Bandwidth allocation provides quality of service to applications.
    • ETS incorporates Data Center Bridging Exchange (DCBX), a discovery and initialization protocol that discovers the resources connected to the DCB cloud and establishes its limits. DCBX distributes the local configuration and detects the misconfiguration of ETS and PFC between peers. It also provides the capability for configuring a remote peer with PFC, ETS, and application parameters. The application parameter is used for informing the end station which priority to use for a given application type (such as FCoE, iSCSI, and so on). DCBX leverages the capabilities of IEEE 802.1AB Link Layer Discovery Protocol (LLDP).
  • 802.1Qau: Quantized Congestion Notification (QCN)
    • This end-to-end congestion management mechanism enables the throttling of traffic at the edge nodes of the network in the event of traffic congestion. QCN is a separate protocol that is independent from PFC and ETS. While PFC and ETS work together, they do not require QCN.
• IETF: TRILL
  • Internet Engineering Task Force (IETF) is developing a new shortest path frame routing in multihop environments. The new protocol is called Transparent Interconnection of Lots of Links (TRILL) and is expected to be completed in the second half of 2010:
    • TRILL provides a Layer 2 multipath alternative to the single-path and network bandwidth-limiting Spanning Tree Protocol (STP) currently deployed in data center networks.
    • TRILL also provides Layer 2 multi-hop routing capabilities that are essential for expanding the deployment of DCB/FCoE solutions beyond access layer server I/O consolidation and into larger data center networks.

DCB, DCE, CEE, and TRILL Terminology

In the normal course of progress, new technologies are developed to advance the state-of-the-art of data centers. Inevitably, new names and acronyms are added to the already long lists of acronyms used daily by IT professionals. Fibre Channel over Ethernet (FCoE) and Data Center Bridging (DCB) technologies are no exception and have added a few acronyms that deserve clarification.

Data Center Bridging (DCB)
Data Center Bridging (DCB) defines the standards development efforts underway by IEEE 802.1 work group. The work is aimed at adding new extensions to Ethernet to turn it into a lossless transport that is suitable for transporting storage traffic without losing customer data. The new enhancements are expected to be ready in the second half of 2010. The new enhancements are:

• 802.1Qbb: Priority-based Flow Control (PFC)
  • Establishes eight priorities for flow control based on the priority code point field in the IEEE 802.1Q tags. This enables control over individual data flows on shared lossless links. PFC allows Fibre Channel storage traffic encapsulated in FCoE frames to receive lossless service from a link that is being shared with traditional LAN traffic, which is loss-tolerant.
• 802.1Qaz: Enhanced Transmission Selection (ETS)
  • ETS provides the capability to group each type of data flow, such as storage or networking, and assigns a group identification number to each of the traffic class groups. The value of this new feature lies in the ability to manage bandwidth on the Ethernet link by allocating portions (percentages) of the available bandwidth to each of the groups. Bandwidth allocation allows traffic from the different groups to receive their target service rate (such as 8 Gbps for storage and 2 Gbps for LAN). Bandwidth allocation provides quality of service to applications.
  • ETS incorporates Data Center Bridging Exchange (DCBX), a discovery and initialization protocol that discovers the resources connected to the DCB cloud and establishes its boundaries. DCBX distributes the local configuration and detects the misconfiguration of ETS and PFC between peers. It also provides the capability for configuring a remote peer with PFC, ETS, and application parameters. The application parameter is used for informing the end station which priority to use for a given application type (such as FCoE, iSCSI, and so on). DCBX leverages the capabilities of IEEE 802.1AB Link Layer Discovery Protocol (LLDP).
• 802.1Qau: Quantized Congestion Notification (QCN)
  • This end-to-end congestion management mechanism enables the throttling of traffic at the edge nodes of the network in the event of traffic congestion.

Data Center Ethernet (DCE)
This term was originally used by Cisco, defining Data Center Ethernet (DCE) as an architecture that is made of a collection of enhancements to "classical, or traditional Ethernet." Recently, Cisco has moved away from the use of DCE in favor of DCB to reduce confusion. Some of the confusion was due to ambiguity of the meaning of DCE and the features it defines or includes.

Despite the lack of consistent definition, the term refers to switches that implement three capabilities: FCoE, Layer 2 multipathing, and Data Center Bridging (DCB). As noted earlier, DCB contains a set of features: Priority Flow Control (PFC), Enhanced Transmission Selection (ETS) including Data Center Bridge Exchange (DCBX), and Quantized Congestion Notification (QCN).

Converged Enhanced Ethernet (CEE)
The term CEE came out of the CEE Authors Conference initiated by IBM and attended by 20+ companies, including Brocade. Initially, five features or technologies were considered as part of the proposed CEE umbrella: PFC, ETS, DCBX, QCN, and TRILL. For practical considerations, the CEE Authors Group decided to bring PFC, ETS, and DCBX to the IEEE work group, but the various member companies would continue to work toward getting QCN and TRILL into full standards.

Brocade used CEE to refer to the enhancements being added to traditional Ethernet as part of the IEEE DCB effort to make it lossless and capable of carrying Fibre Channel storage traffic with a high degree of integrity.

Transparent Interconnection of Lots of Links (TRILL)
TRILL defines the Internet Engineering Task Force (IETF) effort to develop a new shortest path frame routing in multi-hop environments. TRILL will provide a Link Layer (Layer 2) multipath alternative to the single-path and network bandwidth-limiting Spanning Tree Protocol (STP) currently deployed in data center networks. It will also introduce multihop routing capabilities that are essential for expanding DCB/FCoE deployment beyond access layer server I/O consolidation and into larger data center networks. The new protocol is expected to be completed in the second half of 2010.