Ethernet Meets Enterprise Storage, Finally

This article was posted on SNIA Ethernet Storage Blog

And written by  Mike Jochimsen, director of product marketing and alliances at Emulex

Ethernet Meets Enterprise Storage, Finally

Presumptuous, yes, because Ethernet has been a mainstay in enterprises since its early days over 40 years ago. It initially grew to prominence as the LAN connection in the enterprise. More recent advances have enabled Ethernet to become a standard for mission critical storage connectivity for block, file and object storage in many enterprises.

Block storage in large enterprises has long been focused on FC due to its performance capabilities. In order to bring the same performance benefits to Ethernet, the IEEE 802.1 Data Center Bridging Task Group proposed a number of new standards to enhance Ethernet reliability. For example, 802.1Qbb Priority-based Flow Control (PFC) provides a link level flow control mechanism to ensure lossless transmission under congestion, 802.1Qaz Enhanced Transmission Selection (ETS) provides a management framework for prioritized bandwidth and Data Center Bridging Exchange Protocol (DCBX) enabled these features to be used between neighbors to ensure consistency on the network. Collectively, these and other enhancements have brought those enterprise storage networking features to the Ethernet platform.

In addition, the International Committee for Information Technology Services (INCITS) T11 FC committee developed a specification for FCoE (FCoE) in its FC-BB-5 standard in 2009, which allows the FC protocol to run directly on top of Ethernet, eliminating the TCP/IP stack and allowing for efficient performance of the FC protocol. FCoE also depends on the Data Center Bridging standards from IEEE 802.1 in order to ensure the losslessness and flow control needed by FC.

An alternative to FCoE, iSCSI, was designed to run over standard Ethernet with TCP/IP and was designed to tolerate the “lossy” aspects of Ethernet. Its architecture and the additional layers of encapsulation involved can impact latency and performance. However, more recent innovations in iSCSI have enabled it to run over a DCB Ethernet network, which enables iSCSI to inherit some of the enterprise storage features which have always been inherent in FC. For more on this, read last year’s blog How DCB Makes iSCSI Better from Allen Ordoubadian.

In 2013, INCITS submitted the FC-BB-6 standard for review which introduced, among other things, the VN2VN standard. The VN2VN proposal will allow FCoE to work in a standard DCB switching environment without the presence of a FC Forwarder (FCF). An FCF allows for bridging between servers which are communicating with FCoE and storage devices which are communicating with traditional FC. As DCB switches and FCoE storage become more prevalent, the FC-BB-6 standard will allow for end-to-end FCoE connectivity in either a point to point (P2P) or DCB mesh environment. This will result in lower cost for FCoE environments. Products are beginning to appear which support VN2VN and over the next 18 months it is likely that all major vendors will support it. Check out our ESF Webcast How VN2VN Will Help Accelerate Adoption of FCoE for more details.

The availability of CNAs with processing capability allows for offloading storage protocol processing from the host processor, though some CNAs use host-based storage protocol initiators in system software and do selective stateless offloads in the data path. Both FCoE and iSCSI require the storage protocol to be encapsulated in a frame to be sent across the Ethernet network. In an enterprise environment, especially a virtual server environment, CPU utilization is tracked closely and target CPU thresholds are often set. Anything which can minimize spikes in CPU utilization can allow for more workloads to be placed on servers and allows for predictable energy consumption.

For file storage, Ethernet has traditionally been the connectivity option of choice for file servers used as shares for centralized employee document storage. In the 21st century, usage of NAS with NFS has increased for enterprise databases and Hadoop clusters, especially with the availability of 10GbE. New features in NFS 4 and later introduced security and stateful protocol support after development of NFS was taken over by the Internet Engineering Task Force (IETF).

Object storage, has been around for nearly 20 years as a repository for storing data as objects which include not only the original file, but also a globally unique identifier and metadata which describes the object and various parameters about the object. It has been used to store many forms of unstructured data, but found niches in certain areas, such as legal documents with retention policies and archiving photos and videos. More recently, there seems to be a resurgence in object storage as the amount of unstructured data generated by enterprises continues to skyrocket. Open source object storage in Ceph and OpenStack are also helping to drive the adoption. SNIA ESF is hosting a live Webcast on object storage on June 11, 2014, called Object Storage 101. I encourage you to register for this presentation for an unbiased look at the what, how and why of object storage technologies.

When combined with the advances in link speed, throughput capabilities, latency and IO/s (IO/s) in modern 10GbE and 40GbE, these existing and emerging Ethernet standards and storage architectures are having a profound effect on the ability of Ethernet as an enterprise storage networking platform. Vendors and customers are seeing the advantage in one wire, the Ethernet cable, carrying all LAN, WAN and storage traffic.