FCoE or iSCSI which way are you converging?

The recent FCOE announcements have clearly stirred the pot; with plenty of debate on whether FCOE is simply a smash and grab layer violation, or else is a technology with lasting utility. Thankfully I don’t need to take a public stance on the matter since Solarflare by definition supports all upper layer protocols over Ethernet and in particular supports the Data Center Ethernet (also known as CEE) protocols required for software based implementations of FCOE within its Solarstorm Ethernet controller products.

See:

http://www.eetimes.com/showArticle.jhtml?articleID=207100764

http://interconnects.blogspot.com/2008/04/moving-at-10g-rates.html

http://blocksandfiles.com/article/4773

http://blogs.cisco.com/datacenter/2008/04/the_antifcoe_sentiment.html

http://en.wikipedia.org/wiki/OSI_model

http://www.open-fcoe.org/

http://www.fcoe.com

It is however worth understanding the architectural differences between the current crop of FCOE announcements and also the difference between network convergence and unified networks. First, a quick look at the two FCOE architectures:

 

1. Software based FCOE is the natural model to adopt for a commodity LAN vendor who is converging on storage. Here the goal may be to inter-network with legacy FC equipment by running  a FC protocol stack on top of a regular Ethernet driver in a commodity  host operating system. The Ethernet controller looks just like a regular Ethernet controller, except that it is able to partake in the new Ethernet  congestion avoidance protocols currently being defined in the IEEE. The FCOE driver is handed an FCOE frame and needs to perform FC protocol.      Performance is likely to be worse than a hardware based solution because software is now doing FC protocol which would previously have been done by the FC controller and also because FC data integrity is now being done in software. (As an aside, data-integrity performance is becoming less of an issue with this architecture due to protocol neutral CRC support which is increasingly being found in hardware.) Also end customer acceptance is likely to be slow on the uptake because the software FC stack is immature and the FC community is rightly very conservative regarding system reliability and cross-vendor compatibility. However, software based FCOE has the      promise of highly integrated commodity silicon (for the server at least).

 

2. Hardware based FCOE is the natural model to adopt for a storage HBA vendor who is converging on LAN. Here the goal may be to cost-optimize a rack solution by running both FC and LAN      traffic using a single HBA and wire to a top of rack switch which connects out to FC and regular Ethernet networks. (Yes I know that IEEE is also defining inter-switch congestion avoidance protocols which would enable a multi-tier converged Ethernet network, but frankly these are a long, long way from any sort of maturity). The converged HBA looks at the hardware level like two distinct hardware devices: an Ethernet controller and an FC controller with the advantage that the software running on the server can be the existing mature (and loved) driver stacks of the respective Ethernet and FC controller vendors. (Yes, unexpected alliances between FC HBA vendors and well known Ethernet vendors are required for this solution unless customers are to be persuaded that FC vendors overnight have their own mature Ethernet controllers and software). So the advantage of this model is that it can be available as a deployable product in short order. However not easily integrated within volume silicon. Of course if you are an end customer paying HBA prices, then this might be acceptable.

As well as the implementation and business differences between the FCOE architectures, both FCOE and iSCSI illustrate the difference between two different approaches to the unification of networks:

 

1. iSCSI is an example of network convergence, where different network functions are Layered into one protocol stack architecture. Ethernet implementing OSI Layers 1, 2 TCP/IP implementing Layers 3, 4+ and iSCSI implementing Layer 5. By converging on a single protocol stack architecture the features and abstractions of the underlying layers are built upon. For example iSCSI sessions may be routed over different underlying IP networks.

 

2. Whereas FCOE is an example of a unified network in that two different network protocol stack architectures are carried by a single Layer 2 network (Ethernet in this case). By maintaining two distinct network protocol stacks, a unified network is able to support applications which require both network protocol stacks while transitioning the network at a lower layer.

 
These two different approaches are the same double-edged sword for both iSCSI and FCOE and will be the cause of significant continuing religious debate. Understanding the difference is key to deciding whether one or the other is the right approach for a given enterprise to take.