A FCoE Community: Welcome to our community for developers, engineers and resellers interested in FCoE technology innovation.

Last time I wrote about the drive for a copper-based interconnect in FCoE-based data centers and the key factors impacting the decision process for whether to use SFP+ copper or 10GBASE-T links in lieu of SFP+ optical links.  I’ll address each of these factors – cost, performance, power, and reliability – in detail:

Cost: 10GBASE-T clearly has the cost advantage since it is the incumbent interconnect technology in today’s data centers.  Between SFP+ copper and SFP+ optical, SFP+ copper has a lower cost.  However, as I stated in my last blog, it is not enough to consider cost alone.

Performance: SFP+ optical is the clear winner for performance in terms of distance.  10GBASE-T also provides sufficient distance for most rack applications.  SFP+ passive copper cable, however, is turning out to fall short of expectations.  Although DAC was supposed to reach 20 m reliably when it was introduced over two years ago, many companies are discovering that they are only able to reliably reach 7 m with DAC.  This is unfortunate given that it is generally accepted that the minimal distance for rack level interconnect is 10 m.  The recent introduction of active copper (optical active cable) has helped overcome DAC’s distance limitations by extending the reach of DAC to at least 20 m. However, the cost and power consumption of active DAC is higher than passive DAC (see below table).

Power: 10GBASE-T, as it is implemented today, is simply not practical for high-density switch environments.  Typical power consumption is 8 W, with some implementations running as low as 4 W per port.  Power efficiency, however, is improving.  Certainly over time power efficiency will increase, especially given the overall drive to reduce power consumption across all network and communication applications.  While 10GBASE-T may not be ready from a power perspective for FCoE-based data centers today, at some point in the future it may be.

Power Consumption Comparison

This table compares the average power consumption of various interconnects. The discrepancy between 10GBASE-T and other interfaces is quite dramatic.

Reliability: Reliability is the cornerstone of Fibre Channel SANs, and it plays a significantly different role in FCoE networks than it does over traditional Ethernet.  Data always eventually arrives, whether over FCoE or Ethernet.  However, with Fibre Channel and FCoE the physical link is assumed to reliable enough to be considered “lossless”; i.e. there is no need for the protocol layer to concern itself with whether data arrived or not.  In contrast, TCP compensates for the assumed lossy nature of the physical link by retransmitting data at the protocol layers.

Over their respective distances, SFP+ copper and optical links are highly reliable with a bit error rate (BER) equivalent to that of Fibre Channel links. 10GBASE-T offers sufficient reliability when used with TCP but the noise level on the wire is high enough to limit reliability to a BER of 1×10^-12.  This is a full 3 orders of magnitude lower than the minimum BER required for FC of 1×10^-15.  Give the relatively lossy nature of 10GBASE-T, FCoE may be more sensitive to 10GBASE-T compared to TCP, making it difficult to maintain sufficient reliability for compatibility with Fibre Channel.

From a cost perspective, 10GBASE-T would be the ideal interconnect to use given its existing presence in the SAN, if only it offered sufficient reliability and more efficient power consumption.  Likewise, SFP+ DAC would be appropriate to use in more applications if it had sufficient reach.  Although more expensive, SFP+ optical links offer the best reliability and performance for FCoE-based data centers while still providing a cost-effective alternative to Fibre Channel.

Again, the performance, efficiency, and reliability of both DAC and 10GBASE-T will improve over time.  We anxiously expect to see 10GBASE-T-based FCoE products available on the market in the near future.