Economics of Data Center Optics
By Lisa Huff, Discerning Analytics | Posted: 18 March 2016 2:30:43 PM
Twitter | LinkedIn | Facebook | Reddit
The use of optics in data centers has been slowly increasing over the last ten years. While there is promise of one converged data center network with Fibre Channel over Ethernet, Discerning Analytics, LLC (DA) is skeptical that we will ever see FCoE be the entire network. What makes most sense economically in the near term is what has been adopted up until now, that is separate storage, LAN and clustering.
As a result, what has been prevalent in data center cabling over the last few years will most likely continue to be deployed, albeit with increasing data rates to address such issues as latency and the true data center convergence phenomenon – virtualization and software defined networking. While copper twisted-pair (TP) Category data cabling has accounted for more than 90-percent of data center cabling over the last decade, it is now being relegated to very short reaches.
The dynamics influencing data center managers’ decisions on cabling are based on several factors. The early data centers (or computer rooms) were not really designed, but cobbled together as companies tried to figure out what types of computing devices fit their needs. This has made for a hodge-podge of different cabling types ranging most of which ended up being connected point-to-point in an extremely disorganized manner. Today, there are several different topologies that are being used to manage the network and cabling infrastructure within a data center. Which one is chosen is largely dependent on who is selecting it, the equipment they choose and their preferred suppliers.
The battle of copper versus fiber has been raging for many years. Just when fiber proponents think they have finally pushed copper entirely out of computer networks, it somehow reinvents itself and yet again. But with Terabit Ethernet on the horizon and data center managers worried about the cost per kW of power, fiber is seeing increasing adoption, even if it is still slightly more expensive. Presented below is the cost of one cabinet of aggregation-layer switches and their associated hardware and cabling.
|Table 2: Cost Analysis of Cisco Nexus 7010 Data Center Switch – Fiber versus Copper
||Horizontal Cable and Connectivity
||Total per Channel Cost
|Fiber SM FT
|Fiber LOMF FT
|Fiber LOMF PT
- Two Cisco Nexus 7010 10-slot chassis with the following:
- Non-blocking – equal bandwidth up and down – 1.92 Tb/s
- Per chassis:
- Four 12-port 40G module for uplinks (N7K-C7010) with 40G QSFP+ SR and LR4 transceivers for LOMF and SM respectively [for uplink to the core]
- Four 48-port 10G (N7K-F248XT-25E) copper module for copper [for downlink to the access] OR
- Four 48-port fiber module (N7K-F348XP-25) with SFP+ SR and LR transceivers for LOMF and SM respectively [for downlink to the access]
- Distribution prices
- Patch cords are 2m CAT6A, OM3 LOMF or SMF
- Labor rate of $60/hour
- Port cost computed by total cost of switch blade and transceivers/number of ports
Looking at the total installed cost per port, the copper solution is more economical. However, the cost gap between copper and short-reach fiber has been significantly reduced. In 2011, SR optical was nearly twice the cost of copper. Now, it is only about 1.3 times the cost. This is detailed in Figure 1 below. The cost differential has gone from over $3,000 to about $260 in just five years.
Figure 1: Cost Differential Between LOMF and Copper Channels
In addition, when you take into consideration power, cooling and space savings that fiber can give you, many data center managers have decided to implement fiber solutions for everything between cabinets.
The power consumption of the copper solution shown above is 460W/rack more than the short-reach optical solution. The long-reach solution is 1,200W more than the short-reach optical one. Clearly, from purely a power perspective, SR optical is the best option. And, if it is only a few hundred dollars more per port, you may be able to offset that cost with savings in your electricity bill.
Over the last year, the percentage of copper cabling being used inside data centers has decreased from about 67-percent to approximately 59-percent (by volume) and according to several data center managers that we interviewed recently, they would go entirely fiber if they could afford to. Now, it may be possible.
Lisa Huff is Principal Analyst at Discerning Analytics. She is a Certified Data Center Manager and electrical engineer focused on market research and analysis of data center technologies. You can see some of her work at www.discerninganalytics.com
Posted: 18 March 2016 by
Lisa Huff, Discerning Analytics
| with 0 comments