By Yuval Bachar, Principal Engineer, Data Center Architecture at LinkedIn & President and Chairman of the Board at Open19 Foundation
The Illusion of Speed Leaping
If we look at the transition for 40G to 100G it took the industry about 4-5 years to make 25G/50G/100G the dominating networking technology, while in a lot of cases the 40G system have not been upgraded to 100G yet but in most location if you start a new network design it will be with the 25G/50G/100G train and not the 10G/40G train which is trending down and will trend down for the next 3-4 years. This transformation gave us 2.5x capacity and bandwidth jump, it took the industry 4-5 years. Why does it take so long?
The Upgrade Cycle
What does it take to do a network ports speed upgrade? It starts from a real need that even in best cases takes a cycle of 2-3 years to require doubling the port speed, we should not mix scale out and a speed multipliers need. While most of the technologies will try and push the biggest technology leaps, having the need first is usually a base requirement to push a speed leap. Next, we need the switching engines to enable us to step to the next level followed by system design and validation and lastly the required optical and copper signaling technology to enable that leap for end to end transformation. Silicon cycles are best case two years cycles followed by a 6-12 month of systems qualification and about 12 -18 months optics integration and qualification. Building a 4 years cycle between technology readiness for deployments, and now we are back to the actual need to be in place... With the technology and need in place, each one of the mega data center operator needs to deal with upgrades of millions of ports investing hundreds of millions of dollars in that upgrade, on top of the investment in scale out growth, how often we want that leap cycle or brand new technology?
Do We Need Another 4x Leap Now?
While some of the industry mega operators are pushing for an immediate 400G step leveraging the 6.4T and 12.8T emerging silicon, do we need it now? or are we going to churn the whole industry for a technology that will not be available or mature for us for 3-4 years for mass deployment? If we break down the schedule in the industry and the cycle to stabilization, specifically how we push the optical modules suppliers, we will see that we need to be more conscience about our demands. Let's look deeper at the sub components technology and timing.
Switching engines: 6.4T switches are available now or will be available in the next few months, however all of these switching engines are based on 25G NRZ signaling and we will not see any solution for 50G PAM4 in stable, production ready state for another 12 months.
Optical modules and form factor: 25G signaling technology for the optical modules electrical side are all available, proven and being commoditized with the huge 100G volumes ramping this year. We need to leverage this technology for at least 18-24 months not forcing a signaling change now, with 25G signaling and a x8 form factor (OSFP or QSFP-DD) creating a high-density cost effective 100G solution will have a simple direct path and will enable a 2x growth on the same footprint. 50G PAM4 ICs for optical modules are just emerging now, historically it takes a significant amount of time to stabilize these ICs and create a fully integrated production modules out of them.
Optical components in the modules 25G signaling proven and working 50G PAM4 or NRZ on the optical side still not ready for prime time and will not ready at a reasonable cost for a while, again asking why do we push the industry to the same situation that they were in for 100G with new invited technology across the board, and cost them unbelievable price and time. They are just not ready for 400G they are for sure ready for a cost effective Nx100G we have to let them ride this for some time to be successful and be able to fund the 400G modules development. Just a minor note about the form factor. We as an industry have a chance to develop a new form factor every decade or so, please let's break the political aspects and develop together the right one. OSFP has been defined from the ground up to address the future needs from cooling to signaling. QSFP-DD, in the name of backward compatibility that no one needs, is compromising this once a decade opportunity we have. OSFP team please open the doors to everyone to be a part of this standard and let's not have a VHS/Beta war in the next few years for optical modules' form factor.
Switching Systems: Even when just looking at 1RU solution it takes about 12-18 months to stabilize to high quality a switch into production, chassis take much longer. Even if we get the 400G capable switching engines in the next 6-9 months we are looking at best case middle of 2019 systems in production, combine that with the optics development we are looking at 2020 or 2021 for mass deployments, we cannot wait so long for the next step we need an intermediate system level and connectivity density step that will leverage what we worked so hard to build in the last 5 years.
In summary, to reduce our cycle time and enable technology to stabilize we really need to have that technology (electrical signaling, optical signaling, ICs, lasers, optical components, etc.) to have two life cycles in them. While we are all trying to push the technology forward we need to make sure we push the technology developers in a direction that they can be successful to enable us to build the most cost effective and scalable data centers. I am the last person that will say don't take risks, don't push the hard and fast on technology, however the technology push now just does not make sense to me, in my opinion 400G now is not the way. 400G is the following step not the next one, the next one is leveraging 6.4T and 12.8T switching engines with Nx100G optical modules and high-speed server interfaces leveraging existing, maturing and cost-effective building blocks.
Posted: 9 August 2017 by
Yuval Bachar, Principal Engineer, Data Center Architecture at LinkedIn & President and Chairman of the Board at Open19 Foundation
| with 0 comments