By Scott Wilkinson Lead Analyst – Optical Components Cignal AI
The high-profile headlines at OFC 2023 will focus on the latest advancements in high-speed coherent technologies – 1.2Tbps, 1.6Tbps, pluggable 800Gbps, etc. However, OFC 2023 will also offer an opportunity to see how much further and faster direct detect optics will take communications inside the data center.
While the questions of if and when coherent technology will enter the datacenter come up in any discussion of high-speed datacenter optics, the engineers working on direct-detect, PAM4 optics continue to push the technology’s speeds and performance. With inherently lower cost and power, direct detect optics win in any head-to-head comparison against coherent optics for datacenter operations. The performance of current generation 400GbE PAM4-based optics and the initial 800GbE PAM4 optics will keep coherent optics out of the datacenter until at least the advent to 1.6TbE optics – and perhaps even further.
Current generation 400GbE client optics are built on 50Gbps per lane PAM4 electrical connections to the switch fabrics, GPUs, and other silicon chipsets. Next generation 800GbE modules are being built on 100Gbps per lane electrical PAM4. There have been some initial demonstrations of 200Gbps electrical PAM4, which is required for 8-lane 1.6Tbps client optics. At OFC 2023, we expect to see more demonstrations of 200Gbps PAM4 modules, which will give the industry greater insight into how far those optics can operate.
Would you like to know more about optical component technology, shipments, and for both inside and outside the datacenter? Learn more about Cignal AI’s Optical Component Research report.
PAM4 Today – 56Gbps Electrical for 400GbE
Current 400GbE modules use eight lanes of electrical 56Gbps PAM4 connections to provide a full 400Gbps of data. In most 400GbE modules, these eight 56Gbps electrical lanes connect via a gearbox to four 112Gbps optical lanes, either on separate fibers (e.g., DR4) or separate wavelengths (e.g., FR4). In 2022, over 3 million 400GbE modules were deployed in data centers worldwide.
56Gbps PAM4 electrical lanes are also used in current generation 25.6Tbps switch fabrics, which support up to 64 400GbE ports of connectivity. Several hyperscale data center operators are well into the process of upgrading their data centers to these 25.6Tbps switches, thereby driving the continuing demand for 400GbE.
But 400GbE is today’s news, and OFC is where we talk about tomorrow. Coming on the market now are 800GbE modules.
PAM4 Coming Soon – 112Gbps Electrical for 800GbE
The newest 51.2Tbps switch chips supporting 112Gbps PAM-4 interfaces are available now or coming in early 2023 from several equipment vendors. The optics taking advantage of those speeds have been available in small quantities for design and evaluation for several months. At OFC last year, some preliminary 800GbE modules were shown, mainly in 2x400GbE modes. At OFC 2023, we expect on-floor demonstrations from most datacenter optics vendors of not only 2x400GbE modules but also of single channel (8x100Gbps) 800GbE operation.
112Gpbs PAM4 can also be used to “upgrade” lower speed modules (100GbE, 400GbE), although there appears to be limited demand for such upgrades. Standards for QSPF112 (400GbE with four electrical lanes) and SFP112 (100GbE with one electrical lane) have been defined, but few hardware vendors have roadmaps, including those products. The large data center operators are migrating away from 100GbE towards 400GbE and see no benefit to updating older equipment that is already capped in the network. The 400GbE modules being deployed now are designed to interface with current switch chips, both of which are still increasing volume deployments. Rather than restart that process with new 4x112Gbps modules and new switch silicon, operators will wait for the next generation of 1.6TbE modules (8x224Gbps or 16x112Gbps) coming in a few years.
However, at least one hardware vendor focused on service provider networks has indicated plans to build interfaces for the lower-speed modules (QSFP112 and SFP112). Datacenter operators are not interested, but traditional service providers still see a great demand for 100GbE and 400GbE in the metro. A 100GbE module with a single 112Gbps PAM4 input and output will be much lower power than a 4-lane version and can directly interface with the latest 51.2T switch chips. Putting a 100GbE interface into a smaller SFP format also reduces front panel space and the size of the equipment required to support the interface. While 100G SFP112 interfaces may not show up in equipment at OFC 2023, we expect to see them quietly integrated into 51.2T switches this year.
Eight optical lane datacenter optics have typically been short-lived due to the cost and complexity of eight separate lasers, so the goal for 800GbE is to have four optical lanes instead. Single channel 800GbE modules with four lanes of 200Gbps PAM4 will compete with the OIF 800Gbps-LR coherent technology standard, a de-featured and lower-cost coherent solution for operation at 10km or less. The initial word from PAM4 vendors is that 200Gbps PAM4 optics will operate at 6km and may operate even further. Since direct detect PAM4 is significantly less expensive and lower power than coherent optics, PAM4 will win a head-to-head contest between the two technologies over the same distance. At OFC 2023, we expect to see demonstrations and confirmations of how far that distance will extend for 400GbE PAM4 modules. If the modules perform as promised, then the market for 800Gbps-LR coherent will be vanishingly small and only address full 10km applications where PAM4 falls short.
112Gbps interfaces will also enable the first 1.6TbE modules based on the OSFP-XD specification. OSFP-XD is designed to support 16 signal lanes for 1.6TbE and, eventually, 3.2TbE. However, most switch vendors are waiting for 224Gbps PAM4 (4 lanes) for mass 1.6TbE rollouts – assuming that 224Gbps PAM4 can be brought to market within the next two years.
PAM4 Future – 224Gbps Electrical for 1.6TbE
102.4Tbps switch fabrics are on roadmaps, and 224Gbps PAM module definitions are in the early stages, but technology demonstrations have already begun. At OFC 2023, we expect to see more product announcements at the fundamental level (drivers, modulators) and more demonstrations of combined systems.
The primary goal of 224Gbps electrical PAM4 is to support 1.6TbE with eight lanes (8x224Gbps) and, eventually, 3.2TbE (16x224Gbps) with OSFP-XD form factor modules. Supporters of the QSFP format are optimistic that a QSFP-DD1600 module can be designed to support 1.6TbE (8x224Gbps). Still, that effort has some significant challenges to overcome, and most module vendors are gravitating toward OSFP-XD at this early stage. A QSFP224 module has been defined to support 800GbE as well, but it will likely fall into the same category as the niche 112Gpbs modules (e.g., QSFP112 support for 400GbE) with limited actual deployment.
Both eight-lane and sixteen-lane solutions for 1.6TbE will exist, but as mentioned above, economics has always driven datacenter solutions down to, at most, four lanes in large deployments. Four optical lane 1.6TbE modules will require 400Gbps optical PAM4 – a technology that is not even on drawing boards yet and will have a much shorter reach than 200Gbps optical PAM4. Therefore, solutions like 2x800Gbps-LR and 4x400ZR coherent are being seriously considered for 1.6TbE. The first coherent optics inside the datacenter have a good chance of being part of a 1.6TbE module.
Conclusions
While WDM transport coherent optics may command the biggest headlines at OFC 2023, most of the world’s bandwidth rides on direct detect optics inside the datacenter, and the advances in those optics should not be overlooked. PAM4 will continue to challenge coherent optics in the datacenter as a lower cost/lower power solution at least through 800GbE and perhaps even to 1.6TbE. This year, we will get some first looks at cutting-edge PAM4 speeds and gain a better understanding of just how far the technology will go.
Posted: 17 January 2023 by
Scott Wilkinson Lead Analyst – Optical Components Cignal AI
| with 0 comments