The Challenges of 800G Ethernet: Data Center Migration

Over the past few years, the massive use of cloud infrastructure and services has driven the demand for more bandwidth, faster speeds, and lower latency performance. Improvements in switch and server technology require adjustments to cabling and architecture. As a result, the demand for 800G Ethernet for data center migration, especially the demand for speed (including bandwidth, fiber density, and channel speed), has increased dramatically.

Embracing the 400G/800G Ethernet Revolution

In terms of application, the branching application technology of switch ports is being actively promoted. Usually, 400G and 800G high-speed network connections are often split into 4*100G or 8*100G. In terms of capacity, 64*400G ports are currently supported with a bandwidth of 25Tb/s. Broadcom's Tomahawk4 switch chip may support 32x800G ports in the near future. At the physical layer, the approval of the IEEE802.3bs standard in 2017 paved the way for the development of 200G and 400G Ethernet. Therefore, the application of 400G has increased rapidly. It is conceivable that the application of 800G Ethernet is expected to expand at a faster rate in the near future. In order to improve the power consumption and cost per bit, the industry is working together to launch 800G data centers and start to move towards 1.6T and higher rates.

Key Elements in 800G Ethernet Network Migration

1.Increase Switch Port Density

With the advancement of technology and changes in market demand, the design and manufacturing technology of SerDes (Serializer Deserializer) and ASIC (application-specific integrated circuit) has been continuously improved. SerDes technology has the function of converting parallel data into serial signals, and realizes high-speed data transmission based on relatively few physical pins and circuit resources. This enables switches to support more ports with limited hardware resources. In addition, as a specially designed application-specific integrated circuit, ASIC can provide highly integrated hardware solutions. By using ASIC, switches can integrate multiple ports or interfaces on a single chip, thereby reducing physical space occupation and increasing switch port density.
Therefore, the total number of switches required for 400/800G networks is reduced accordingly, which further increases the demand for new optical modules and structured cabling.

2.Optical Module Technology
New Generation Of Optical Module Formats

The CFP formats was first released by the CFP MSA Association and applied to early 100G optical modules. With the development of chip technology, MSA also launched the CFP2, CFP4 and CFP8 standards. The CFP8 package was launched in 2017 and applied to early 400G optical modules, which can support 16 channels of 25GNRZ signals for 400G transmission. Later, it was gradually replaced by the smaller QSFP-DD (double-density quad-channel) small pluggable format and OSFP package. The OSFP package supports eight groups of high-speed electro-optical module channels and can provide connection interfaces up to 400Gbos (8x50G PAM-4). Now QSFP-DD and OSFP have become the preferred packaging technologies for most manufacturers.
In general, with the upgrade of optical module packaging technology, the power consumption of fiber module is getting lower and lower, and the size is getting smaller, which is of great significance to the migration of 400G/800G data centers.

Higher-speed modulation scheme

For a long time, 1G, 10G, and 25G optical modules have been using non-return-to-zero (NRZ) modulation technology. Unlike NRZ, PAM4 uses four signal levels, and each symbol period can represent two bits. At the same baud rate, PAM4's throughput is twice that of NRZ, effectively reducing the loss of the transmission channel and improving bandwidth utilization.

In 400G/800G data centers, the NRZ modulation scheme requires a large amount of optical fiber. In addition, the timing margin, transmission link loss and size of the optical module chip cannot meet the requirements of 400G/800G Ethernet. With the development of big data and cloud computing, PAM4, as the most efficient modulation technology, has become an inevitable trend in the development of 400G high-speed ports. At present, the 400G interface standards of 4*100G PAM4 and 8*50G PMA4 have been formally proposed by the IEEE working group and applied to 200G/400G/800G optical modules. In the near future, PAM4 will become the mainstream method of high-speed Ethernet signal modulation with its own advantages (such as high performance, etc.). With the migration of Ethernet, signal modulation technology will continue to develop and innovate in a more complex direction.

3.Fiber Connector Options

MPO fiber connectors and LC fiber connectors are widely used in 400G/800G optical modules. MPO fiber connector are usually used for multimode fiber transmission - the MPO connector has a high-density design that allows multiple fibers to be connected simultaneously. In the MPO cabling solution, 400G Ethernet is usually transmitted through 8 optical fibers, each with a rate of 50Gbps; 800G Ethernet through 16 optical fibers, each with a rate of 50Gbps.

Duplex LC fiber connector are generally used for single-mode fiber transmission, and each LC connector connects two optical fibers. In order to increase the number of channels and channel rates, duplex connectors with a smaller footprint can provide more flexible branching options for high-speed modules. The 400G connection solution uses four duplex LC connectors, and 800G uses eight duplex LC connectors. By using duplex connectors, single-mode fiber can achieve high bandwidth and long-distance transmission. At the same time, single-mode fiber has good isolation capabilities, which improves the transmission quality of the signal.

In summary, MPO fiber connectors and duplex LC fiber connectors play an important role in the high-speed, bandwidth transmission of 400G/800G Ethernet. At present, the choice of these two types of fiber patch cord is no longer determined by the rate alone, but is affected by more factors, such as the number of data channels supported, the space occupied, and the price of optical modules and switches. In addition, in order to meet the needs of different scenarios, the choice of optical module connectors is also showing an increasingly diversified trend. In the future, more customized optical module connector designs will be produced for larger 800G data centers. The organization will also propose new standards to meet the market demand brought by 800G Ethernet.

4.Advances in Cabling Technology

To meet the challenges of 800G Ethernet migration, fiber optic cabling brings opportunities in the following aspects: OM5 fiber type, wavelength division multiplexing (WDM), ultra-low loss (ULL) components, reducing the number of server leaf switches (TOR), higher fiber count cabling, single-mode fiber and the right combination of multi-fiber applications.
●OM5 fiber: Compared with previous generation products, OM5 fiber has two major advantages - the attenuation (3dB/km) is lower than OM3 and OM4 (3.5dB/km); the effective mode bandwidth (EMB) specification is extended from 850nm to 953nm, which is more conducive to the application of SWDM technology.
●WDM: Currently, data centers mainly use two WDM technologies: coarse wavelength division multiplexing (CWDM) and dense wavelength division multiplexing (DWDM). They can provide more wavelengths and expand the capacity of each optical fiber, which is suitable for longer distance applications and saves costs.
●ULL: This technology includes high-quality components such as low-loss connectors, sockets, optical fibers, and splitters. With ULL components, optical modules have lower insertion loss and return loss, and can provide more stable and high-quality signal transmission.

How to Implement 800G Ethernet Migration?

1.Designing a higer-speed infrastructure
To redesign the data center for 800G Ethernet, we need to consider the points mentioned in the previous section.
●Switch Port Density: Increase the port density of each switch or layer the switch architecture to reduce the number of switch fabric layers.
●Optical Module Technology: QSFP-DD and OSFP formats (backward compatible QSFP+ and QSFP28).
●Fiber-optic cabling: New cabling designs such as 200mm and crimpable ribbon fiber that minimize footprint, or leaf-spine architectures that optimize direct paths for server-to-server communications.
2.Arrange on existing infrastructure
●Fiber Core: Use existing 8/12/24 fiber subunit trunks while adopting 16 fiber based designs in new high speed infrastructure.
●Instrument Testing and Verification: Test channel performance (including IL and RL) using handheld test equipment.

Conclusion

With the continuous development of technology and the promotion of commercial needs, it is expected that 800G Ethernet will enter the commercial stage in the next few years. In the future, the technological innovation and improvement of high-speed Ethernet will continuously improve its performance and application scope, bringing more convenience and opportunities to people.