SC17 Denver, CO

The Ethernet Portfolio for HPC

Authors: John D'Ambrosia (Ethernet Alliance, Huawei)

Abstract: Over the past 15 years, Ethernet has been heavily leveraged as the interconnect backbone for many HPC systems that have shown up on the Top500. Demand for increased bandwidth has driven the development of new Ethernet solutions operating at rates up to 400 Gb/s. The introduction of these standards will drive the performance of tomorrow’s networks to never-before-seen levels and bring about the Next Ethernet Era. This session will provide the audience with an overview of the Ethernet solutions and technologies that will be leveraged for the next generation of computing, networking, and storage, and discuss their deployment.

Long Description: Over the past 15 years, Ethernet has been heavily leveraged by the HPC Industry. Consider Ethernet-based systems hold a 70% or greater share of the largest segment, “Industry”, reported by the Top500. These systems were initially based on Gigabit Ethernet, but in the past 5 years have shifted to 10 Gigabit Ethernet. This fact is an indication of the significance of the importance of commodity Ethernet-based systems to deployments in cost-efficient HPC environments.

Cost pressures and demand for increased performance continues unabated, and the Ethernet community has responded. In 2015, 25 GbE was standardized. This rate of Ethernet emerged from 100 GbE development, which uses four lanes 25 Gb/s to achieve 100 GbE. The use of serial 25Gb/s I/O enabled a higher density and lower cost server interconnect than 40GbE, which requires four differential pairs in each direction. Products are now finding their way into deployments.

With a generation beyond 10 Gb/s now available to the HPC community, the IEEE set off on developing the next generation server interconnect, focusing on 50 GbE and 200GbE. This work is building on the decisions made during the development of 400GbE, which saw PAM4 modulation to achieve 50 Gb/s lane rates for both electrical and optical signaling adopted, as well as the use of forward error correction (FEC), to create 400GbE links.

The introduction of a new modulation scheme, as well as the use of forward error correction, is fundamental to the shift to higher lane rates, which in turn are being used in a bonded fashion to create faster Ethernet pipes. While these technologies, in general, are not new, their application into many Ethernet physical layer specifications are. This introduces new challenges in an environment where multiple vendors will be present and interoperability will be a necessity. The availability of testing tools to verify operation per specifications will also be critical.

This Ethernet Alliance session will leverage the expertise of its membership, which consists of various participants in the HPC eco-system, to provide the HPC community an update on Ethernet’s future portfolio. The session will focus on providing education in three areas – 1) the state of standards, physical layer specifications, and technologies development for 25GbE, 50GbE, 100GbE, 200GbE, and 400GbE; 2) the deployment of these technologies in today’s and tomorrow’s architectures, and; 3) progress of testing and multi-vendor interoperability. In addition, updates to the Ethernet Alliance Ethernet Roadmap, based on the latest IEEE 802.3 Ethernet standards activities will be presented. Today’s Ethernet standards and related technologies that are being introduced for 25GbE and developed for 50 / 200 / 400 GbE are the commoditized technologies of tomorrow. These solutions will be the basis of tomorrow’s Ethernet-based HPC systems, and enable performance levels never seen. This session will help to prepare the attendee for the coming Ethernet HPC upgrade!

Conference Presentation: pdf

Birds of a Feather Index