Big Data, High-end Performance, and the Exascale Future of Scientific Research

The transformative power of technology innovation is pushing the boundaries of many fields, and none more significantly than research and education (R&E), which increasingly relies on advanced optics to meet the connectivity demands of unprecedented, multi-petabyte data volumes. Underpinning each new scientific discovery and research breakthrough is a high-performance network with unique capabilities.

Last week, we were excited and honored to take part in ESnet and Lawrence Berkeley National Laboratory’s formal unveiling of ESnet6 , the newest generation of the U.S. Department of Energy’s (DOE) high-performance network. This network represents a transformational change, delivering  significant capacity with improved automation and intelligence to enable cutting-edge research and scientific breakthroughs.

ESnet6 is built on ~15,000 miles of dark fiber and connects all the DOE’s national laboratories with tens of thousands of researchers providing over 46 terabits of aggregate capacity. In 2021 the network carried over 1.1 exabytes of data, and demand for capacity continues unabated. As a matter of fact, the traffic on ESnet increases by a factor of 10 every four years. As a long-standing ESnet partner, Infinera has helped power ESnet6 with advanced solutions that include the Infinera GX Series Compact Modular Platform and FlexILS Open Optical Line System.

ESnet Executive Director Inder Monga said, “​​ESnet6 provides the foundation for the future of the DOE mission science as we enter an age where discoveries will rely on the integration of scientific experimental facilities, supercomputers and global science researchers collaborating as if they are collocated. ESnet6 interconnects all of these resources to create a holistic science discovery system.”

Scale, agility, resiliency, and low-latency performance are critical requirements to meeting the broad range of science applications that geographically distributed researchers and educators rely on to advance their initiatives. These applications include high-performance computing for large-scale simulations and real-time analysis of experimental data.

Key advances in optical networking that offer new levels of performance in the world’s leading research and education networks include:

• Coherent optical transport – Leveraging sophisticated digital signal processors (DSPs) and advanced photonics, coherent optical technology has revolutionized DWDM transport, enabling wavelength speeds to go from 10 Gb/s in the pre-coherent era to 100 Gb/s and now 800 Gb/s with the latest embedded coherent optical engines. The addition of advanced features, such as soft-decision forward error correction (SD-FEC), Nyquist subcarriers, and probabilistic constellation shaping (PCS), has extended the capacity-reach of coherent optical wavelengths to thousands of kilometers, minimizing or eliminating the need for signal regeneration. For R&E network infrastructure, coherent optics provide the scale and bandwidth efficiencies required to enhance the lowest-latency services with maximum capacity.

Figure 1: High-speed optical evolution – wavelength capacity over a given distance by generation

• Disaggregation and open networking – Open optical networking provides operators of R&E network infrastructure with more choice and enables them to innovate each optical network layer at its own pace due to the disaggregation of transponders and line systems. Coherent optical transceivers leverage both the silicon performance improvement cycle described by Moore’s law and advances in photonic technology, resulting in a much faster innovation cycle relative to the optical line system. An open networking implementation enables fast onboarding of new technologies regardless of vendors, and operators can select from the market and implement new solutions faster whenever they become available. The key elements of open networking architectures are network layer disaggregation and standards-based open APIs, as illustrated in Figure 2.

 Figure 2: High-speed optical evolution – disaggregation and standards-based open APIs

• Automation/SDN/machine learning – Proactive, customizable, and more automated operation of optical networks and services represents another important enabler of R&E networks. In addition to increasing efficiency of operations, automation helps improve network visibility, planning and configuration, troubleshooting, network analytics, and more. Standards-based open APIs with common data models as well as streaming telemetry are key ingredients of modern open networking systems. Moreover, open APIs enable improved, more unified, and simplified management and operation of larger networks and are paving the way for enhanced network automation. In R&E networks, these capabilities can help redirect massive, high-bandwidth workflows of science data more seamlessly while, in the long term, enabling scientists to independently reserve network resources and services on demand.

From 800G coherent transport to open networking  and automation, Infinera is committed to delivering operators of industry-leading R&E network infrastructure like ESnet technologies purpose-built to manage the demands of big data and helping them deliver superior connectivity performance for their unique communities of users. With exascale-size data sets just around the corner, our partnership with ESnet will continue to pave the way for greater scale and enhanced capabilities unmatched in the field of global research and education.