© Rockport Networks 2021

rockport-networks-logo

IDC Technology Spotlight: Switchless Datacentre Networking: Defeating Network Congestion in Performance-Intensive Computing

As high-performance computing (HPC) evolves, it is converging with artificial intelligence (AI) and data analytics to create a new category that IDC calls performance-intensive computing (PIC).

In this new analyst report, IDC's Brad Casemore explores trends related PIC, including:

  • the challenges congestion causes in PIC workloads
  • the advantages of a direct interconnect architecture
  • how Rockport addresses congestion and latency

Enter your email to download the full white paper now.

Learn how to significantly reduce the network congestion that slows workload processing and hinders delivery of business outcomes.

Get a sneak peek below

Introduction

As high-performance computing (HPC) evolves, it is converging with artificial intelligence (AI) and data analytics to create a new category that IDC calls performance-intensive computing (PIC). IDC anticipates that the emergence of PIC will result in increased enterprise adoption of massively parallel computing to address artificial intelligence, business data analytics (BDA), and modelling and simulation (M&S) workloads in enterprise datacentres. Moreover, IDC recognizes that the optimal performance of PIC infrastructure depends on having the right datacentre network, including interconnects for host processor-accelerator-memory connections as well as networking between the nodes and between nodes and storage devices in a cluster. 

Unfortunately, while compute and storage infrastructure are adapting to accommodate the emerging requirements, conventional networks are struggling to keep pace in providing faster, simpler, and lower-latency interconnects. In this networking context, a particularly acute problem involves congestion mitigation.

To keep reading, download the full white paper today.

DOWNLOAD NOW

Addressing the Congestion Challenge

Congestion poses an intractable and costly problem, delaying PIC workload completion times and reducing overall cluster efficiency for workloads. By slowing workload completion, chronic congestion contributes to latency and makes PIC environments unpredictable and uncertain; this makes it difficult to reliably schedule cluster resources, compounding inefficiencies. In the most demanding environments, high-cost cluster resources can remain idle while waiting for delayed data to arrive.

Those responsible for managing these environments have tried to resolve the congestion issue in various ways. Some choose to add switches to their networks, while others increase available bandwidth, and others take both measures. Unfortunately, these efforts are often unsuccessful, adding complexity and cost without solving the underlying problem. 

Perhaps a better approach, capable of yielding an effective solution, involves a fundamental reappraisal of network architecture. Traditional switch-intensive network architectures have existed for decades, well before the arrival of modern HPC and PIC workloads. As such, traditional network architectures, and the switches they comprise, were not designed to provide the performance, elastic scale, economics, or simplified operations that are required now. 

The inherent design of traditional network architectures tends to inhibit PIC workload performance through both complexity, involving the number of switches and switch ports, and the exponential increase in interprocess communication (IPC) processes that occurs with increasing workload scale. Moreover, wiring complexity and high operational costs associated with endlessly adding switching capacity make scaling the network to meet PIC requirements untenable. As workloads scale, a greater number of IPC messages are generated to achieve synchronization. The growth in number of IPC messages is exponential, resulting in latency issues. This exacerbates an existing problem because traditional architectures are known to struggle with latency under load.

Whatʼs more, the problem is likely to worsen as high-performance applications advance, presenting the network with further challenges. Traditional architectures are destined to meet those challenges with the suboptimal and costly response of massive additions in network equipment, greater complexity, and higher operational costs. 


IDC Spotlight Cover