High-Speed Training Networks

High-speed training networks are the specialized interconnect fabrics that link thousands of GPUs and AI accelerators within datacenters, enabling the distributed computation required to train large AI models. Training a frontier foundation model involves synchronized computation across tens of thousands of GPUs, and the network connecting them is often the bottleneck that determines training speed, cost, and feasibility.

The fundamental challenge is data movement. During distributed training, GPUs must constantly exchange gradients, activations, and model state. A training cluster of 10,000 GPUs running data-parallel and pipeline-parallel training generates petabytes of inter-node traffic per day. The network must handle this with minimal latency and maximum throughput, because any GPU waiting for data from another GPU is wasting expensive compute time.

InfiniBand (from NVIDIA/Mellanox) has been the dominant AI training interconnect, offering 400 Gb/s per port (with 800 Gb/s and 1.6 Tb/s generations arriving) and RDMA (Remote Direct Memory Access) for low-latency GPU-to-GPU communication that bypasses the CPU entirely. NVIDIA's DGX SuperPOD and GB200 NVL72 systems use InfiniBand backbones combined with NVLink for intra-node GPU communication at 900 GB/s per GPU.

Ethernet alternatives are emerging as competitors. Ultra Ethernet Consortium (backed by AMD, Broadcom, Cisco, and others) is developing AI-optimized Ethernet extensions. Google's custom Jupiter network fabric uses Ethernet at scale for TPU training clusters. The argument for Ethernet is ecosystem breadth, cost, and avoiding NVIDIA lock-in; the argument for InfiniBand is proven performance and tight integration with NVIDIA's GPU stack.

Optical interconnects are increasingly critical. As bandwidth demands exceed what electrical cables can deliver over datacenter distances, optical links using silicon photonics and co-packaged optics are moving from the network edge to within racks and even within systems. This enables the bandwidth density needed for next-generation training clusters.

Network topology matters significantly. Fat-tree topologies provide full bisection bandwidth (any GPU can communicate with any other at full speed) but are expensive. Dragonfly and rail-optimized topologies reduce cost by providing higher bandwidth within local groups. The choice of topology directly affects which parallelism strategies (data parallel, tensor parallel, pipeline parallel, expert parallel) are efficient for a given model architecture.

The networking challenge will intensify as training clusters scale from thousands to hundreds of thousands of GPUs. Multi-datacenter training — synchronizing computation across geographically distributed facilities — introduces wide-area networking challenges where speed-of-light latency becomes a constraint that no amount of bandwidth can overcome.