HBM vs Optical Interconnects

Different Layers of the Same Bandwidth Problem

The AI compute stack has a layered bandwidth hierarchy, and HBM and optical interconnects address different — but equally critical — levels. HBM operates at the innermost layer: the interface between processor cores and memory dies, separated by just a few millimeters of silicon. HBM achieves its extraordinary bandwidth (up to 8 TB/s in HBM3e configurations) by stacking DRAM dies vertically and connecting them with thousands of through-silicon vias, creating an ultra-wide 1024-bit or 2048-bit data bus. Optical interconnects, by contrast, handle everything beyond the package edge — chip-to-chip links within a server, server-to-switch uplinks, and the datacenter fabric that ties thousands of AI accelerators into a single training cluster. Neither technology can substitute for the other: faster HBM cannot compensate for a slow network fabric, and faster optical links cannot help if the GPU is bottlenecked reading weights from memory.

The Memory Wall vs. The Network Wall

AI workloads face two distinct bandwidth walls. The memory wall limits single-accelerator performance: during LLM inference, each generated token requires reading the entire model's weights from HBM, making token generation rate directly proportional to memory bandwidth. NVIDIA's B200 with 8 TB/s of HBM3e bandwidth can generate tokens roughly 2.4× faster than the H100's 3.35 TB/s, purely from the memory bandwidth improvement. The network wall limits multi-accelerator scaling: when training a model across thousands of GPUs, gradient synchronization requires all-reduce operations that push massive data volumes across the interconnect fabric. As clusters grow from hundreds to hundreds of thousands of accelerators, the aggregate network bandwidth must scale superlinearly — a challenge only optical interconnects can meet without the power budget consuming the entire datacenter.

Energy Efficiency at Different Scales

Both technologies are engaged in a race to reduce energy per bit, but at very different operating points. HBM3 achieves approximately 2.5 pJ/bit for memory access — a 68% improvement over GDDR6X — and HBM4 is pushing below 2 pJ/bit through shorter TSV paths and optimized signaling. Optical interconnects tell a more nuanced story: the fiber transmission itself is extraordinarily efficient (0.05–0.2 pJ/bit), but the electro-optical conversion at each end — the lasers, modulators, photodetectors, and transimpedance amplifiers — adds significant overhead. Current pluggable transceivers consume roughly 15 pJ/bit end-to-end, but co-packaged optics (CPO) have driven this down to 5–6 pJ/bit by eliminating power-hungry SerDes circuits. NVIDIA reports that CPO reduces networking power consumption by up to 3.5× compared to pluggable modules. For AI datacenters where interconnect power can consume 20–30% of total facility power, this efficiency gain is transformative.

Supply Chain Dynamics and Strategic Risk

HBM faces one of the most constrained supply chains in the semiconductor industry. Only three companies — SK Hynix, Samsung, and Micron — can manufacture it, and the complex 3D stacking process yields fewer good units per wafer than standard DRAM. HBM has become the single most supply-constrained component in AI infrastructure, often more so than the GPUs themselves. SK Hynix commands roughly 50% market share and has seen its revenue multiply as AI demand surged. Optical interconnects have a broader but still concentrated vendor ecosystem: Broadcom leads in switch ASICs and CPO integration (with its Tomahawk 6-Davisson achieving the first 102.4 Tbps CPO switch), while companies like Coherent, Lumentum, and II-VI supply the optical components. The transition to CPO introduces new supply chain dependencies on silicon photonics fabrication, which relies on TSMC and GlobalFoundries process nodes — adding another layer to an already complex AI supply chain.

Convergence: When Memory Meets Optics

Perhaps the most exciting development is the emerging convergence of these two technologies. Researchers and companies are developing optical memory interfaces that would use photonic links to connect HBM stacks to processors, potentially achieving bandwidths exceeding 100 TB/s — an order of magnitude beyond current electrical interposers. This would effectively extend HBM's reach from millimeters to meters, enabling disaggregated memory architectures where pools of HBM could be shared across multiple accelerators via optical fabric. NVIDIA's roadmap hints at this direction: its 2026 platforms integrate CPO for switch-level interconnects, and future generations may bring optical I/O directly onto GPU packages. Intel has demonstrated a fully integrated optical I/O chiplet that could serve as a bridge between on-package memory and optical fabric. If realized, this convergence would dissolve the current boundary between the memory hierarchy and the network hierarchy entirely.

Investment and Market Trajectory

Both technologies are experiencing explosive market growth driven by AI demand. The HBM market is projected to exceed $25 billion by 2026, driven by insatiable demand from NVIDIA, AMD, and hyperscaler custom accelerators. The optical interconnect market for AI datacenters reached $9.94 billion in 2025 and is projected to hit $31 billion by 2033, with 800G transceiver shipments doubling year-over-year in 2025 alone. The critical difference is maturity: HBM is a proven, volume-production technology with a clear generational roadmap (HBM4 through HBM8), while CPO — the most transformative optical interconnect architecture — is just entering initial commercial deployment in 2026, with large-scale adoption not expected until 2028–2030. For infrastructure planners, HBM choices are constrained by what accelerator vendors offer, while optical interconnect decisions involve active architecture choices between pluggable modules, linear-drive optics, and co-packaged solutions.