Blog

Why Memory Bandwidth and Interconnect Now Limit AI Performance, and Where the Semiconductor Value Is Moving

Mobility Foresights Technology Research | Hub Analysis | September 2026

The raw compute of AI accelerators has grown far faster than the speed at which data can be moved to them. Performance is now limited by two "walls":

• The memory wall: how fast data can be fed from memory to the processor.
• The network wall: how fast thousands of processors can exchange data when training or serving a model.

The industry has responded with three waves of investment. High-bandwidth memory (HBM) has become one of the most supply-constrained products in semiconductors. Rack-scale scale-up fabrics connect dozens of GPUs as one system, and new open standards such as UALink and Ultra Ethernet challenge proprietary interconnects. Optics is moving ever closer to the chip. At the other end of the market, AI is moving into PCs, phones, cars and microcontrollers, creating a second, high-volume compute market.

This hub brings together 68 Mobility Foresights market studies across six areas: HBM, DRAM and AI memory; memory modules, CXL and storage; AI accelerators and chip design; AI networking and interconnect; optics, photonics and cabling; and edge and consumer AI compute.

Key Takeaways

• HBM is the most strategic memory product. HBM4 moves the base die onto logic foundry processes and enables customer-specific designs, tying memory makers more closely to foundries and chip designers.
• Memory is diversifying beyond HBM. LPDDR-based server modules, CXL expansion and storage-class memory address capacity and power needs that HBM alone cannot.
• Custom silicon is gaining share. Hyperscaler-designed accelerators and inference chips are growing alongside merchant GPUs, and AI is now used to design chips.
• Interconnect is a battleground of standards. Scale-up and scale-out fabrics, SmartNICs and connectors are growing faster than compute itself.
• Optics is approaching the chip. 1.6T modules, co-packaged optics, active electrical cables and micro-LED links compete to move data at lower power per bit.

How We Segment AI Compute, Memory and Interconnect

1. HBM, DRAM and AI Memory

HBM stacks DRAM dies vertically next to the processor, delivering bandwidth that conventional memory cannot match. Supply has been sold out well ahead, and the leading memory makers have shifted capacity toward it. Four reports cover the core:

• Our AI data center memory market report sizes all memory in AI servers.
• The AI infrastructure memory market study adds edge and enterprise AI infrastructure.
• The DRAM for AI market report covers the DRAM wafer demand that HBM, server DIMMs and LPDDR draw from.
• The AI GPU memory stack market study covers the memory attached to each accelerator.

The HBM roadmap is changing the structure of the industry:

• The HBM3E 12-high market report covers the current volume generation.
• With HBM4, standardized by JEDEC in 2025, the base logic die at the bottom of the stack can be made on foundry logic processes. The HBM4 base-die foundry market study covers this new foundry opportunity.
• The custom HBM market report covers base dies designed for a specific customer's accelerator.

Inference and power efficiency are creating new memory categories:

• The AI inference memory market report covers memory configurations optimized for serving models, where capacity per dollar matters as much as bandwidth.
• The low-power AI memory market study covers power-efficient memory for AI devices and servers.
• The LPDDR6 DRAM market report covers the next mobile memory standard, used increasingly in servers as well as phones and PCs.

2. Memory Modules, CXL and AI Storage

Beyond HBM, AI servers need large pools of system memory. Four reports cover server memory:

• Our AI server memory market report sizes system memory in AI servers.
• The AI server memory module market and AI memory module market studies cover DIMM and module formats, by server and across AI systems.
• The AI server SOCAMM market report covers a new compact module format based on low-power DRAM, introduced for AI servers in 2025.

Memory can now be pooled and tiered:

• The CXL memory expansion market report covers Compute Express Link devices that add memory capacity beyond what processor channels allow.
• The AI storage class memory market study covers memory that sits between DRAM and flash in speed and cost.
• The DDR memory controller market report covers the controllers that manage these memory interfaces.

AI training and inference also depend on fast storage:

• The AI-optimized SSD market report covers high-capacity and high-performance SSDs designed for AI data pipelines.
• The cloud storage controller market study covers the controllers inside them.
• The memory products for on-device AI market report covers memory in AI PCs, smartphones and edge devices.

3. AI Accelerators and Chip Design

Merchant GPUs dominate AI training, but large cloud providers have developed their own accelerators for training and inference, often designed with custom-silicon partners. Two reports cover the market:

• Our AI accelerator semiconductor market report sizes GPUs, custom ASICs and other accelerators.
• The AI inference accelerator market study focuses on inference, where cost per query and power efficiency favor specialized designs.

Chip design itself is being transformed by AI:

• The AI-native chip design platforms market report covers electronic design automation (EDA) platforms built around machine learning.
• The agentic AI in EDA market study covers AI agents that automate verification, layout and optimization tasks.

Two reports cover newer architectures. The multimodal AI chip market report covers processors optimized for combined vision, language and audio models. The laser-cooled GPU chips market study covers experimental approaches that use photonics to remove heat from dense processors.

4. AI Networking and Interconnect

AI clusters need two kinds of network. "Scale-up" links a rack or pod of accelerators so they act as one large processor. "Scale-out" connects thousands of these pods across a data center. Proprietary scale-up fabrics set the pace, and the UALink consortium, formed in 2024, and the Ultra Ethernet Consortium are building open alternatives. Five reports cover scale-up:

• Our AI GPU interconnect market report covers GPU-to-GPU links.
• The AI accelerator interconnect market and AI XPU connectivity market studies cover links for GPUs, ASICs and other accelerators.
• The AI rack-scale interconnect market report covers the copper and optical backplanes that tie racks together.
• The AI memory fabric system market and AI memory connectivity market studies cover fabrics that let processors share memory.

Scale-out and cluster networking:

• The AI data center networking market report sizes AI data center networking.
• The AI cluster networking market and AI scale-out networking market studies cover InfiniBand and Ethernet fabrics for large clusters.
• The AI fabric switch market report covers the switches.
• The AI inference networking market study covers networks optimized for low-latency serving.
• The network interface controller market and AI-driven network offload and SmartNIC market reports cover the adapters that connect servers to the fabric and offload networking and security tasks from CPUs.

Four reports give the connectivity view, and two cover connectors:

• The AI infrastructure connectivity market, AI datacenter connectivity market and AI datacenter interconnect market reports cover connectivity at the infrastructure, facility and inter-data-center levels.
• The AI accelerator connectors market and GPU server connectors market studies cover high-speed connectors, where signal integrity at 200G per lane is a major engineering challenge.

5. Optics, Photonics and Cabling

Moving data by light becomes necessary as bandwidth and distance grow. Two reports cover pluggable optics and their lasers:

• Our AI datacenter optics market report covers 800G and 1.6T pluggable modules.
• The AI data center electro-absorption modulated laser (EML) market study covers the lasers inside many high-speed modules, a recurring supply bottleneck.

Photonics is moving onto the package:

• The AI silicon photonics market report covers photonic circuits built on silicon.
• The in-package optical interconnects market study covers co-packaged optics that place optical engines next to switch and processor chips. Major switch vendors announced co-packaged optics products in 2025.

Copper still dominates short links:

• The AI data center active electrical cable market report covers cables with built-in signal conditioning that extend copper reach inside and between racks at lower power than optics.
• The AI datacenter cabling market study sizes all cabling in AI facilities.

Newer optical approaches:

• The AI data center micro LED market and AI server micro LED data transmitter market reports cover micro-LED-based links that promise very low power over short distances.
• The analog optical computer (AOC) market study covers research into optical systems that perform AI calculations directly with light.

6. Edge and Consumer AI Compute

AI is moving onto devices for privacy, latency and cost reasons. PC makers introduced AI PC categories requiring dedicated neural processing units (NPUs) in 2024, and flagship smartphones now run generative models locally. Five reports cover consumer AI compute:

• Our AI-optimized PC processor market report covers AI PC processors.
• The AI smartphone compute market study covers smartphone AI compute.
• The consumer AI compute market and AI consumer edge compute market reports size consumer AI hardware across devices.
• The edge AI in consumer electronics market study covers AI features in wearables, TVs and home devices.

At the chip level:

• The edge AI NPU market report covers neural processing units.
• The low-power AI accelerator market study covers ultra-efficient accelerators.
• The edge AI microcontroller market and edge AI on MCUs and embedded processors market reports cover microcontrollers with built-in AI acceleration, a category that expanded rapidly from 2024.

Industrial edge AI and edge infrastructure:

• The industrial and IoT edge AI semiconductor market and edge AI accelerators for industrial systems market reports cover industrial edge AI chips.
• The AI compute at the edge market study sizes edge compute overall.
• The edge AI inference infrastructure market, edge AI connectivity market and edge AI networking infrastructure market reports cover the servers and networks that serve AI close to users.

Where Value Is Created: The AI Compute Value Chain

Regional perspective

• United States. The center of AI accelerator, networking and EDA design, and the largest buyer of AI infrastructure.
• South Korea. The leading source of HBM and DRAM production.
• Taiwan. The hub for leading-edge foundry, advanced packaging, servers and optics assembly.
• China. Building domestic accelerators and memory under export controls, creating a partly separate AI hardware ecosystem.
• Japan and Europe. Important suppliers of materials, equipment and photonic components.

What to Watch Through 2032

1. HBM4 and custom HBM ramps: yields and the role of foundries in base-die production.
2. Open interconnect adoption: UALink and Ultra Ethernet deployments against proprietary fabrics.
3. Co-packaged optics: reliability and serviceability in production switches.
4. Custom accelerator share: hyperscaler silicon against merchant GPUs, especially for inference.
5. On-device AI: whether AI PCs and phones create a replacement cycle.

Frequently Asked Questions

What is high-bandwidth memory (HBM)?

HBM stacks multiple DRAM dies vertically and places them next to the processor on the same package, providing much higher bandwidth than conventional memory. It is essential for AI accelerators.

What changes with HBM4?

HBM4 doubles the interface width and allows the base die at the bottom of the stack to be manufactured on logic foundry processes, enabling custom features for specific accelerators.

What is CXL?

Compute Express Link is an open interconnect that lets processors access additional memory devices and share memory pools, increasing capacity beyond what the processor's own memory channels support.

What is the difference between scale-up and scale-out networking?

Scale-up networking links accelerators within a rack or pod so they behave like one large processor. Scale-out networking connects many such pods across a data center for large training and inference jobs.

What are co-packaged optics?

Co-packaged optics place optical transceivers in the same package as a switch or processor chip, shortening electrical paths and reducing power per bit compared with pluggable modules.

Why is AI moving to edge devices?

Running AI on PCs, phones, cars and industrial devices improves privacy and responsiveness and reduces cloud costs. Dedicated NPUs and AI-enabled microcontrollers make this practical.

What do Mobility Foresights AI hardware reports include?

Each linked report provides market size and multi-year forecasts, technology and supply-chain analysis, segmentation by product, application and region, competitive landscape and company profiles, and recent developments. Customization is available on request.

Work With Mobility Foresights

Mobility Foresights provides syndicated and custom research across AI accelerators, memory, networking, optics and edge AI. Each report linked above can be purchased individually or tailored to a specific component, technology or region. To request a sample or commission a bespoke study, contact our technology research team.