HBM vs CXL: Two Different Approaches. One Bigger Vision for the Future of HPC Memory

Speed or Scale? Why Next-Generation Supercomputers Need Both

The race toward Exascale computing and trillion-parameter AI models is exposing a challenge that every HPC architect recognizes: The memory subsystem is becoming the limiting factor.

Modern CPUs and GPUs continue to deliver unprecedented compute capability, but feeding data to these processors efficiently has become increasingly difficult. As AI training datasets expand and scientific simulations become more complex, memory architecture is emerging as one of the most important design considerations in next

The New Memory Bottleneck

For decades, the performance of supercomputers was primarily driven by faster processors and increasing core counts. Today, the challenge has shifted. Modern AI accelerators, GPUs, and HPC processors can execute trillions of operations per second, but they often spend valuable cycles waiting for data.

As High-Performance Computing (HPC), Artificial Intelligence (AI), and Exascale Supercomputing continue to push computational boundaries, the industry's focus is rapidly shifting from compute capability to memory architecture.

Today's processors and accelerators can execute trillions of operations per second, but many applications remain constrained by how quickly data can be delivered to the compute engines. This has elevated memory from a supporting component to a strategic differentiator.

Two technologies have emerged at the center of this transformation:

High Bandwidth Memory (HBM) and Compute Express Link (CXL).

At first glance, they appear to address similar challenges. In reality, they solve entirely different problems.

HBM is about delivering data at unprecedented speed.

CXL is about making memory scalable, shareable, and composable.

Rather than competing technologies, they represent complementary building blocks for the next generation of AI systems, HPC clusters, and data-center infrastructure.

Introduction

The rapid growth of Artificial Intelligence (AI), High Performance Computing (HPC), Large Language Models (LLMs), and data-intensive applications is pushing modern server architectures beyond traditional memory limitations. Today's processors and accelerators require both enormous memory bandwidth and massive memory capacity, yet achieving both simultaneously remains a significant engineering challenge.

Two technologies have emerged as key enablers of next-generation computing systems:

• High Bandwidth Memory (HBM), which focuses on delivering unprecedented memory bandwidth and ultra-low latency.
• Compute Express Link (CXL), which focuses on memory expansion, pooling, sharing, and scalability.

Although both technologies address memory-related challenges, they tackle fundamentally different problems. HBM accelerates computation by feeding data to processors faster, while CXL enables flexible and scalable memory architectures that extend beyond the physical limitations of a single server.

As AI models grow from billions to trillions of parameters and HPC simulations consume terabytes of memory, understanding the relationship between HBM and CXL becomes critical for designing future data centers and supercomputers.

The Growing Memory Challenge

Modern workloads face two major bottlenecks:

Bandwidth Bottleneck

Processors and GPUs can perform calculations much faster than traditional memory can supply data.

Capacity Bottleneck

Applications often require more memory than can physically fit inside a server.

These challenges have driven the development of two different architectural approaches:

Challenge	Technology
Need data faster	HBM
Need more memory	CXL

Understanding HBM: Bringing Memory Closer to Compute

HBM (High Bandwidth Memory) is an advanced DRAM technology that places multiple memory dies in vertically stacked layers directly beside or on the same package as the processor or accelerator.

The goal is simple:

Minimize the distance between compute and memory. By dramatically increasing the number of memory channels and shortening physical interconnects, HBM provides:

1. Extremely high memory bandwidth
2. Very low latency
3. Improved power efficiency
4. Higher accelerator utilization

High Bandwidth Memory (HBM) is a specialized memory technology designed to maximize memory throughput.

HBM achieves this through:

• 3D stacked memory dies
• Wide memory interfaces
• Short physical distance between processor and memory
• Advanced packaging technologies

HBM is commonly found in:

• NVIDIA H100/B100 GPUs
• AMD Instinct accelerators
• AI training systems
• HPC supercomputers

The primary goal of HBM is simple:

Move massive amounts of data to the processor as quickly as possible.

Advantages of HBM

1. Extremely high bandwidth
2. Very low latency
3. Excellent power efficiency per bit transferred
4. Ideal for AI training and scientific computing

Limitations of HBM

1. Expensive
2. Limited capacity
3. Difficult to expand
4. Requires advanced packaging

Understanding CXL:

HBM (High Bandwidth Memory) is an advanced DRAM technology that places multiple memory dies in vertically stacked layers directly beside or on the same package as the processor or accelerator.

The goal is simple:

Minimize the distance between compute and memory. By dramatically increasing the number of memory channels and shortening physical interconnects, HBM provides:

1. Extremely high memory bandwidth
2. Very low latency
3. Improved power efficiency
4. Higher accelerator utilization

Compute Express Link (CXL) is a cache-coherent interconnect technology built on top of PCIe.

Unlike HBM, CXL is not a memory technology itself.

Instead, it enables:

• Memory expansion
• Memory pooling
• Memory sharing
• Composable infrastructure

CXL allows CPUs and accelerators to access external memory resources while maintaining coherency.

The primary goal of CXL is:

Make memory more scalable, flexible, and efficiently utilized.

Advantages of CXL

1. Expand memory beyond motherboard limits
2. Memory pooling across multiple systems
3. Better memory utilization
4. Composable infrastructure
5. Lower infrastructure cost

Limitations of CXL

1. Higher latency than local memory
2.  Lower bandwidth than HBM
3.  Additional fabric complexity

HBM vs CXL: Side-by-Side Comparison

Feature	HBM	CXL
Type	Memory Technology	Interconnect Technology
Primary Goal	Speed	Scalability
Bandwidth	Extremely High	Moderate
Latency	Very Low	Higher
Capacity Expansion	Limited	Excellent
Memory Pooling	No	Yes
Memory Sharing	No	Yes
Cost per GB	High	Lower
Best Use Case	AI Training	Large Memory Workloads

A Real-World AI Example

Consider an AI server training a trillion-parameter model.

Without CXL :

GPU + 192GB HBM

The GPU performs computations extremely quickly but eventually runs out of memory.

With CXL :

GPU + 192GB HBM

       |

      CXL

       |

   4TB Memory Pool

Now:

• HBM stores active tensors and frequently accessed data.
• CXL stores less frequently accessed model data.
• Applications can work with significantly larger datasets.

The combination provides both performance and capacity.

Which Technology Will Have a Bigger Impact?

This is the most interesting question.

In the Short Term: HBM

HBM will continue driving AI innovation because training and inference performance depend heavily on memory bandwidth.

Without HBM:

• Modern GPUs cannot achieve peak performance.
• Large AI models become computationally inefficient.

For AI accelerators, HBM is indispensable.

In the Long Term: CXL

Many industry experts believe CXL could have a broader architectural impact.

Why?

Because CXL changes how data centers are built.

Instead of:

CPU + Fixed Memory

Future systems may use:

Compute Pool

Memory Pool

Accelerator Pool

Storage Pool

connected through a CXL fabric.

This enables:

• Dynamic resource allocation
• Better utilization
• Lower costs
• Larger memory footprints

The transformation is similar to how virtualization changed compute infrastructure years ago.

The Future: HBM + CXL Together

The future is unlikely to be "HBM versus CXL."

Instead, it will be:

HBM + CXL

where:

HBM

Acts as the performance tier.

Stores:

• Active tensors
• Hot datasets
• Frequently accessed data

CXL

Acts as the capacity tier.

Stores:

• Large datasets
• Checkpoints
• Embeddings
• Shared memory pools

This creates a hierarchical memory architecture optimized for both speed and scale.

Conclusion

HBM and CXL represent two complementary approaches to solving modern memory challenges. HBM delivers the extreme bandwidth and low latency required by AI accelerators and HPC processors, while CXL introduces a new paradigm of memory expansion, pooling, and composable infrastructure.

HBM focuses on making computation faster. CXL focuses on making memory larger, more flexible, and more efficiently utilized.

Rather than replacing one another, these technologies are expected to work together in future server architectures. HBM will remain the high-performance memory closest to the processor, while CXL will provide scalable memory resources across servers and data-center fabrics.

As AI models, scientific simulations, and data-intensive applications continue to grow, the next generation of computing systems will likely be defined not by HBM alone or CXL alone, but by the intelligent combination of both technologies.

Final Thought

HBM wins the performance battle. CXL wins the scalability battle. The future of HPC and AI belongs to architectures that successfully leverage both.