What is Compute Express Link (CXL)?
Learn all about CXL, understand how it works, read use cases, and much more.
CXL is a cache-coherent open interconnect standard for high-speed CPU connection to memory and other devices. Compute Express Link leverages the standard PCIe® physical layer and runs as a supported alternate protocol. By creating a common memory space for connected devices, the CXL standard brings performance advantages for hyperscalers and other advanced applications.
- Compute Express Link utilizes a flexible processor port that can operate in either PCIe or CXL modes. Both device classes can achieve data rates of 32 GT/s in PCIe5.0 or up to 64 GT/s in PCIe6.0.
- The CXL Consortium was founded in 2019 by nine industry-leading organizations to develop technical specifications, support emerging use case models, and advance CXL technology development and adoption.
- Artificial Intelligence (AI), Machine Learning(ML), and cloud infrastructure are among the applications that benefit most from the extremely low latency and coherent memory access provided by the CXL interface.
In summary, the Compute Express Link framework establishes coherency between the memory of the CPU and each connected device. This allows memory resources to be pooled and shared efficiently even as the software stack complexity is reduced. To enable memory pooling, both the host and peripheral device(s) must be CXL-enabled. Data transfer is completed using low-overhead flow control units or “flits”.
-
CXL Switching allows the host to fan out to multiple devices while maintaining high throughput in each direction. Resources including accelerators and any available CXL memory can be dynamically re-assigned as the server workload changes.
-
The CXL 2.0 specification also includes a standardized fabric manager. This ensures a seamless user experience with consistent configurations and error reporting regardless of the pooling type, host, or usage model.
As the interface has evolved, unique use cases and applications have led the CXL Consortium to define three discrete device types.
CXL Device Types
- Type 1 Devices: Accelerators and other devices that lack local memory and therefore must rely on the CPU are classified as Type 1. The CXL.io and CXL.cache protocols enable these devices to communicate and transfer memory capacity from the host processor more efficiently.
- Type 2 Devices: Products that include their own data storage capabilities but also leverage CPU memory are known as Type 2. All three CXL protocols combine to promote coherent memory sharing between these devices and the CPU.
- Type 3 Devices: Memory expanders or devices designed to augment existing CPU memory are classified as Type 3. The CXL.io and CXL.memory protocols enable the CPU to access these external sources with improved bandwidth and latency performance.
Compute Express Link Benefits
By streamlining connectivity and resource sharing, CXL technologies provide numerous enhancements that improve high-capacity workload performance while reducing system complexity and cost. These attributes become increasingly valuable as next-generation data centers and emerging technologies drive demand for faster data processing and lower total cost of ownership (TOC).
- Coherency CXL memory pools to remain consistent with respect to data validity. This allows for faster and more efficient resource sharing between devices and processors.
- Heterogeneous architecture, combining processors of varying types and generations, is fully accommodated by the CXL standard. This is especially useful for complex AI neural networks and machine learning systems as elements of the infrastructure evolve.
- Lower latency is the result of strategically pooled persistent memory, improved CXL switching efficiency, and standardized memory management. Reduced latency is considered a key enabler of next-generation use cases and future PCIe 6.0 adoption.
The release of the CXL 1.0 standard in 2019 was a significant milestone marked by CPU access to shared accelerator device memory. Compute Express Link protocols and standards have continued to improve and expand since this successful debut.
- CXL 1.1 improved compliance and interoperability aspects of the original standard while maintaining backwards compatibility with release 1.0.
- CXL 2.0 added switching capabilities for fan-out configurations, resource pooling, and persistent memory support while minimizing the need to overprovision resources. Link-level Integrity and Data Encryption (CXL IDE) were also incorporated to improve security.
- CXL 3.0, released in August of 2022, doubled the data rate to 64GT/s and expanded packet size from 68 to 256 bytes. Fabric management, memory sharing, and peer-to-peer communications were enhanced without introducing additional latency.
- Sub-protocols developed for CXL specification 1.0 have remained consistent throughout the compute express link lifecycle:
- CXL.io is based on the PCIe 5.0 protocol and is used for discovery, configuration, and to register access functions. CXL.io must be supported by all Compute Express Link devices to function.
- CXL.cache manages interactions between the CPU (host device) and other Compute Express Link-enabled devices. This sub-protocol supports the efficient, low-latency caching of host memory and direct device access to CPU memory using a request and response process.
- CXL.memory provides modes of access for the host to provision attached device memory using load and store commands. In this configuration, the CPU acts as a master with the Compute Express Link device(s) acting as subordinates.
CXL and PCIe
PCI Express (PCIe) has become the de-facto high-speed serial bus architecture over the past two decades, with point-to-point topology providing high-speed links to connected devices. Despite the proficiency of PCIe for bulk data transfer, shortcomings become obvious in larger data center applications. Memory pools remain isolated from one another, which makes significant resource sharing nearly impossible and adds to the latency deficit for newly connected devices.
- PCIe 6.0 is the latest backward-compatible generation of the Peripheral Component Interconnect Express standard. When fully adopted, CXL for PCIe6.0 will result in a doubling of throughput from PCIe5 along with support for alternate protocol deployment that is now being utilized by the CXL interface.
- Operating over the PCIe physical layer, CXL protocols build upon the versatility of standard PCIe architecture by integrating new memory-sharing functionality within the transaction layer. CXL memory pooling enables system latency reduction and improved accelerator performance.
- Standard PCIe devices and CXL software can be supported over the same link. A flexible processor port can quickly negotiate either standard PCIe or alternate protocol CXL interconnect transactions.
VIAVI CXL Products
VIAVI Xgig Analyzer solutions for PCIe 5.0 support CXL.cache/memory transactions and triggers. The Xgig captures valuable real-time metrics and performs detailed analytics over multiple traces simultaneously.
VIAVI Xgig Exerciser solutions support CXL compliance and traffic generation.
VIAVI PCIe 5.0 interposers like the Xgig PCIe 5.0 16-lane CEM Interposer can be used to capture CXL traffic running on a PCIe physical layer. The interposer creates a bi-directional interface between the protocol analyzer and system under test.
VIAVI and The CXL Consortium
Established in 2019, the CXL Consortium is an open industry standard group formed to develop technical specifications that facilitate breakthrough performance for emerging usage models while supporting an open ecosystem for data center accelerators and other high-speed enhancements.
VIAVI is a proud member of the CXL Consortium, contributing decades of high-speed serial bus test and validation expertise to new working groups and specification development as the unique test requirements evolve.
The History of Compute Express Link
CXL was first introduced to the world in March of 2019 with the release of the CXL 1.0 specification. Encompassing only minor updates, CXL 1.1 was released just three months later. The significant upgrade represented by version 2.0 in November of 2020 included link switching, link security, and other important features that directly addressed system performance and fast-tracked adoption. Hot-plug flows were also defined to add or remove resources reliably for applications like CXL over Ethernet.
Many Compute Express Link devices are currently in development, and compliance programs are beginning, underscoring the need for reliable CXL protocol analysis and validation solutions.
CXL 2.0 and 3.0 are available today supporting 32GT/s and 64GT/s links respectively.
