Scaling AI Infrastructure With PCIe Switches And Retimers

Artificial intelligence is changing more than the amount of compute inside the data center: it is changing how architects think about where resources live, how they connect, and how efficiently they are used.

AI training, inference, high-performance computing, analytics, and cloud services all demand greater data movement and more accelerators. However, these workloads do not share one ideal architecture. Some need resources located as close together as possible, while others benefit from expanding across servers. Still, others can gain efficiency by sharing expensive devices through a common fabric.

The result is not a wholesale move from traditional servers to one universal disaggregated design. It is a hybrid future built from scale-up, scale-out, and selective disaggregation with PCI Express (PCIe) providing a familiar foundation for connecting critical resources.

Fig. 1: Different workloads create different requirements for bandwidth, latency, locality, and utilization.

One pressure, three architectural responses

Scale-up brings more compute and acceleration into a tightly coupled system. This approach can deliver excellent locality and low latency, making it attractive for demanding AI workloads. However, it eventually encounters practical limits in processor lanes, enclosure space, power, cooling, and the number of devices a host can support.

Scale-out expands performance across multiple servers using Ethernet or InfiniBand. It is essential for distributed computing and cluster growth but introduces network latency and requires software that can coordinate workloads and data across nodes.

Disaggregation separates selected resources such as GPUs, NVMe storage, or SmartNICs/DPUs from fixed server ownership, and makes them available through a shared fabric. This can improve utilization and allow compute and endpoints to scale independently. However, sharing also introduces new considerations around contention, isolation, management, and latency.

To decision makers, the distinction matters because each model changes system performance, capital efficiency, deployment flexibility, and operational complexity. The right question is therefore not, “which architecture wins?” but rather “which resources should stay local, which should scale across servers, and which are valuable enough to share?”

PCIe 7.0 creates new bandwidth headroom

PCIe is already the standard attachment technology for CPUs, GPUs, accelerators, SmartNICs/DPUs, and NVMe storage. Its established hardware and software ecosystem makes it a practical starting point for next-generation infrastructure.

PCIe 7.0 doubles the raw per-lane data rate of PCIe 6.x to 128 GT/s and delivers up to 512 GB/s of bidirectional bandwidth in a x16 configuration. That bandwidth supports growing accelerator counts and increasing data-intensive platforms while preserving backward compatibility with the broader PCIe ecosystem. Not only is PCIe 7.0 a major force in data centers today, but PCIe 5.0 and PCIe 6.x also remain major building blocks for today’s infrastructures.

Of course, higher speed does not automatically create a scalable system. As signaling rates rise, channel loss, reflections, crosstalk, noise, and jitter become more difficult to manage. Architects also need a way to connect more endpoints than native host ports alone can accommodate. This is where PCIe switches and retimers take on complementary roles.

Fig. 2: PCIe raw transfer rate per lane has doubled with each generation, reaching 128 GT/s with PCIe 7.0.

Switches provide scale and traffic management

A PCIe switch expands connectivity by receiving transactions, identifying their destination, and forwarding them between upstream and downstream ports. It enables higher fan-out, more flexible topologies, and peer-to-peer paths among connected devices.

But port count and peak speed tell only part of the story. In real systems, GPUs, storage, and control traffic may compete for bandwidth at the same time. Arbitration, buffering, credit management, oversubscription policy, and avoidance of head-of-line blocking all affect usable performance.

Switch topology matters as well. Every additional hop adds latency and creates another potential point of contention. A well-designed fabric must provide the required connectivity while minimizing unnecessary stages on latency-sensitive paths.

For silicon teams, configurability is especially valuable. Support for different port widths, upstream-port options, endpoint counts, and traffic patterns can allow one switch architecture to serve multiple products and deployment models.

Retimers provide reach and signal integrity

As resources move farther from the host – across longer board traces, connectors, cables, expansion chassis, or rack-oriented designs – the electrical channel becomes more challenging. A PCIe retimer recovers the clock and data, terminates one electrical segment, and retransmits a clean signal into the next.

Retimers can restore channel margin and give architects greater physical flexibility, but they do not make reach unlimited. Each retimed segment must still be engineered and validated. Designers must account for latency, power, thermals, equalization, diagnostics, and interoperability across hosts, endpoints, PHYs, connectors, cables, and firmware.

The goal is not to add as many retimers as possible. It is to use the minimum number required to meet reach and signal-integrity objectives with acceptable system tradeoffs.

Fig. 3: Switches expand and direct connectivity; retimers extend reliable electrical reach to shared resource pools.

From faster links to better infrastructure

In simple terms, a switch decides where traffic goes and a retimer helps it arrive reliably. Rambus PCIe Switch and Retimer Controller IPs provide complementary building blocks for ASIC and FPGA implementations. The Switch IP addresses connectivity, fan-out, and traffic management while the Retimer IP addresses signal regeneration and reach. Together, they can help design teams build configurable PCIe subsystems spanning local expansion, shared resource domains, and longer-reach topologies.

The strongest evaluation begins with a representative workload and topology, not a feature checklist. Teams should define required bandwidth and latency, expected fan-in and fan-out, oversubscription, traffic mix, channel loss, power, manageability, and interoperability targets. They can then judge switch and retimer capabilities against measurable technical and business outcomes.

AI infrastructure will continue to evolve through a mix of tightly coupled systems, larger clusters, and selectively shared resources. PCIe 7.0, along with PCIe 6.x and 5.0 supply important bandwidth headroom, while switches provide scale and retimers provide reach. Used together, they give architects practical tools to connect more resources, place them more flexibly, and extract greater value from next-generation infrastructure.

Links

The post Scaling AI Infrastructure With PCIe Switches And Retimers appeared first on Semiconductor Engineering.