PCI Express goes zoom-zoom

I’ve seen relatively little coverage about yesterday’s approval of the PCI Express 2.0 specification by PCI-SIG, the industry consortium that, well, defines the PCI Express specification.

With the introduction of PCI Express (aka, PCIe) a couple of years ago, server expansion cards got a huge performance boost, with the biggest impact on RAID and other disk controllers, and also on high-speed, low-latency interconnects, like you’d see in high-performance clusters. (PCI Express is found on desktop systems, too, with the biggest payoff in graphics cards. There, PCIe replaced older standards like AGP.)

The beauty of the PCI Express system is that it’s transaction-based. Yes, this makes it considerably more complex than many previous bus designs. In fact, PCIe incorporates a full networking stack. However, that gives it much greater flexibility and scalability.

Each PCI Express interface and peripheral engages in transactions using some number of asynchronous one-bit-wide serial lanes, which are used to aggregate bandwidth. Devices can use links that are between 2 and 32 lanes wide. (That’s in contrast to older buses, which were synchronous and parallel at 16, 32 or 64 bits, and which were therefore hard to sync at faster clock speeds.)

The PCI Express 2.0 spec doubles the data rate from 2.5GT/s (that’s 2.5 billion transactions per second) to 5.0GT/s. Each lane’s performance jumps from 2.5Mb/s to 5.0Mb/s.

That extra bandwidth opens up great possibilities. Think of what it can mean when linking CPUs and GPUs, for example. It also raises the ceiling for low-latency interconnect performance for clusters.

According to PCI-SIG, the PCI Express 2.0 spec adds additional features: device drivers can dynamically control link speed, for example. The bus also supports devices that consume higher power.

As servers and desktops get faster (think about two quad-core processors in a desktop, or four quad-core in a typical server) the bottleneck isn’t the processor or the peripheral. It’s the bus. PCI Express 2.0 is a solid step forward.