Today, InfiniBand is primarily used as a high-performance fabric interconnect for linking a cluster's compute nodes within a high-performance data centre. Until now, use of InfiniBand for wide area networks has been restricted, due to the inability to transport native InfiniBand traffic over large distances. As such, implementation across computing systems that cover several geographically separated data centres has also been restricted.

Extending InfiniBand across the metropolitan area network (MAN) will allow enterprises to greatly enhance not only their supercomputing, but also their disaster-recovery and storage-networking applications, using InfiniBand as the underlying unified network architecture.

To address all these market requirements, ADVA Optical Networking, Obsidian Strategics and Voltaire have partnered to deliver a complete InfiniBand-over-distance solution for HPC and enterprise applications. The major building blocks of the joint solution are director-class InfiniBand switches (Voltaire) interconnected via InfiniBand range extenders (Obsidian) and bit-transparent, low-latency WDM transport devices (ADVA Optical Networking).

The HLRS solution comprised the following elements. More than 500 computing nodes of the new Baden-Württemberg Grid Cluster were clustered via two 288-port Voltaire Grid Director 2012 switches. They were interlinked using Obsidian's short-reach Longbow Campus devices for serializing and range-extending the 4X SDR InfiniBand signal. The ADVA WDM FSP devices (FSP 2000 or FSP 3000) were then used to interconnect the sites using protocol-agnostic, Dense Wavelength Division Multiplexing (DWDM) that provided bit-transparent, lowest-latency transport. The whole solution was built using generally available devices.

"High performance, low latency and its unified fabric capability have made InfiniBand a steadily more popular connectivity solution for inside the data centre. Until now, InfiniBand has been limited as an inside-the-data-centre solution", stated Prof. Dr.-Ing. M. Resch, Director HLRS. "The ability to extend the protocol over distance at full throughput offers totally new HPC scenarios."

"We believe that WDM transport is the natural choice for latency-sensitive, high-throughput protocols such as InfiniBand. Its increasing adoption rate in research networks as well as in the financial industry motivated us to develop a solution that helps to further consolidate data centre traffic onto a unified fabric. InfiniBand-over-WDM offers unbridled throughput for enterprises' most-demanding applications", stated Dr. Christoph Glingener, chief technology officer for ADVA Optical Networking.

Asaf Somekh, vice president of strategic alliances, Voltaire Ltd., stated: "InfiniBand has established itself as the advanced interconnect standard of choice to power many of the world's biggest and fastest computer systems and enterprise data centres. This deployment at HLRS shows InfiniBand can be easily extended for scalable WAN storage capabilities."

Added Dr. David Southwell, president and chief executive officer with Obsidian Strategics: "In the face of the sustained exponential growth of intra- and inter-data center bandwidth demand, we are pleased to work with ADVA Optical Networking and Voltaire, deploying Longbow solutions that unlock the enormous potential of both WDM and InfiniBand technologies for high-performance data transport."