LIGHTNING: a scalable dynamically reconfigurable hierarchical WDM network for high-performance clustering

LIGHTNING is a dynamically reconfigurable WDM network testbed project for supercomputer interconnection. This paper describes a hierarchical WDM-based optical network testbed that is being constructed to interconnect a large number of supercomputers and to create a distributed shared memory environment. The objective of the hierarchical architecture is to achieve scalability yet avoiding the requirement of multiple wavelength tunable devices per node. Furthermore, single-hop all-optical communication is achieved: a packet remains in the optical form from source to destination and does not require intermediate routing. The wavelength multiplexed hierarchical structure features wavelength channel re-use at each level, allowing scalability to very large system sizes. It partitions the traffic between different levels of the hierarchy without electronic intervention in a combination of wavelength- and space-division multiplexing. A significant advantage of this approach is its ability to dynamically vary the bandwidth provided to different levels of the hierarchy. Each node in LIGHTNING receives traffic on n channels in an n-level hierarchy, one channel for each level. Each node monitors the traffic intensities on each channel and can detect any temporal or spatial shift in traffic balance. LIGHTNING can dynamically reconfigure to balance the traffic at each level by moving wavelengths associated with each level up or down depending on need. Bandwidth re-allocation is completely decentralized-any node can initiate it, achieving highly fault tolerant system behavior. This paper describes the system architecture, network and memory interface, and the optical devices that have been developed in this project.