LIGHTNING network and systems architecture

This paper describes a hierarchical WDM-based optical network testbed project that is being constructed to interconnect a large number of high performance computers to create a distributed shared memory environment. The objective of the architecture is to achieve scalability yet avoid the requirement of multiple wavelength tunable devices per node. It is a singlehop all-optical network; a packet remains in the optical form from source to destination. It features wavelength channel reuse at each level, allowing scalability to very large system sizes. It partitions the traffic between different levels of the hierarchy without electronic intervention in a combination of wavelength-division and space-division multiplexing. A significant advantage of this approach is its ability to vary dynamically the bandwidth provided to different levels of the hierarchy. Each node monitors the traffic intensities on each channel and can detect any temporal or spatial shift in traffic balance. LIGHTNING can dynamically reconfigure to balance the traffic at each level by reassigning wavelengths associated with each level in the hierarchy to a higher or lower level depending on need. Bandwidth reallocation is completely decentralized, achieving highly fault tolerant system behavior. This paper describes the system architecture, and the network and memory interface that have been developed in this project.