A scalable parallel formulation of the backpropagation algorithm for hypercubes and related architectures

We present a new technique for mapping the backpropagation algorithm on hypercube and related architectures. A key component of this technique is a network partitioning scheme called checkerboarding. Checkerboarding allows us to replace the all-to-all broadcast operation performed by the commonly used vertical network partitioning scheme, with operations that are much faster on the hypercubes and related architectures. Checkerboarding can be combined with the pattern partitioning technique to form a hybrid scheme that performs better than either one of these schemes. Theoretical analysis and experimental results on nCUBE and CM5 show that our scheme performs better than the other schemes, for both uniform and nonuniform networks