Which comes first: the architecture or the algorithm?

There is a constant tension between the designers of algorithms and architectures. Each designs for the other's previous generation. Several factors are making this an increasingly inadequate approach. On the hardware side, the growing disparity between the performance of memory and CPU has pushed many architectures to hierarchical memories that reward significant data reuse (and punish algorithms that use data items a small number of times). On the algorithmic side, at least for a large class of algorithms in scientific computing, the emphasis on increasingly efficient algorithms, measured by the amount of work (floating point operations) per solution value, has led to algorithms that touch data only a few times. Further, algorithmic techniques that lead to adaptive methods (computing only with as much data as is required to accurately represent the solution) often lead to irregular or unpredictable accesses to memory. While these trends in architectures and algorithms are away from each other, there are other trends in algorithms that provide both a degree of greater memory locality without sacrificing algorithmic optimality. These are hierarchical methods, such as multigrid and general domain decomposition. These hierarchical methods place some requirements on architectures; primarily that there be no distinguished collection of processes/threads in a computation. Another opportunity in algorithms is the potential for exploiting split-phase or two-step operations.