The program of grand challenge problems: expectations and results

The US high performance computing community still uses the term "Grand Challenge" for a variety of difficult problems in computational science. Though it was primarily a means of communicating computing goals to nonpractitioners, it also serves the useful purpose of letting practitioners focus on defining goals more carefully. For purposes of Grand Challenge computing, it is essential to have precise goals, and a way of measuring progress toward those goals. Many Grand Challenges have neither. A particularly common error is to measure the size of a computing problem with some integer "N" that represents the number of grid points or the number of particles or some other count of a discrete quantity. Another common error is to use measures of hardware activity, such as floating point operations per second, as a valid goal for an application programmer. The paper presents an approach to measuring the progress of physical simulations that shows promise for putting computational efforts on firmer scientific ground.