Refresh rate modulation for perceptually optimized computer graphics

The application of human visual perception models to remove imperceptible components in a graphics system, has been proven effective in achieving significant computational speedup. Previous implementations of such techniques have focused on spatial level of detail reduction, which typically results in noticeable degradation of image quality. We introduce Refresh Rate Modulation (RRM), a novel perceptual optimization technique that produces better performance enhancement while more effectively preserving image quality and resolving static scene elements in full detail. In order to demonstrate the effectiveness of this technique, we have developed a graphics framework that interfaces with eye tracking hardware to take advantage of user fixation data in real-time. Central to the framework is a high-performance GPGPU ray-tracing engine. RRM reduces the frequency with which pixels outside of the foveal region are updated by the ray-tracer. A persistent pixel buffer is maintained such that peripheral data from previous frames provides context for the foveal image in the current frame. Applying the RRM technique to the ray-tracing engine results in a speedup of 3.2 (260 fps vs. 82 fps at 1080p) for the classic Whitted scene without secondary rays and a speedup of 6.3 (119 fps vs. 19 fps at 1080p) with them. We also observe a speedup of 2.8 (138 fps vs. 49 fps at 1080p) for a high-polygon scene that depicts the Stanford Bunny. A user study indicates that RRM achieves these results with minimal impact to perceived image quality. We also investigate the performance benefits of increasing physics engine error tolerance for bounding volume hierarchy based collision detection when the scene elements involved are in the user's periphery. For a scene with a static high-polygon model and 50 moving spheres, a speedup of 1.8 was observed for physics calculations.