System for game enhancement using scene recognition in emulated consoles

This paper investigates the modification of an existing game console emulator utilizing scene recognition to perform audio-visual enhancement of vintage video games. Such games featured low-resolution graphics with limited colors and low-quality electronic music. Information contained about the virtual hardware state information is exploited to provide high-resolution, full-color graphics and a recorded soundtrack, complete with atmospheric effects, in real-time using the original game logic. The system developed as part of this research uses a modified version of the FCEUX emulator for the Nintendo Entertainment System. The system is capable of loading a commercially-released game, marking scenes for later recognition, mapping scenes to audio-visual events, and recognizing those scenes to enable those events. The result is an immersive game experience that far exceeds what was possible in the original game hardware.