Realization of Keyboard Instruments on Mobile Platform with Mixed Reality

Today's smart mobile devices are used for communication purposes as well as for many digital and nondigital devices and objects. With evolving technology, mobile devices also support technologies such as augmented reality, virtual reality and mixed reality. In this article, we present the realization of keyboard instruments for mobile platforms with mixed reality. With the pilot implementation, users are expected to be able to use Launchpad and piano instruments with their smartphones. With the developed mixed reality application, the user can use these instruments through an A3 or A4 size paper template printout. The performance of the application was evaluated by FPS measurement tests on 20 different devices under different conditions. As a result of the evaluation, average 54 in the wait state, 40 in finding and tracking the target, and 31 FPS during the virtual button usage were obtained.