EffectON — Enriching impressions of motions and physically changing motions via synchronous sound effects

In this paper, we introduced EffectON, a system that enables users to hear sound effects, i.e., audio feedbacks, of their routine motions. We believe that there are three main benefits of people hearing sound effects corresponding to their motions: sonification of the motion or action, modification of the impression of motions, and a physical change in the motion itself. The unique hypothesis of this research proposes that by changing the condition of the sound effect itself and its presentation, all the three benefits would have different characteristics. These benefits must be highly subject to the way sound is presented, i.e., it would depend on parameters of sound, such as the length, volume, timing, and type, as well as the conditions of motion chosen (e.g., types, sites, parameters, etc.). We have developed two prototype devices: a step-based device and a knee-bend based device. The report first focuses on the third benefit of physically changing the users' motion by changing the “duration” of sound effects. To be specific, the step-based device is mainly used in this report to focus on the “gait,” and four sound effects of two different types and durations were implemented to see if the walking speed of the subjects would change on hearing them. The experimental result showed that users naturally walk faster on hearing shorter duration sounds. This implies that there is great potential for naturally controlling or inducing a certain motion by appropriately choosing the sound effects that correspond with the motion.