Investigate the Impact of Colour to Grayscale Conversion on Sound Recovery via Visual Microphone

Visual microphone is one of the remote sound acquisition methods utilizing the subtle vibrations caused by sound on an object to partially recover the sound that causes them via a video footage of the object. Although good quality sounds have been successfully recovered using the visual microphone technique, no study has considered the impact of the chosen colour-to-grayscale conversion methods on the sound recovery. Studies in computer vision areas such as image recognition found that the method used to convert colour to grayscale had significant influence on the performance and results. Thus, this work was performed to evaluate the impact of different colour to grayscale conversions on sound recovery using the visual microphone technique. An intelligibility metric was used to evaluate the sounds recovered using the different conversions. The results showed that the choice of colour to grayscale conversion did have an impact on sound recovery via the visual microphone technique. The conversion based on weighted averaged of pure colour channels (red, green and blue), e.g., Intensity and Luminance, and perceptually uniform grayscale representation used in International Commission on Illumination (CIE) 1976 L*a*b*, performed better when compared to other methods.