Generation of Photo Slideshow with Song based on Closeness between Concept of Lyrics and That of Images

This paper proposes a method that allows users to easily convert a large number of still images into movies by displaying photos in sync with memories or favorite songs, which offers a new media viewing method for users to look back at the vast number of photos in their photo folders. We assume here that users select a song and have images stored in local PC. One method of synchronization involves selecting images based on elapsed time of song. However, since lyrics convey meaning, it can be better to display the images that align with both lyrics’ meaning and images’ meaning matched. In order to do this, we need the space which shares the concept of both lyrics and images, and select colors as the concept. Displayed images are determined based on the distance between points that are mapped from words and images to color space. Specifically, the color concepts from words are extracted using the Color Image Scale, and that from images are determined as the most frequent colors by cluster analysis. Experimental results show that it is possible to easily create movies where images with colors that match the mood of the lyrics are displayed.