Merging Neuroimaging and Multimedia: Methods, Opportunities, and Challenges

Neuroimaging and brain mapping can provide meaningful guidance to multimedia analyses. and advanced computational multimedia analysis can be used to better understand the functional mechanisms of the human brain. Essentially, brain imaging and brain mapping techniques can serve as a bridge that links the digital representation of multimedia and the perception and comprehension of its content. This paper summarizes methods that integrate brain imaging with multimedia analysis and discusses the opportunities and challenges in this interdisciplinary field. In general, quantitative modeling of brain responses during multimedia comprehension has advanced content-based multimedia studies such as image and video classification and tagging. Multimedia analysis has promoted functional brain mapping by using naturalistic multimedia as stimuli during neuroimaging. Challenges and opportunities in merging neuroimaging and multimedia include the quantification of the brain's responses, the quantification of multimedia, and the mapping between brain responses and computational multimedia features.