Robust highlight extraction using multi-stream hidden Markov models for baseball video

This paper proposes a robust statistical framework to extract highlights from a baseball broadcast video. We applied multi-stream hidden Markov models (HMMs) to control the weights among different features. To achieve robustness against new highlights, we used a common simple structure for all the HMMs. In addition, scene segmentation and unsupervised adaptation were applied to achieve more robustness against the differences of environmental conditions among games. The precision rate of high-light extracting experiments for eight kinds of highlights from 4.5 hours of digest data was 77.4% and was increased to 78.7% by applying scene segmentation. Furthermore, the unsupervised adaptation method improved precision by 2.7 points to 81.4%. These results confirm the effectiveness of our framework.