Toward an Application of Content-Based Video Indexing to Computer- Assisted Descriptive Video

This paper presents the status of a project targeting the development of content-based video indexing tools, to assist a human in the generation of descriptive video for the hard of seeing people. We describe three main elements: (1) the video content that is pertinent for computer-assisted descriptive video, (2) the system dataflow, based on a light plug-in architecture of an open-source video processing software and (3) the first version of the plug-ins developed to date. Plugs-ins that are under development include shot transition detection, key-frames identification, keyface detection, key-text spotting, visual motion mapping, face recognition, facial characterization, story segmentation, gait/gesture characterization, keyplace recognition, key-object spotting and image categorization. Some of these tools are adapted from our previous works on video surveillance, audiovisual speech recognition and content-based video indexing of documentary films. We do not focus on the algorithmic details in this paper neither on the global performance since the integration is done yet. We rather concentrate on discussing application issues of automatic descriptive video usability aspects.