Multimedia content analysis for e-learning application

Summary form only given. Web-based learning (a.k.a. on-line learning or e-learning) is rapidly emerging as an alternative to traditional classroom-based education. Many universities and industrial organizations have started offering remote education and training programs. As a result, the amount of instructional videos available on corporate intranets and the Internet is dramatically increasing. This, on one hand, opens up exciting possibilities for self-driven education with the flexibility to set one's own pace and focus; while on the other hand, it poses great challenges on the task of efficient content access, browse and retrieval. In this talk, I first give a general overview of e-learning application and briefly introduce related international standards, then present some of my recent work on automatically extracting semantics from e-learning content based on the analysis of multiple media information. These extracted metadata can then be used to construct video's table-of-content and facilitate nonlinear content access, browse and retrieval. To achieve this goal, an audio classification scheme is first constructed to partition a video into homogeneous audio segments using the support vector machine technique, then discussion scenes where students interact with the instructor by asking questions or making comments, are detected using statistical approaches. These discussion scenes are then further classified into either 2-speaker or multi-speaker discussions to help differentiate Q&A from classroom discussions. Meanwhile, the narration scenes where the instructor continuously lectures, are analyzed to identify scenes that contain different types of visual contents such as close-up views of the instructor, shots of presentation slides, Web-pages, or blackboard/whiteboard. Various audio and visual features are exploited to achieve above tasks.