Impression Prediction of Oral Presentation Using LSTM and Dot-Product Attention Mechanism

For automatically evaluating oral presentations, we propose an end-to-end system to predict audience's impression for speech videos. Our framework is a multimodal neural network including two Long Short-Term Memory systems and a dot-product attention mechanism to learn linguistic features and acoustic features, respectively. An attention network is also used to consider the correlation between different feature representations for feature fusion. We utilize 2,445 videos with official captions and users' ratings from TED Talks. The experimental result shows the good performance for our proposed system, which can recognize 14 types of audience impressions with an average accuracy of 85.0%. Our proposal has the advantage of making noticeable improvements to the accuracy of predicting audiences' impressions.