A Real-Time Semantic Model for Relevance and Novelty Detection from Group Messages

Group brainstorming is a common method to generate ideas for open-ended, complex problems. Groups are known to generate fewer and less novel ideas than an equal number of individuals working separately. One cause of this is the tendency of groups to converge prematurely to a small number of ideas and neglect others. It is known that adding structure to brainstorming in the form of external hints tends to increase group performance. Our goal is to develop a cognitively-inspired feedback system that evaluates ideas in real-time and decides what kind of intervention to make in a group brainstorming session, e.g., flag an idea for further exploration, move the conversation in a new direction, etc. In this work, we develop a computational model for detecting relevant, novel, and less elaborated ideas from group messages in real-time. The flagged ideas can be fed back into the group discussion for further elaboration. We explore multiple approaches for relevance detection that do not require out-of-domain data: a probabilistic language model, a zero-shot classifier, and a one-class support vector machine. We compute two types of novelty, (1) absolute novelty with respect to domain knowledge; and (2) relative novelty with respect to previously seen ideas within the group. For absolute novelty, we explore methods to represent the domain semantic knowledge in a compressed way for faster computation. We tested our model on a dataset of ideas generated by over 50 groups. It was found that our probabilistic relevance method is the most accurate and that either topic cluster centers or common domain topics offer similar novelty results compared to full domain representation.