Wearable Structured Mental-Sensing-Graph Measurement

The emotional assessment under the Internet of Things (IoT) architecture can support researchers to establish the relationships between human social and physiological signals and emotions. In this article, a wearable emotion-sensing system is developed under narrowband Internet of Things (NB-IoT) wireless communication technology. The wearable sensing device integrates socially linked sensors including voice, activity, and heart rate. Using this system, a dating experiment is set up to investigate multimodal factors of male’s attractiveness perception. In particular, the multimodal data are fused in a graph structure, and this further leads to a graph convolutional neural networks model for emotion evaluation and a mental-sensing-graph intelligent interpreter. Different types of mental-sensing-graphs are fused during the training stage, and the model achieves a verification accuracy of 0.93. The intrinsic relationships among the multimodal data have been captured by the subgraphs which have star-shaped structures, and the center of the subgraphs are mostly audio nodes. The obtained results show that the attractiveness perception of the male participants in dating is more aligned with language communication. The results also reveal that when the male participants date highly attractive women during the experiments, a significant correlation is observed between the multimodal features and the attractiveness perception levels.