Building Trust During Negotiations: Exploring the Relationship Between Cooperation and Exploitation Through an Integrated Analysis of Speech and Facial Expressions

Negotiation is an important means of creating value beyond individual interests. Still, it is necessary to first assess the personality of the other party (exploitative or cooperative) and the feelings of the parties during the negotiation. By adapting to the situation, it is possible to build trust and lead to valuable negotiation outcomes. Previous studies have not adequately linked personality and behavior (statements and facial expressions) to negotiation outcomes. Many human-AI negotiation studies are also based on mere assumptions about the AI's emotional expressions and the way it speaks. Therefore, this study aims to collect human-to-human negotiation data for a more realistic analysis. Specifically, using a multi-argument ultimatum game that allows both cooperative and exploitative behavior, we will combine the analysis of statements using a large-scale language model (LLM) with the analysis using facial expression recognition technology to clarify the relevance of negotiation outcomes as well as the correlation between statements and facial expressions. As a result of this research, we will model the compatibility between behaviors, facial expressions, and negotiation outcomes, leading to a foundational technology for more reliable AI systems.