FoodFlavorNet: A Multimodal Deep Learning Model for Food Flavor Recognition

Food plays an vital role in our daily life. With the development of computer science technology, there is an increasing emphasis on enhancing food-related experiences through Human-Food Interaction (HFI), a subfield of Human-Computer Interaction (HCI), dedicated to improve interactions between humans and food. However, existing studies often focus on recommendations based on user data, neglecting the mining of food intrinsic properties the critical role of flavor in determining food preferences and experiences, in this paper, we propose a multimodal deep learning model named FoodFlavorNet, to implement food flavor recognition in HCI. In this model, we design a Cross Attention Fusion Module (CAFM) to extract text feature and make fusion with image features extracted by Convolutional Neural Network (CNN), to recognize the flavor characteristics of food. This model is designed for HCI applications, where users can capture food images and input ingredient information to a server with model inference to botain detailed flavor recognition results, thus enhancing the overall dining experience. The experimental results indicate that the model is effective for food flavor recognition, demonstrating the potential of deep learning in HCI applications, offering a new solution for food flavor recognition and advancing the development of both HFI fields.