Content and Relation Fuzzy Mitigation Framework for Intent Perception

In this article, we tackle the content fuzzy and relation fuzzy in image-based intent perception. Current research primarily focuses on additional modeling mechanisms and multimodal information, but it is often difficult to deal with the relation fuzzy under inaccurate annotation in intent perception. We exploit the powerful representation capabilities of large language models and innovatively use them to solve the image-based intent perception task. Comprising two innovative modules: multidimension adapter and fuzzy harmony refiner. Multidimension adapter projects intent image features of different difficulty levels to feature dimensions of different sizes to alleviate content fuzzy of different difficulties caused by image diversity. Besides, intent category labeling is somewhat subjective, and some categories are more likely to appear collaboratively. We design fuzzy harmony refiner to mine the relationship between various types to enhance perception results. Different from traditional expert-specified rules, we learn the co-occurrence frequencies between different intent categories from the training data, generate a fuzzy harmony matrix for correcting the score, and alleviate the relationship fuzzy in intent perception. We achieved new state-of-the-art performance on the Intentonomy dataset, with macro F1 score of 42.52%, micro F1 score of 54.80%, samples F1 score of 57.57%, and average F1 score of 51.63%. Compared with the current existing methods, our method improves by 9.8%.