L-VEIGe: A Proposal for Quantitative Evaluation Model and Database for Error Vocabulary Images

Vocabulary learning is a crucial aspect of second language acquisition. Learning vocabulary in context is recognized as effective for acquiring word concepts. However, beginners often struggle with the burden of reading large amounts of text. To address this challenge, methods that use visual information as contextual support have gained attention. Despite the increasing use of visual aids in vocabulary learning, the focus has largely been on correct answers. Errors, however, have received little attention. In second language acquisition, there is also the issue of fossilization, where repeated errors become ingrained. To tackle this problem, we developed L-VEIGe, a system designed to prevent repeated errors. It does so by visualizing learners’ incorrect answers through image generation, which promotes introspection. While L-VEIGe has proven effective, a Feature Disappearance problem was identified. Some generated error images lacked the necessary information to understand the errors. This study aims to address this issue. We define the essential information that error images should contain from a Cognitive Fidelity perspective. We then construct a model for automatically and quantitatively evaluating these features. This is achieved using image captioning, LVLM-based VQA, and object detection. Additionally, we developed an annotation dataset to train a model that can infer these evaluation values from a human cognitive perspective.