Subjective and Objective Quality Assessments of AI-Generated Images for Language E-Learning

AI-generated images (AIGIs) are becoming popular and can be employed in many applications, owing to Generative AI (GAI). Researchers have developed models that can be used to generate images for different scenarios. In addition, researchers have proposed datasets of natural scene images for language learning and different AIGI-quality datasets for general applications. For e-learning, particularly in the context of language learning, no AIGI dataset is currently available. To fill this gap, we first proposed an AIGI quality dataset for language learning. Both subjective and objective assessments have been conducted on the proposed dataset. The findings from subjective assessment show that higher perceptual quality also corresponds to a more substantial alignment. It also shows that the average MOS scores of images generated from Stability AI models are similar and lower than images generated by the Dall.E3 model. The results of the objective assessment indicate that the performance of off-the-shelf quality models is generally low. In addition, results from finetuning learning-based quality models show that significant gains and improvements can be achieved using the dataset. The results of the alignment evaluation show that the HPS model is the best, and realistic images in the dataset produced the best alignment correlation compared to the other styles in the dataset. The findings also show that multimodal large language models, such as vision-enabled GPT-4 (GPT-4V), still struggle to produce alignment scores that correlate with humans.