Web Chat-based Application with Large Language Model and Transformers from Hugging Face for Self-Learning on Storytelling Skills

The ability of storytelling greatly affects a person’s success. These abilities are usually taught by parents at home or teachers at school. With the development of Artificial Intelligent technology, it is now possible to automatically generate stories using Large Language Model (LLM) which can understand and create language like humans. In this paper, the authors propose a self-learning system in storytelling by utilizing and combining four models from Hugging Face Hub. The proposed system is a web chat-based application so that users can communicate with LLM where LLM has received an image input from the users. The four models are as follows. Falcon 7B Instruct model as LLM that gets caption information from BLIP Image Captioning Large model. Its responses in the form of text can be read by the users and can be heard through audio synthesized by the MMS TTS Eng model. The user can also see the detected objects in the image which is detected by DETR ResNet 50 model. Our experiments show that the proposed system is sufficient to produce a good story and fit the context of the image, with an average user score of 89.76.