Creating a Digital Human Twin: Cloning Voice, Face, and Attitude

This paper presents and explores the development of digital human twin through mobile app integrated with chat-bot and aimed to deliver dynamic and realistic user interactions. By utilizing advanced technologies such as reinforcement learning, lip-sync animation, and facial expression modeling. This project aims to develop a personal virtual assistant using various technologies. The system integrate various deep learning models such as 3D avatar generation, speech recognition, clone voice from the user, Large Language Model (LLM), 3D avatar after uploading images from the user, and engages in lifelike human conversation. In the evaluation of the system there's a potential to improve user experience. Moreover, the paper also includes a review of related datasets, highlighting the capabilities and challenges of creating a realistic virtual avatar.