SpeechSyncNet: Speech to Talking Landmark via the fusion of prior frame landmark and the audio

Accurately generating talking landmarks from audio is critical for creating authentic talking head animations. This is a significant issue in the field of landmark generation from audio and has potential applications in areas such as virtual assistants, education, and entertainment. Previous methods evaluate the effectiveness to generate the talking landmark from audio, however, these methods have limitations such as bias and inconsistencies in landmark generation due to the lack of visual information from the previous frames. In this research, we propose SpeechSyncNet, a baseline that integrates a fusion of the prior landmark information and the audio feature to provide the lack of visual information from audio, also this method aims to improve the consistency of the landmark motion. Our proposed method yields competitive results compared to existing state-of-the-art methods and helps to improve the quality of talking face landmark generation.