Video description method based on sentence semantics and length loss calculation

The KL divergence loss function commonly used in existing models can represent a single mapping relationship of length distribution by learning specific visual content features, but it cannot accurately predict the length distribution of sentences in a one-to-many mapping relationship, resulting in the description of semantic information. incomplete. Second, the loss is calculated using a character-level negative log-likelihood function, ignoring sentence-level semantics, resulting in inaccurate semantic information of the generated descriptions. In this paper, a new length loss function is designed to adaptively adjust the error penalty by measuring the distance between the prediction and the reference length, so that the model can learn the optimal description length distribution in highly similar visual content. Secondly, through the semantic hidden state calculation of the decoding layer, a description generation loss function based on sentence semantics is designed, and the optimal sentence semantic description is iteratively obtained by comparing the semantics of the prediction and the reference description at the sentence level. The experimental results show that the model is tested on the MSVD and MSR-VTT datasets, and various indicators are significantly improved, which are better than the existing models.