Identifying Document Images Based on Multi-Modal Large Models

To accurately identify the behavior of holding fake electronic entry and exit identity certificates, aiming at the problem that the traditional image discrimination model takes image data as input and cannot fully utilize the implicit semantic information in the image, this paper takes the real entry and exit identity certificate photos of a certain government agency as input, designs and implements an entry and exit identity certificate photo authenticity discrimination model based on a multimodal large model. Methodology Based on the Mini GPT4 image description, the model was generated and then introduced successively. The model was improved by methods such as Boostrapping, LoRA, and NEFtuning to generate image description information for multi-modal fusion of identification identity certificates, and was trained and verified respectively in models such as VGG16, ResNet, Xception, and ViT. The experimental results show that the ROOUGE-L value of the improved Mini GPT-4 large model is approximately 0.46, which is nearly 25 % higher than that of the original model; The accuracy of authenticity discrimination has been increased by nearly 2 % by using the photo data of entry and exit identity certificates with multimodal fusion. The research results of this paper can be applied to practical business such as the detection and interception of counterfeit and fake photos of entry and exit identity certificates.