Multimodal Document Classification Based on Two-Stream Adaptive Feature Fusion

The number of tobacco image files is enormous, and there is an urgent need for automated processing to improve efficiency and accuracy. While computer vision and deep learning have been widely applied to document classification, relying solely on visual analysis is insufficient to achieve the fine-grained classification required in real-world scenarios. To address this, we propose a multimodal tobacco document classification network called DSANet (Dual-Stream Adaptive Network), which integrates both image and text features of tobacco documents. The network employs a DSAFF (Dual-Stream Adaptive Feature Fusion) module to fuse the dual-stream features, enabling adaptive adjustment of weights according to the importance of features from different modalities. This approach enhances flexibility and precision in multimodal feature fusion while increasing complementarity between different modalities and eliminating redundant features, thereby strengthening information relevant to the classification task. Experimental results demonstrate that the DSANet network outperforms previous classification methods on tobacco document datasets, providing a new solution for the automation and digital transformation of tobacco industry documents.