A Novel Attention-Based Multi-Modal Modeling Technique on Mixed Type Data for Improving TFT-LCD Repair Process

In Thin-Film Transistor Liquid-Crystal Display (TFT-LCD) manufacturing, conducting a machine learning based system with multiple data types has become actively desired to solve complicated problems. This paper proposes a multi-modal learning approach:
TabVisionNet
, which is modeled by utilizing the information from both tabular data and image data. A novel attention mechanism called
Sequential Decision Attention
was integrated into the multi-modal modeling framework that improves the comprehension of the information from two modalities. This cross-modal attention mechanism can capture the complex relationship between modalities then gain better generalization and faster convergence in the training process. Conducting an experiment, the performance of our novel approach was significantly better than single-modal and other multi-modal learning approaches in our real case scenario.