TextFormer: Component-aware Text Segmentation with Transformer

In recent years, deep learning techniques have made significant advancements in text segmentation. However, most existing methods do not take into account that characters are composed of smaller components, such as strokes and other local patterns. Furthermore, the similarities between text components are crucial for effective text segmentation. With this in mind, we propose a multi-level Transformer-based method for text segmentation that incorporates a recognition module. To enhance the interaction between text components and extract features at different granularities, we introduce Global and Local Self-Attention blocks. Our recognition module is trained jointly with the segmentation module to improve the model’s ability to focus on text details and improve its perception of texts. By aggregating features from multiple granularities, our segmentation module produces accurate pixel-level mask predictions. The experimental results demonstrate the effectiveness of our approach on several text segmentation benchmarks and show that it outperforms existing methods.