Enhancing Text Segmentation With Positional Encoding and Sentence Embeddings: An Unsupervised Clustering Approach

Text Segmentation involves dividing documents into thematically cohesive segments and it is essential for improving the effectiveness of document-level applications, such as summarization, question-answering, and information retrieval. In this paper, we propose a novel unsupervised approach for segmenting text content that combines advanced sentence embeddings generated by the Sentence-BERT model with document-level positional encoding and K-means clustering. By incorporating positional information, we enhance the embedding vectors to capture both the semantic relationships and the structural context of sentences within a document. This enriched representation aids in producing more contextually aware and coherent clusters. We evaluate the proposed method using the Choi dataset and compare the performance of clustering-based segmentation with and without positional encoding. Our experimental results show that the inclusion of positional encoding significantly improves segmentation quality, as indicated by the increment of the average V-measure from 0.53 to 0.73. This unsupervised approach provides a scalable and versatile solution for Text Segmentation, effectively bridging the gap between semantic coherence and document structure while eliminating the need for annotated training data.