Fine-Tuned training method for semantic text similarity measurement using SBERT, Bi-LSTM and Attention Network

The Semantic Textual Similarity (STS) problem measures the degree to which two text fragments are similar. In QA systems, it is a crucial tool for extracting useful information from documents. In STS challenges, optimizing lexical gaps between phrases has been a crucial issue. Text similarity may be affected by the distance between major parts of the sentence and the absence of context in the text. Numerous studies have developed answers to this issue. In recent years, however, Sentence-BERT has become widespread in semantic textual similarity tasks. Siamese BERT-Networks generate embeddings that are more context-aware and semantically intact. In addition, BiDirectional LSTM provides a deeper grasp of the input and captures long-range dependencies, whereas the Attention mechanism highlights the most important information aspects from the data. Due to the success of SBERT and Bi-LSTM in exploiting lexical gaps and semantics, we introduce SBERT-Bi-LSTM-Attention (SBiLA), a training model that combines SBERT, Bi-LSTM, and a simple attention strategy. The model improves similarity by addressing the lexical gap between short and long texts and reduces training loss to 0.052, MSE to 0.042, and RMSE to 0.206 on the SQuAD 2.0 and WikiHow datasets.