Negative Siamese Network for Classifying Semantically Similar Sentences

In semantically similar sentence-level classification, BERT often neglects the subtle semantic differences in sentences and performs poorly because of its inherent anisotropy regarding semantic space. This paper explores methods to enlarge the space distance between labels assigned to different sentences. First, the sentence embedding is transferred into the isotropic space through a message passing neural network (MPNN). Combined with structure encoding, semantic encoding can alleviate the anisotropy character of BERT. In addition, we propose the Negative Siamese Network (NSN) to analyze hard negative examples. The NSN focuses on widening the distance between hard negative samples and positive samples. After fine-tuning the NSN model, we conduct two tasks on four standard datasets. Experiment results show that our model achieves better performance than state-of-the-art models in capturing the subtle differences between semantically similar sentences.