Sentence Embedding Using Supervised Contrastive Learning on Hierarchical Categories

Supervised contrastive learning techniques have the potential to improve the expressive power of sentence embeddings; however, selecting appropriate positive and negative samples remains a significant challenge. In this paper, we propose a new sampling method, Hierarchical Classification Data-based Contrastive Sentence Embedding (HCD-CSE), which utilizes the hierarchical structures of datasets. One key insight of our approach is that datasets often possess hierarchical categories, which enable the selection of rich positive and negative samples. Specifically, given an anchor category, samples from closer categories (e.g., siblings) can be used as positive samples, while those from more distant categories serve as negative samples. We demonstrate the enhanced expressiveness of the sentence representations derived from our HCD-CSE approach through experimental results on category classification tasks.