Classification of Relevant Comments from Competitive Programming Discussions

In competitive programming, understanding a problem often requires more than just the official solution. Users typically turn to comments in the contest’s thread for additional insights. These comments often contain irrelevant information, necessitating the manual identification of relevant ones. This paper introduces CommentThreadFilter, a system designed to classify comments as either Relevant or Irrelevant for the main thread post. Leveraging base models like BERT, RoBERTa, and SciBERT, we evaluate the system’s performance on the newly created CFComments dataset. The dataset is the first of its kind, comprising 19 labelled comment threads in the competitive programming domain, manually annotated by two experts, alongside 1131 unlabeled comment threads. The proposed models, combined with a weak augmentation on the text, achieve an F1-score of 86%, outperforming the 77% F1-score obtained using gpt-3.5-turbo. By effectively filtering comments as Relevant or Irrelevant, our system enhances the user’s ability to gain valuable insights and better comprehend the underlying problem.