Emotion Mining and Sentiment Analysis in Software Engineering Domain

In last few years, researchers' interest in sentiment analysis in the software engineering domain has risen significantly. Most of the studies show the use of standard sentiment analysis tools such as NLTK API and Sentistrength to find sentiments in issue reports extracted from some version maintenance software. Since these tools are not trained on software engineering texts and comments, their results may not be as accurate as one might hope. In this study we find to what extent do our human evaluators who are familiar with the software engineering terms agree with sentiment analysis tools on presence or absence of emotions. We perform this study on our own supervised dataset (Apache JIRA). In addition to this, we evaluate the performance of the metrics when a new tool is introduced and concluded that disagreement between tools leads to disagreement among results.