Less Is More: Deep Learning Using Subjective Annotations For Sentiment Analysis From Social Media

Acquiring reliable training annotations for the huge amounts of data collected in large-scale applications is often infeasible, especially for inherently subjective tasks, such as sentiment analysis. In these cases, the data are usually annotated using semi-automated methods. Even when crowd-sourcing is used, ensuring the quality of the acquired annotations can be challenging. Therefore, a number of important questions arise when annotating such datasets: Does using more data always increase the accuracy of a model regardless the quality of the annotations? Is there any way of selecting which data samples we should use when the annotations are unreliable? Is there any point at which using unreliable annotations actually harms the performance of deep models instead of helping? In this work we provide an extensive study on training deep sentiment analysis models with unreliably annotated data, as well as propose a simple, yet effective semi-supervised learning method to overcome the aforementioned limitations.