History-Based Article Quality Assessment on Wikipedia

Wikipedia is widely considered as the biggest encyclopedia on Internet. Quality assessment of articles on Wikipedia has been studied for years. Conventional methods addressed this task by feature engineering and statistical machine learning algorithms. However, manually defined features are difficult to represent the long edit history of an article. Recently, researchers proposed an end-to-end neural model which used a Recurrent Neural Network(RNN) to learn the representation automatically. Although RNN showed its power in modeling edit history, the end-to-end method is time and resource consuming. In this paper, we propose a new history-based method to represent an article. We also take advantage of an RNN to handle the long edit history, but we do not abandon feature engineering. We still represent each revision of an article by manually defined features. This combination of deep neural model and feature engineering enables our model to be both simple and effective. Experiments demonstrate our model has better or comparable performance than previous works, and has the potential to work as a real-time service. Plus, we extend our model to do quality prediction.

I. Introduction

In peer production communities, quality assessment is indispensable. Take Wikipedia for an example, assessing article quality is a traditional and important task, which, in practice, can assistant readers to distinguish high-quality content from massive information and, at the same time, instruct editors to improve poor-quality content. On Wikipedia, there are two methods for users to acquire article quality information. One is the ‘Featured articles’¹

https://en.wikipedia.org/wiki/Wikipedia:Featured_articles

, which are officially regarded as the best articles. However, on English Wikipedia, only 0.1% articles are featured. The other way is ‘WikiProject assessment’²

https://en.wikipedia.org/wiki/Category:Wikipedia_1.0_assessments

in which WikiProject members rate articles with 7-class quality labels. However, this assessment is voluntary. When and how often to assess articles totally depend on users, and thus the tagged quality label is probably out-of-date and unreliable. Therefore, an automatic article quality assessment is necessary.