Order Effects in Multidimensional Rating Systems

Online rating systems are an important source of information in online consumer decision-making. However, the effects of the design of the rating systems on raters’ evaluations are only partially understood. In the context of multidimensional rating systems—i.e., systems that ask users to provide ratings on individual dimensions of a product or service, often in addition to an overall rating—we examine the mutual effects of dimensional ratings on each other. Drawing on the accessibility-diagnosticity framework, we seek to determine in how far ratings on earlier dimensions influence subsequent ones. In an online experiment of movie ratings, we found that people gave a 14.83% higher score to a negative aspect when this rating was preceded by more positively-valenced dimensions. These results suggest that rating-system providers should adjust rating presentation to account for these biases.