A Consideration on Efficient Detection Method of Anormal Responses in High-dimensional Questionnaire Data

We have been studying to detect anormal responses in high-dimensional questionnaire data, that may affect the overall analysis results and are to be removed in the preprocess, more efficiently. In this paper, we apply principal component analysis (PCA) and multiple correspondence analysis (MCA) as dimension reduction methods, and x-means and gaussian mixture model (GMM) as clustering algorithms to the high-dimensional questionnaire data. Then, we examine the combinations of these methods for detecting anormal responses that are significantly far from the cluster centers or the distribution centers. Also, we employ principal component pursuit (PCP), where the absolute value sum for each response in the sparse matrix is used as anormal score to directly detect anormal responses. As a result, we find both of MCA+x-means and PCP achieve to detect reasonable anormal responses with shorter execution time.