Supporting UX Evaluation with Open Text Word Clustering

This research work proposes using word clustering to identify concepts associated with the user experience (UX) evaluation of a version of the standardized User Experience Questionnaire (UEQ), where the usual written of responses is replaced by input through a voice interface. The clusters were made from a Word2vec model of word embeddings that was built based on sentences contributed by participants who evaluated the implementation of the voice interface, using traditional quantitative questionnaires and which they complemented with open text comments. The results show clusters around keywords such as ’assistant’, ’understand’, ’response’ and ’survey’, which allow the identification of words associated with both the voice implementation and the UEQ questionnaire itself, and provide information about attitudes researchers could investigate in more detail about the assistant implemented, for example in a subsequent evaluation to be carried out through a focus group or semi-structured interviews.