Monitoring conceptual development with text mining technologies: CONSPECT

This paper evaluates CONSPECT, a service that analyses states in a learner's conceptual development. It combines two technologies - Latent Semantic Analysis to analyse text and Network Analysis (NA) to provide visualisations - into a technique called Meaningful Interaction Analysis (MIA). CONSPECT was designed to help both online learners and their tutors monitor their conceptual development. This paper reports on the validation experiments undertaken to determine how well LSA matches first year medical students in clustering concepts and in annotating text. The validation used several techniques, including card sorting and Likert scales. CONSPECT produces almost `peer' quality results and what remains to be tested is whether it improves with more advanced learners. One of the experiments showed an average 0.7 correlation between humans and CONSPECT.