Improving the User-Friendliness of AAT through a Staged Evaluation

With the rapid adoption of online learning technologies, there is a pedagogical imperative to evaluate the success of online courses and analyze how student interact with electronic communication and learning resources. There is substantial data in learning management systems (LMSs) to inform this research, and many software tools aimed at mining LMS data or providing statistics or visualizations about those data. However, these tools provide either overviews or are very specific to the information they show. In this paper, we intro-duce the latest revision of the Academic Analytics Tool (AAT), which empowers users (e.g., teachers, learning designers) to do their own investigations into complex educational data stored in LMS databases. AAT allows non-technical users to answer almost limitless questions about learners' behavior and the impact of teaching methods and learning designs. To improve the user-friendliness of AAT, which is one of the core goals of the tool, a staged, qualitative evaluation study has been conducted. By improving AAT's user-friendliness, the tool gets more effective in supporting users and enabling them to use the huge amounts of educational data to learn how students interact with courses and learning materials as well as about the effectiveness of teaching methods and learning designs.