Privacy-Preserving On-Screen Activity Tracking and Classification in E-Learning Using Federated Learning

E-learning, a modern method of education that utilizes electronic technologies such as computers, mobile devices, and the internet, has experienced a significant surge in adoption and usage in recent years. While it has the potential to reach every corner of the world, it also creates an opportunity for time and resource wastage. In almost all cases students use the same device for studying and for entertainment purposes. Being one click away from ever-addicting social media, it is very difficult for students to stay focused on studying using digital devices and not waste time on it. The issue is quite significant as online education will be practised more and more in the future. In spite of that, detecting the on-screen activity of students is an underexplored region of research, and to our best knowledge, no research takes protecting their privacy into consideration. Therefore in this research, a privacy-preserving architecture is proposed to detect whether students are utilizing their time on their computer or wasting it while the user’s privacy is protected with federated learning. A dataset containing over 4000 screenshots of different activities of students is used to classify them into categories using several pre-trained models where our proposed FedInceptionV3 achieves a state-of-the-art test accuracy of 99.75%.