MMAuth: A Continuous Authentication Framework on Smartphones Using Multiple Modalities

With the wide use of smartphones, more private data are collected and saved in the smartphones. This raises higher requirements for secure and effective user authentication scheme. Continuous authentication leverages behavioral biometrics as identity information and shows promising characteristics for user verification in a continuous and passive means. However, most studies require users to operate the smartphones in a specific mobile application or perform user-defined touch operations. This paper studies the continuous authentication on smartphones in the wild, where it is hard to characterize touching behavior accurately due to the complexity of usage context and cross-use of various types of touch gestures. Towards this end, in this paper, we propose a continuous authentication framework using multiple modalities, named as MMAuth, which integrates the heterogeneous information of user identity from multiple modalities (e.g., motion movement pattern, touch dynamics, usage context). A time-extended behavioral feature set (TEB) and a deep learning based one-class classifier (DeSVDD) are developed for performing more accurate authentication. Evaluations are conducted using a novel unconstrained smartphone usage dataset collected from 100 volunteers in real world as well as a public laboratory dataset. Extensive experimental results demonstrate that the state-of-the-art authentication performance of MMAuth in both unconstrained and laboratory environment, and the effectiveness of its two proposed modules (the TEB feature set and the DeSVDD classifier). Additional experiments on system robustness, in terms of usability to different touch gestures, sensitivity to various mobile applications, and scalability to user space, are also provided to examine the applicability of MMAuth.