USR: Enabling identity awareness and usable app access control during hand-free mobile interactions

We propose a new approach for improving user experience and privacy protection during human-mobile speech interaction by considering factors, such as user identity, application privacy level, usable app access control, and application function class. To integrate these factors into speech recognition on the mobile device, we design and implement a unified speech-speaker recognizer (USR) framework, which will recognize both speech content and speaker identification, and respond accordingly. This USR framework consists of an application interface, a speaker recognition module, a speech recognition module, and an identity management module. A comparison study was conducted contrasting the benefits and limitations of USR framework to the established original speech recognition and app access control framework on mobile device, such as Google Voice, and AppLock. Our results show that, while USR framework intuitively improved mobile privacy by serving only the phone owner for specific customized applications, USR framework was also able to provide better user experience across a number of tasks.