Realizing Emotional Interactions to Learn User Experience and Guide Energy Optimization for Mobile Architectures

In the age of AI, mobile architectures such as smartphones are still “cold machines”; machines do not feel. If the architecture is able to feel users’ feelings and runtime user experience (UX), it will accordingly adapt performance/energy to find the optimal system-operating state that consumes the least energy to satisfy users. In this paper, we will utilize users’ facial expressions (FEs) to learn their runtime UX. We know that FEs are the natural and direct way for humans to convey their emotions and feelings. Our study reveals that FEs also reflect UX. Our research for the first time quantifies the link between FEs and UX. Leveraging this link, the architecture will be able to use the front camera to see FEs and feel users’ UX. Based on UX, the architecture can appropriately provision computing resources. We propose Vi-energy system to realize the above idea. Our evaluation shows that Vi-energy reduces energy consumption by 52.9% at maximum and secures UX.