ECHOES: Input Sensing and Reconstruction Model Utilizing CNN and LSTM with Mobile Sensor Data

The widespread integration of sensors such as accelerometers and gyroscopes in smartphones has significantly enhanced user experience, but also introduced new vulnerabilities exploitable by side-channel attacks. These attacks can infer user input by analyzing motion patterns generated during touch interactions. To mitigate this threat and improve input recognition accuracy, we propose ECHOES, a side-channel analysis framework designed to recover user input on touchscreen keyboards. ECHOES comprises two key modules: (i) a sensing module that collects inertial sensor data and virtual memory change patterns, employing convolutional neural networks to detect the user's motion state and classify touch interactions into predefined keyboard regions; and (ii) a language inference module that leverages the click region sequence and natural language processing to reconstruct the user's intended input. To further improve localization under spatial ambiguity, the keyboard is partitioned into five regions to constrain candidate outputs. Experimental results demonstrate that ECHOES achieves 94.01% accuracy in motion state recognition and 91.63% accuracy in input restoration, underscoring its potential for exposing security risks in mobile input systems.