Bidirectional Transfer Learning Between Activity and User Identity Recognition Tasks via 2D CNN-LSTM Model for Wearables

We implement transfer learning (TL) to attain efficiency in training and feature extraction by freezing the connection weights of shallow layers of deep learning (DL) models, already trained for a given task, to be used in another. This eliminates the need to retrain the whole model from scratch for the new task, resulting in significantly reduced training times, while maintaining the classification accuracy at acceptable levels. Specifically, we investigate bidirectional feature transfer between the human activity recognition (HAR) and user identity recognition (UIR) tasks through the use of wearable motion sensors. Both tasks have recently drawn considerable attention and are significant because of their relevance in a broad range of applications. Knowledge transfer between tasks via TL brings many advantages. We employ the features extracted by our recently proposed hybrid DL model which has proven superior to commonly used state-of-the-art (SoTA) DL models. The model can extract features more effectively because of its parallel-running 2D convolutional neural network (CNN) and long short-term memory (LSTM) branches that receive complementary input types, unlike many hybrid models that implement a series connection that receives a single type of input. To our knowledge, TL between the above-mentioned tasks has not been implemented to date. We demonstrate high-accuracy bidirectional feature transfer between the HAR and UIR domains that shortens the training time by a factor that varies between 14.3–34.4, without much degradation in the classification accuracy. To provide a typical figure, the accuracy degrades by 2.45% for a group of three dynamic activities. The key technical challenge of this work is maintaining the classification accuracy at acceptable levels while benefiting from the advantages that TL offers. We expect the results of this work to have impact by way of reducing the demanding computational requirements in training resource-limited wearable edge devices.