A New Data Augmentation Method for Time Series Wearable Sensor Data Using a Learning Mode Switching-Based DCGAN

This letter describes a new image augmentation method based on a DCGAN considering mode switching in terms of transient learning accuracy threshold on the problem of human gesture recognition through imaging of wearable sensor time-series data and a deep CNN structure. Because the discriminator in GANs learns faster than the generator, it is known that mode collapse occurs, in which only image modes biased to a specific image type are augmented among various image forms. In this study, to solve the mode collapse caused by the learning difficulty mismatch between networks, we add a learning mode switching layer between the generator and discriminator and receive feedback from both networks’ transient learning accuracies to switch the learning mode if predefined thresholds are exceeded. We confirm that the proposed approach balanced the learning rate between the generator and discriminator networks, resolved the mode collapse problem, and increased the test accuracy of a deep CNN trained with an augmented image set by approximately 20.35% compared to a conventional DCGAN. In addition, it showed better accuracy on a performance comparison with other improved DCGAN methods.