Dataset Augmentation Strategies for Visual Activity Recognition in Deep Neural Networks

Dataset augmentation involves synthetically increasing the size of a dataset by introducing appearance variations that help a machine learning model to generalize when trained over the dataset. Several augmentation strategies have been proposed and evaluated in the context of image recognition problems [7], however empirical studies on effectiveness of data augmentation in video recognition are few and far between. In this paper, we explore a number of dataset augmentation strategies and quanti-tatively assess their value on the task of visual activity recognition in a highly controlled manner. We extend a number of image augmentation strategies to the video domain, and also propose an entirely novel augmentation approach drawing inspiration from the idea of “Dropout” [20] in neural networks directly to the data domain and calling it “Blackout” augmentation. We thoroughly evaluate these augmentation strategies on the recently proposed dataset from Kumar et al. [15] which includes a complex manufacturing assembly activity controlling for variations in camera pose and human actors. Our detailed evaluation reveals interesting trends hitherto undiscussed in literature, and a major performance boost of 20.0 percentage points.