Empirical analysis of generalization through augmentation for classifying images of vernacular handwritten texts

Handwritten image recognition [1] is a challenging problem that has seen tremendous research efforts in recent years from the computer vision community. Although many deep learning-based classifications algorithms are studied in the literature for handwritten image recognition [2], most of the works are concentrated on a few widely used benchmark datasets [3]. Hence, it is difficult to understand how well a certain model generalizes across different optical characters with vast complexity levels. The existing research works for classification from handwritten image datasets are primarily focused on developing better model architectures, better training procedures, better regularization methods such as Image Augmentation, Dropout, Dropblock, etc. [4–6], hard example mining techniques [7], better Loss Functions such as Focal loss [8], Weighted Cross Entropy loss, etc. In this paper, we focus on image augmentation techniques along with an example mining technique to establish a robust classification baseline for handwritten datasets using a novel combination of augmentation methods and online hard example mining. Through our experiments, we show the combination of advanced image augmentation techniques best suited for achieving the highest macro averaged recall score and highlight a few important trends therein.