Adam Adaptive Optimization Method for Neural Network Models Regression in Image Recognition Tasks

We consider the problem of applying adaptive optimization methods in neural network regression tasks like estimating image quality or aligning objects in the frame. Such tasks are usual for preprocessing in image analysis. Two sample tasks are presented. The first is evaluating the degree of blurring in the iris recognition system. Eye images are taken from BATH and CASIA databases, and samples are generated by Gaussian blurring. The second sample task is the alignment of the face in an image. Training samples are obtained by rotating face images, and the rotation angle is estimated. Both tasks are solved by direct estimation of parameters with a neural network. The resulting accuracy of parameter estimation is acceptable for practical use. The Adam algorithm and its modifications, such as AdamW and Radam, are applied. The modification of the Radam algorithm, changing weight decay strategy, is proposed. This modification reduces the error 1.5 times in comparison with the model trained by the original algorithm.