A Novel Target Value Standardization Method Based on Cumulative Distribution Functions for Training Artificial Neural Networks

Function approximation by artificial neural networks (ANNs) are often carried out via a collocation grid approach. However, for certain combinations of grids and functions, the training data for the relevant ANN can be extremely skewed and hence affect training efficiency, thereby necessitating standardization or normalization techniques. The choice of collocation grids often allows uniform training features but not targets. In this paper, we contribute a comparison between methods of target standardization - the relatively common min-max, z score, normalized log methods and the uncommon empirical cumulative distribution function (CDF) method - in terms of the resulting approximation capabilities of the ANNs using the standardized target values. We demonstrate that the empirical CDF is a general and robust standardization method that allows for efficient training and good approximations insituations of extreme target skewness. Novel modifications using a mix of true CDF and empirical CDF in standardizing targets are used to successfully reduce biases arising from the empirical CDF standardization method.