A novel objective function based on a generalized kelly criterion for deep learning

Neural networks have demonstrated promising results for a wide range of applications. The proposed techniques employ different architectures and objective functions to adapt to the application while enabling a feasible implementation. Commonly used objective functions for network optimization are based on the cross entropy between the empirical distribution of the training data and the model distribution. Another approach uses the Dice coefficient to mitigate class imbalances without a need to sample reweighting. However, use of the Dice coefficient hinders backpropagation from the output of the softmax to the previous network layers. In this paper, we propose a novel objective function based on a generalized Kelly criterion for optimizing feed-forward neural networks. In each forward pass of a mini-batch through the network, this criterion determines the optimal bets (allocation fractions) of each sample for betting on multiple horses (classes) whose win probabilities are known. Then the network parameters get optimized by a minimization of the average unspent wealth (reserve rate) of the samples via backpropagation from the output of the softmax layer to the previous network layers. On a fully convolutional residual neural network, called V-net, the proposed function is evaluated against a weighted cross entropy-based and a Dice-based objective function for automatic volumetric medical image segmentation. These evaluations are done after a dedicated hyperparameter optimization for each objective function. They reveal the superiority of the proposed method with regard to the classification accuracy and the convergence speed of the training.