General bias/variance decomposition with target independent variance of error functions derived from the exponential family of distributions

An important theoretical tool in machine learning is the bias/variance decomposition of the generalization error. It was introduced for the mean square error. The bias/variance decomposition includes the concept of the average predictor. The bias is the error of the average predictor, and the systematic part of the generalization error, while the variability around the average predictor is the variance. We present a large group of error functions with the same desirable properties as the bias/variance decomposition. The error functions are derived from the exponential family of distributions via the statistical deviance measure. We prove that this family of error functions contains all error functions decomposable in that manner. We state the connection between the bias/variance decomposition and the ambiguity decomposition and present a useful approximation of ambiguity that is quadratic in the ensemble coefficients.