Optimality of Least-squares for Classification in Gaussian-Mixture Models

We consider the problem of learning the coefficients of a linear classifier through Empirical Risk Minimization with a convex loss function in the high-dimensional setting. In particular, we introduce an approach to characterize the best achievable classification risk among convex losses, when data points follow a standard Gaussian-mixture model. Importantly, we prove that the square loss function achieves the minimum classification risk for this data model. Our numerical illustrations verify the theoretical results and show that they are accurate even for relatively small problem dimensions.