This paper highlights the impact of channel correlations on the capacity and performance of MIMO communications, with a focus on the so-called diagonal correlations. Based on this analysis, the limitations of simplified mathematical representations, such as the Kronecker or the diagonal-decorrelation models, are pointed out. Finally, the correlation properties of popular geometry-based statistical models are studied in order to analyze whether the correlation structure of these models can be adequately represented by simplified mathematical models, as well as to quantify the errors introduced by these simplifications. With respect to channel correlations, neither the Kronecker nor the diagonal-decorrelation assumptions are good representations of the correlation structure of the investigated geometry-based statistical models. When comparing capacity and symbol error probability results, it is found that the diagonal-decorrelation model may yield significant errors on both considered metrics. The Kronecker model generally yields errors less than one order of magnitude on the symbol error rate, but relative errors on ergodic or outage capacity may be more significant.