On the Unsurprising Behaviour of Kernels in High Dimensions

Kernels are employed in ML algorithms not only as a means of measuring similarity but also for their interesting theoretical interpretations and computational advantages. Among them Gaussian kernels have been extensively employed for their ease of interpretation and locality. However, it has been observed [4] that they lose many of their advantages in high dimensions. Hence a suitable modification of it has been proposed to overcome this loss. In this work, we firstly show that despite these modifications not all the lost advantages have been recovered and leads us to consider alternate kernels. However, we contend that either a change or a modification of an underlying kernel only treats the symptoms and discuss what could be the main malaise.