We propose a computationally efficient method for determining anomalies in hyperspectral data. In the first stage of the algorithm, the background classes, which are the dominant classes in the image, are found. The method consists of robust clustering of a randomly chosen small percentage of the image pixels. The clusters are the representatives of the background classes. By using a subset of the pixels instead of the whole image, the computation is sped up, and the probability of including outliers in the background model is reduced. Anomalous pixels are the pixels with spectra that have large relative distances from the cluster centers. Several clustering techniques are investigated, and experimental results using realistic hyperspectral data are presented. A self-organizing map clustered using the local minima of the U-matrix (unified distance matrix) is identified as the most reliable method for background class extraction. The proposed algorithm for anomaly detection is evaluated using realistic hyperspectral data, is compared with a state-of-the-art anomaly detection algorithm, and is shown to perform significantly better.