This paper investigates the performance of an adaptive filter, (Frequency-Response-Shaped Least Mean Square (FRS-LMS) algorithm) for canceling impulsive components when the nominal process (or background noise) is a correlated, possibly nonstationary, Gaussian process. The performance of the algorithm in estimating a BPSK signal corrupted by a white and correlated impulsive noise is investigated. The algorithm does not require a priori knowledge about the noise parameters, but requires knowledge of the signal frequency which can easily be estimated from its periodogram. The performance of the FRS-LMS is compared to that of the conventional LMS, the leaky-LMS (L-LMS), and the modified leaky LMS (ML-LMS) algorithms in terms of mean square error (MSE), convergence speed and bit-error-rate (BER). The results indicate that the FRS-LMS algorithm performs approximately twice as better than the LMS and L-LMS algorithms in white impulsive noise environments, while the ML-LMS algorithm fails to converge. Also, it provides superior MSE and BER performance in correlated impulsive noise environments, while the other algorithms fail to converge. The performance gain is due to the frequency shaping and the outlier reduction properties of the algorithm.