Anomaly detection of self-similar network traffic data is a difficult problem in network management. Due to network traffic may have the property of long term dependent, it can be approximated with a finite order Euler-Cauchy system based on ARMA (autoregressive moving average). As a result, AR (auto-regressive) model based on time serial analysis theory was used to deal with the problem of self-similar network traffic. On this basis, a wavelet generalized likelihood ratio (WGLR) algorithm was developed for anomaly diagnosis of network traffic with self-similar characteristics. WGLR algorithm combines generalized likelihood ratio (GLR) algorithm and wavelet transform method, and captures the failure point in real time. That will improve the accuracy of anomaly detection. Simulating and experiment results accord with the conclusions suggested in the paper.