Data preprocessing for prediction of rerecirculating water chemistry faults

The water quality data in some petrochemical company are stored in the lims database, in order to support the field operation's decision-making according to these data, it's necessary to do some appropriate data mining. However, the accuracy of the results of data mining directly associated with the quality of source data, so data preprocessing on the raw data is necessary in the data mining process. The process of data preprocessing is as follows: First, the classification of the raw data, here it is based on the frequency difference of the data acquisition. Second, data cleaning on the raw data, including cleaning the noisy data, missing data and redundant data (here mainly refers to the attribute redundancy).For the noisy data it take the method combining computer with artificial, for the missing data it mainly take the interpolation method using the mean data, and the redundancy property items were deleted; Third, data transformation for the later data processing. It mainly do the normalization to eliminate the difference of the dimension and magnitude on the raw data, so that all data can put together to make a comprehensive analysis. It uses the mean and standard deviation method to preprocessing the raw data and then make a compare of the result of the two different methods and it get the conclusion that the mean method is a better normalization method; fourth, to conduct data reduction, which refers to reduce the data storage space as far as possible while it must ensure the data integrity, it uses the principal component analysis method to do the job.