A Clustering Approach for Domestic Smart Metering Data Preprocessing

The widespread application of smart metering will lead to a huge increase of amounts of data into electricity utilities. As the Automatic Metering Infrastructure (AMI) is being deployed in countries such as Colombia, problems with data acquisition have arisen, including data loss, and data transmission and storage errors. In these cases, descriptive statistics are not suitable if aggregation processes are conducted. In this paper, we propose a segmentation method based on the K-means algorithm to effectively identify and remove anomalous information in domestic smart metering data. This method aims to characterize the pattern of electricity use for each consumer stratum and location, and to preprocess the data for its subsequent analysis. For this purpose, an open-source library based on the software environment R was developed to standardize smart metering data, to conduct a statistical analysis, and to correlate consumption patterns with geographical information. A study case is discussed with annual data acquired from 6060 smart meters located in city of Bogotá, Colombia, where outliers are identified over a pre-filtered data set encompassing average real power measurements along a four-month period. We show that our method is able to characterize patterns of consumption among the predefined user strata and locations in Bogotá to successfully identify and remove anomalous data for a subsequent statistical and geographical analysis.