Discovering electricity consumption over time for residential consumers through cluster analysis

This paper presents an innovative and scalable methodology named CONDUCTS (CONsumption DUration Curve Time Series) to discover residential electricity consumption behaviours over time. CONDUCTS exploits data stream processing in time windows jointly with unsupervised machine learning on time-independent data. Specifically, time series consumption data for every consumer is split into N time windows. For a particular time window, a duration curve is calculated providing significant shape-based information disregarding temporal aspects. Each duration curve is sampled according to statistical characteristics and its relevant shape is captured. Therefore, every individual is represented by the evolution of N simplified duration curves. A cluster analysis, based on the K-means algorithm and the Euclidean distance, provides the different consumer profiles in a given time window. CONDUCT's current implementation runs on Apache Spark, a state-of-the-art distributed computing framework. As a case study, CONDUCTS has been experimentally assessed on the real hourly metered data collected in the time frame of one year for a large number of Spanish residential consumers. The experiments highlighted CONDUCT's ability to identify time-variable well-cohesive and well-separated groups of individual electricity consumption patterns with similar characteristics.