A solution to the segmentation problem based on dynamic programming

Describes an algorithm that uses dynamic programming to identify the location of changes in a linear model of a time series. The technique is currently being used to analyse sales data in order to improve the efficiency of the supply chain for retailers, but it is equally applicable to other applications of the segmentation problem, such as fault detection and tracking. The algorithm recursively creates an array of costs associated with fitting least squares models of order p to all possible segments of the time series. It is shown that this is equivalent to running n-p Kalman filters, where in order to reduce the sensitivity to numerical errors, the filters are run backwards through the data. Within this recursion a second recursion is used to determine whether the data in each of the segments could be adequately described by a lower order model. Having created the array of costs, dynamic programming is used to identify the positions of the jumps between the segments that result in the lowest least squares cost over the whole of the n points in the data set. The optimum number of jumps is taken to be that which provides the most parsimonious representation of the whole data set, based upon the MDL cost.<>