The Perils of Reliability Benchmarking

It is becoming increasingly common for utility commissions to compare the reliability of utilities in their state to industry benchmark surveys. Utilities are typically categorized into performance quartiles, with the top quartile corresponding to 25% of participating utilities with the best reliability indices. Taken at face value, this type of benchmark comparison is of little to no worth. Of more concern is the tendency of people to draw incorrect conclusions, to unfairly criticize certain utilities, to unfairly praise others, and to set inappropriate reliability improvement targets. Reliability benchmarking activities can be extremely valuable when used appropriately, but often it is not. This article discusses the many problems with benchmark comparisons, recommends their proper use, and suggests an approach for setting appropriate reliability improvement targets.