Dynamic Programming Method to Optimally Select Power Distribution System Reliability Upgrades

This paper presents a novel dynamic programming (DP) technique for the determination of optimal investment decisions to improve power distribution system reliability metrics. This model is designed to select the optimal small-scale investments to protect an electrical distribution system from disruptions. The objective is to minimize distribution system reliability metrics: System Average Interruption Duration Index (SAIDI) and System Average Interruption Frequency Index (SAIFI). The primary input to this optimization model is years of recent utility historical outage data. The DP optimization technique is compared and validated against an equivalent mixed integer linear program (MILP). Through testing on synthetic and real datasets, both approaches are verified to yield equally optimal solutions. Efficiency profiles of each approach indicate that the DP algorithm is more efficient when considering wide budget ranges or a larger outage history, while the MILP model more efficiently handles larger distribution systems. The model is tested with utility data from a distribution system operator in the U.S. Results demonstrate a significant improvement in SAIDI and SAIFI metrics with the optimal small-scale investments.


I. INTRODUCTION
A. MOTIVATION P OWER distribution system operators (DSOs) are continuously trying to improve their system reliability. However, improving their reliability is becoming increasingly difficult due to aging infrastructure, deferred investments pushing grid infrastructure further to its limits, and increased renewables causing more system variability [1]- [3]. Utilities are often incentivized to become more reliable through performance-based regulation (PBR). There are multiple types of PBR incentives that increase utilities' revenue if they improve or maintain a certain reliability threshold [4]. Typical reliability metrics are System Average Interruption Duration Index (SAIDI) and System Average Interruption Frequency Index (SAIFI). SAIDI is how long the average customer is without power during a given time horizon (often reported quarterly or yearly) while SAIFI reflects how frequently the average customer loses power during a given time horizon. Reliability events are defined in this paper as high-probability, low-consequence events. Examples include animals, lightning, trees, and wind damaging grid components, leading to small-scale outages.
Utilities often have specific budgets for small-scale investments to improve their reliability to common outage causes such as animals, trees, lightning, and car accidents. Some examples of these small-scale investments include animal preventive measures, added barriers, additional reclosers and fuses. To decide which upgrades to purchase, utilities regularly go down the list of their worst outages over the past year and apply upgrades to prevent these outages from recurring. Although this routine does improve system reliability, there are likely more effective investment strategies. This paper presents a dynamic programming (DP) model to generate optimal small-scale investments.

B. LITERATURE REVIEW
There is significant research in improving system reliability through reliability-constrained expansion planning [5]- [9]. Most of this work focuses on the transmission system and large-scale investments (e.g. new transmission lines or generation), and this paper fills a research gap on small-scale investments for distribution systems. References [10], [11] improve power system reliability by focusing on specific methods such as increasing spinning reserve, renewable energy controls, capacitor bank placements, and energy storage. Many facets of power system reliability have been explored through [12]- [17]; these important works use particle swarm optimization and fuzzy decision analysis. These methods permit fine system detail and produce good results; however, solutions are generally not provably optimal. In contrast, approaches that use DP and MILP generally prohibit such fine system detail but generate provably optimal results. DP has been applied to several important power distribution system applications. In [18] and [19], DP is used to determine the optimal feeder routing for distribution planning, considering both cost and reliability. A genetic DP approximation algorithm is applied in [20] for power distribution network expansion scheduling with the goal of minimizing costs associated with losses, maintenance, new installations, and reconductoring. A DP algorithm is applied in [21] to improve power distribution system restoration after a blackout. The DP algorithm presented in this paper expands this line of research by focusing on small-scale distribution system investments to improve reliability metrics utilizing historical outage data.
In addition, various power distribution system planning problems have been solved with mixed-integer linear programming (MILP). Reference [22] provides an overview of existing distribution system problems solved with MILP. [23] formulates a MILP which optimizes reliability metrics through sectionalized switching and expansion planning. A MILP for optimal distribution system routing and siting/sizing of distribution substations is presented in [24], [25]. In [26] a MILP is formulated to select power distribution system small-scale investments that improve reliability. The DP algorithm proposed in this paper efficiently solves that MILP model to improve scalability.

C. CONTRIBUTIONS TO GRID RELIABILITY RESEARCH
This paper presents a novel DP algorithm that determines globally optimal small-scale investment strategies to improve power distribution reliability metrics. The optimization model uses historical outage data, along with investment cost and impact data, to determine the optimal investments for improving SAIDI and SAIFI metrics.
The DP technique in this paper leverages the fact that the MILP investment model presented in [26] has a nonlinear objective and only a knapsack constraint. The classic 0-1 knapsack DP algorithm [27], [28] is generalized so that it can exactly solve the investment model.
Computational experiments compare scalability of the investment model through DP versus MILP branch-andbound. These experiments explore scaling the data set through the number of outages, size of distribution system, and budget. This paper uses real utility data to demonstrate results. The tradeoff between budget and reliability improvement is demonstrated through solving the model for a range of budgets. Furthermore, the tradeoff between emphasizing SAIDI verses SAIFI in the investment decisions is demonstrated for a selection of budgets. Finally, convergence of reliability objective with respect to increasing number of outages is investigated to demonstrate that only a minimal number of outages are necessary to obtain optimal reliability investment decisions.
The rest of this paper is as follows. Section II describes the investment model and model inputs. Section III defines the generalized dynamic programming algorithm. Section IV presents computation experiments. Section V presents results on a utility data set. Finally, Sections VI, VII, and VIII give conclusions and future work, acknowledgements, and references respectively.

A. NONLINEAR KNAPSACK PROBLEM
The following stochastic nonlinear mixed integer program is solved using both traditional MILP techniques as well as a generalized reliability dynamic programming algorithm (GRDP) which minimizes the normalized sum of SAIDI and SAIFI through investment purchases under the restriction of a budget.
The optimization problem is effectively formulated as a 0-1 knapsack problem with a nonlinear objective function (1) subject to a budget constraint (2). The objective function is defined by SAIDI and SAIFI, standard metrics for the reliability of a power system. These metrics are defined in equations (3)- (6).
Equation (1) minimizes the sum of normalized SAIDI and SAIFI. Equation (2) is the budget constraint, constraining the total cost of all purchased investments. Equations (3) and (4) calculate SAIDI and SAIFI based on number of customers affected and duration of outages after all investments have been implemented.
The duration and/or the number of customers affected in (5) and (6) change if an upgrade is applied that affects outage o. If the optimization selects multiple upgrades that affect a single outage, the minimum in (5) and (6) guarantees that the model uses the lowest P u o,u and T u o,u respectively for SAIDI and SAIFI calculation. These equations account for how multiple upgrades affect a single outage. Note that the applicable upgrades for outage duration improvement may not be the same as the applicable upgrades for improving number of customers affected. This distinction is made through the sets U o and V o . Constraints (3), (5), and (6) are nonlinear. However they are easily linearized through classic techniques as described in [26].
Without loss of generality, the MILP model can further be developed as a deterministic model [26]. For computational experiments shown in section IV, the model is coded in the Python-based mathematical programming language Pyomo [29], [30] and solved using the CPLEX solver.

B. DATA CONSIDERATIONS AND SCENARIO GENERATION
The methodology presented in this paper requires good statistical models for outages. That is, years of outage data is required to create these models. These models are then employed to generate synthetic future outage scenarios where each scenario represents a possible year of outages. The synthetic scenarios are based on the probability density functions of the actual historical outage data [26]. For example, in Fig. 1, each scenario, developed from sampling the probability density functions (PDFs) of the historical outage data will have similar characteristics as the historical data (e.g., similar SAIDI and SAIFI). An electric distribution utility in the U.S. supplied the authors with over five years of historical outage data that included over 60,000 individual outages. The results in this paper use this data, and because this data is from a real source, the data cannot be released. Note that the results in Section IV and V are based on this U.S. utility dataset.
The results indicate a significant improvement can be made in the utility's SAIDI and SAIFI metrics when using the optimal investments chosen by the DP model presented in this paper. Although the specific input data cannot be published, an example of the input data is seen in Table 1. See Section V for full results. TABLE 1. Historical outage data provided by a distribution system operator in the U.S. and used as the primary input to the reliability investment optimization model. Showing only 3 rows, rather than the full data set of more than 60,000 outages.
The primary input to the optimization models is historical outage data from a DSO. Ideally the distribution system operators have every outage recorded in detail for multiple years. Each outage should include the duration, number of customers affected, device type that failed, cause of the failure, and specific location, e.g., feeder ID and device ID, or GPS location.
The historical data is characterized into conditional PDFs. The PDFs are sampled in a Monte Carlo process to create synthetic outage data that is based on the historical outage data. This allows for a large sample size for the optimization model to give valid results, and to compare the two optimization techniques' efficiency when changing parameters such as how large the system is and how many outages are in the data. In addition, during the Monte Carlo sampling, special emphasis can be added to fault location, and device age [26]. The synthetic data may be placed into yearly data sets, each yearly set representing a scenario of the next future year of outages for the power system under question. For the models presented in this paper, specific scenarios are not needed, but a large enough set of outage data is necessary for accurate results; this is further discussed in the results section of the paper, Section V.
Furthermore, the DSO needs to provide investment options, approximate cost for those options, and how those upgrades will impact certain types of outages based on the outages' device type and cause. Typical investments considered in this paper include: replacing fuses with reclosers, adding additional fuses in the system to break the system into smaller sections, adding animal preventive measures for squirrels or birds, and other small, relatively inexpensive upgrade options to improve reliability.

A. TRADITIONAL DYNAMIC PROGRAMMING ALGORITHM
The 0-1 knapsack problem can be solved with either a branch-and-bound or dynamic programming (DP) algorithm. The traditional DP approach recursively evaluates a binary tree to determine a solution, as seen in Fig. 2. Each node of the tree represents an upgrade that can be purchased, so there are as many levels as there are upgrades to purchase. The two edges leaving each node represent whether that node's upgrade is purchased.
In addition, the algorithm stores the added value of each upgrade option in the edge representing the action of purchasing the upgrade. The other edge stores zero, denoting the lack of improvement due to not purchasing the upgrade. Additionally, each node stores the remaining budget. If that budget is negative, the remaining subtree is pruned so that no more purchasing can occur. The simple example in Fig. 2 illustrates how the algorithm works on a binary tree.
The basic algorithm [27], [28], most easily and elegantly programmed recursively, is given by the following pseudocode in Algorithm 1: Optimality is achieved since the algorithm effectively searches the entire solution space. It does this efficiently by storing previously computed results of subproblems to minimize repeated computation.
The traditional DP algorithm does not work on the MILP investment model due to the interdependency of objective terms on upgrades. Since two upgrades may alleviate the same outage, their impact is dependent on whether the other is applied. This interdependency is overcome by bundling upgrades on the same device into packages, to consider which package should be applied. Specifically, the upgrade binary variable y i,d,u for an upgrade u depends on feeder i and device type d. It follows that outages can be partitioned by their feeder ID, device type pairs so that the bundles of outages VOLUME 8, 2021 and their associated collection of applicable upgrades are independent. The selection of a single package among many is notably different than the binary decision of inclusion underlying the binary decision tree of the traditional knapsack DP algorithm.

B. GENERALIZED RELIABILITY DYNAMIC PROGRAMMING ALGORITHM
A generalized reliability dynamic programming (GRDP) algorithm tailored to this problem effectively works by recursively evaluating a tree. In contrast to the traditional DP algorithm where each node represents a single object for purchase, the nodes in GRDP represent a (feeder ID, device ID) pair, which in turn represents a bundle of outages and their applicable upgrades. The collection of edges leaving a node represents the possible upgrade packages-including purchasing no applicable upgrades or all applicable upgrades-that are applicable to the node's outages as well as the package's contribution to the objective function postupgrade. The algorithm-instead of proceeding down two edges leaving each node as in the traditional DP-proceeds down a number of edges equal to 2 |I | where I is the set of possible upgrades for the node. Similar to the traditional DP, each node is updated with the remaining budget after the terminating edge's upgrade package cost is deducted, and subtree pruning occurs if the remaining budget becomes negative. This algorithm yields an optimal collection of upgrade packages since the outage bundles' contribution to the objective function are independent of one another, i.e., purchased upgrades that affect one outage bundles' contribution to the objective function will have no effect on another outage bundle's contribution. See Fig. 3 for an illustrative example of GRDP. The efficiency of the GRDP stems from its use of a cache to avoid a function call if it has already been made using the same arguments.
The GRDP algorithm just described can similarly be programmed recursively as follows in Algorithm 2:

C. ALGORITHMIC ENHANCEMENTS
The following enhancements to the GRDP algorithm improve computational performance: 1. Consolidation of outages with the same feeder ID and device ID so that the consolidated outage durations and customers affected have the same effect on SAIDI and SAIFI as the unconsolidated outages. Additionally, this algorithm greatly benefits from parallelization. The architecture of the decision tree lends itself well to division of work across computational cores since each core may explore independent branches given different immediate selections, while relying on a common cache. In addition, the values of each upgrade package may easily be precomputed, allowing computationally intensive portions of the recursive code to be parallelized. The models were developed in the hybrid C/Python language, Cython [31]. Python was used for easy data preprocessing, while the optimization algorithm was implemented in C. Since the implementation did not rely on external optimization libraries, C was selected due to its efficiency in highperformance computing.
Finally, the GRDP algorithm can use its cache to enhance performance when the same problem needs to be solved with multiple budgets. See Fig. 4 for an example.

IV. COMPUTATIONAL EXPERIMENTS
Experimental tests were performed to compare the computational performance of the MILP versus GRDP for varying numbers of outages, feeders, and budgets. All computations were performed on an Intel R Xeon R CPU E7-4850 v2 @2.3GHz, 529 GB RAM, 4 sockets, 12 cores per socket, 2 threads per core Linux workstation. Only one thread was used for each GRDP run while three cores are used by the CPLEX solver. The following experimental tests compare various facets of computational scaling for MILP versus GRDP.

A. VARYING NUMBER OF OUTAGES
The number of outages in the system affects MILP and GRDP differently. An increase in outages increases the number of constraints in the MILP and adds more terms to several constraints. For GRDP in contrast, having more outages VOLUME 8, 2021 only adds more terms to the upgrade packages' objective value contribution. The experiment in Fig. 5 varies the number of outages from 10,000 to 200,000 in increments of 10,000 while keeping the budget fixed at 150,000. The results indicate that the GRDP is more efficient as the number of outages increase.

B. VARYING NUMBER OF FEEDERS
The number of feeders in the system impacts the MILP model directly through only an increase in the number of binary variables y i,d,u . However, in general, as the number of feeders in a system increases, so does the number of outages. In GRDP, the number of (feeder, device type) pairs determines the level of recursion. The experiment in Fig. 6 varies the number of feeders from 100 to 1000 in increments of 100, while keeping the number of outages approximately equal and using a budget of 150,000. These results indicate that the MILP may be more efficient for large power systems (with several hundred feeders).

C. VARYING BUDGETS
Budget affects both MILP and GRDP by impacting the number of possible upgrades. MILP is solved through a solver's branch-and-bound algorithms. A smaller budget generally means that pruning of the branch-and-bound tree will occur earlier than with a larger budget, but not always, thus the more sporadic computational behavior of the MILP. Budget, similarly, affects how early pruning occurs in GRDP. The experiment in Fig. 7 varies the budget while keeping all other problem parameters and outages constant. The results indicate that, with respect to budget, the GRDP model is more scalable.

V. RESULTS
The results in this section are based on applying the presented GRDP optimization model and the MILP model in this paper to full utility data (∼ five years of outage data) supplied by a utility in the United States. Exact cost of upgrades is not used in this section, but rather estimated. Note that these results can be generated from either model (MILP or GRDP), though each model performs differently in terms of computation time, both models reach the same solutions.

A. BUDGET VERSUS OBJECTIVE
With a large budget, SAIDI and SAIFI can be greatly reduced, and as budget increases, the objective function decreases as in Fig. 8. However, decreasing marginal returns may justify restricting the budget used in improving reliability. Additionally, not all budget increases yield an improvement. The coarsest budget spacing where no improvement is  possible between consecutive budgets is the greatest common divisor of all upgrade costs.

B. INVESTMENT IMPACTS
The optimal investments will have an impact on the objective function by improving specific outages in the data sets. The following overlapped bar charts in Figs. 9-10 list all the outages that were affected by the upgrades chosen (64 outages would be affected by the upgrades). Fig. 9 presents the outage number of customers affected before and after upgrades, and Fig. 10 presents the outage durations before and after upgrades. A 3,000 budget was used with a 10,000-outage data set. The model output can be seen in Table 2, which includes the two upgrades chosen by the DP model along with the impact of those upgrades on the future SAIDI and SAIFI metrics. For visual purposes, a small budget and data set were used, and the outages in Fig. 9-10 are organized by the number of affected customers. Every upgrade chosen affects either the number of customers affected by the outage or the duration of the outage or both. Note, that in this case, the small budget limited the number of upgrade options considerably and the upgrades chosen primarily affected the number of customers rather than the duration. In addition, this model may inherently choose to improve the number  of customers affected rather than the duration, because the number of customers affected is a factor in both SAIFI and SAIDI, whereas duration is only a factor in SAIDI. For this reason, the objective function can weight SAIDI and SAIFI improvements differently which is shown in Section V.C.

C. SAIDI VERSUS SAIFI TRADEOFF
It may be of interest for an investor to see the tradeoff between SAIDI and SAIFI for the purposes of investment decisionmaking. A Pareto frontier in Fig. 11 is useful for seeing such tradeoffs. It is generated by replacing the optimization model VOLUME 8, 2021 objective in (1), with a convex combination of its terms. This shows that since SAIDI and SAIFI are so closely correlated (both are dependent on the number of customers), the solution does not have a drastic change when weighting SAIDI over SAIFI or vice versa. Changing the weights will however change the investment decisions.

D. CONVERGENCE OF OBJECTIVE WITH INCREASING OUTAGES
A desirable result is convergence of objective value as the number of sampled outages increases. This is shown in Fig. 12. This type of convergence practically makes it unnecessary to sample more outages once the objective value has sufficiently converged. That is, sample size is important. This gives the utility an idea of how many outages are needed in the data set to reach a reliable solution.

VI. CONCLUSION
This paper presents a generalized reliability dynamic programming (GRDP) algorithm that selects the optimal small-scale investments to improve power distribution system reliability metrics SAIDI and SAIFI. In addition, the GRDP model is compared and validated against a MILP model. The models suggest how power grid distribution reliability may be improved through investments under a fixed budget. Both models are shown to reach the equally optimal solutions on real utility data but vary significantly in computation performance for different cases. Scaling computational experiments indicate that MILP is more beneficial for large distribution systems with many feeders, but the GRDP is more efficient for large outage data sets and large budgets. Finally, results are presented on a full utility outage data set provided by a utility in the United States. The results indicate the usefulness of the GRDP algorithm for picking optimal small-scale investments to improve power distribution system reliability metrics.

ACKNOWLEDGMENT
Sandia National Laboratories is a multimission laboratory managed and operated by National Technology and Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International, Inc., for the U.S. Department of Energy's National Nuclear Security Administration under contract DE-NA0003525. The views expressed in the article do not necessarily represent the views of the U.S. Department of Energy or the United States Government.