Detecting Power Lines Using Point Instance Network for Distribution Line Inspection

Power outages can disrupt daily domestic activities as well as the economy as operations are hampered when they occur. They can decrease work productivity by delaying operations that require electricity. The key solution to this problem is to ensure that there are fewer or no power interruptions. This can be achieved by ensuring secure and continuous network operations through regular maintenance and inspection. However, the traditional inspection technique of foot patrol is risky, laborious, and time-consuming. A preferable contemporary technique uses an unmanned aerial vehicle (UAV) for inspecting distribution lines. Detecting power lines are crucial for real-time motion planning and navigation of UAVs. Previous techniques that depend on conventional filters and gradients may fail to detect power lines because of noisy backgrounds. Thus, this study proposes a novel technique by adopting the Transfer Learning approach. The process involves re-training the Point Instance Network (a road lane detection model) with images for power line detection. The proposed method extends the PINet model by adding a comparator for rotation block before it and a postprocessing block after it. This study generates four versions of the model, each of which was trained on one of the following datasets (i) self-gathered images captured by a handheld camera, (ii) a drone, (iii) publicly accessible images from the Power Line Dataset of Mountain Scene (PLDM), and (iv) Power Line Dataset of Urban Scene (PLDU). Experimental results on each dataset confirm the feasibility of the proposed approach.


I. INTRODUCTION
Electricity is an indicator of a sustainable community since it associates with economic growth and social equity [1]- [3]. A significant number of socio-economic activities such as jobs to be accomplished at offices, homes, schools, businesses, and factories, rely on electricity regularly. Electricity is a necessity for human beings [1].
One of the major components of electrical infrastructure is the power line. It is used for transmitting electrical energy from one location in the country to another. The reliability of a power line is an essential responsibility of the power sector because of the requirement to ensure a continuous supply of electricity to consumers. Power outage, a disruption in the The associate editor coordinating the review of this manuscript and approving it for publication was Amin Zehtabian . delivery of electricity, has both direct and indirect economic consequences. Direct impacts include, but not limited to, restart costs, loss of production, equipment damage, and raw material spoilage. Indirect impacts include, but are not limited to, the cost of income postponement and declining market share. The power outage also affects society because of the loss of consumer welfare, leisure time, health and safety risks, or discomfort at a work due to nonoperating appliances, such as electric fans and air conditioners [4]. Consequently, the power outage causes substantial financial loss to both producers and consumers.
Power outages are mostly caused by power line faults in a distribution system. According to a study conducted by the Electric Power Research Institute (EPRI), tree contact is one of the leading causes of power line faults [5]. Equipment damage, severe weather condition, and animal/human intervention are other major causes of power line faults. According to a 2017 report, one in five power line faults is because of fallen trees or trees close to power lines. According to a report issued by the National Grid Corporation of the Philippines (NGCP) in 2016, the line-to-ground fault caused by encroaching untrimmed banana trees planted along the distribution line's right-of-way is a major cause of the power outage in the Zamboanga Peninsula.
Persistent maintenance, surveillance, and inspection of power lines represent vital operations for reducing the frequency of power outages. Early detection of faults in power equipment can prevent severe and costly damage [6]. Manual conduct is one feasible approach to inspecting distribution lines. For instance, the power lines and other electrical infrastructures in the Philippines are often surveyed, inspected, and maintained by lineworkers on duty. However, the inspection by lineworkers has disadvantages. First, the operation is labor-intensive because additional lineworkers are needed to help the pole climbers. Second, it is dangerous because electrocution can occur at any time. Third, it is time-consuming patrolling on foot, and scaling poles to cover an area takes time. Lastly, lineworkers are prohibited from patrolling over wreckage or rubble in the event of a disaster. Otherwise, their safety is compromised. To overcome these limitations, an unmanned aerial vehicle (UAV), known as a drone, is used to perform the task. UAVs can fly close to the power lines, which have several benefits such as being a convenient, flexible, inexpensive, and safe way to acquire data from the distribution line system [6]- [12].
Image processing is used extensively in aerial inspection [13]. The motivation of the proposed technique in this study is to take the knowledge obtained in the previous task and apply it to a different but related problem. The authors fine-tuned a pretrained model from the road lane detection problem to detect power lines. Fig. 1 shows that the weights from a road lane detection model are fine-tuned to perform seamlessly on another but related task. The authors obtained images using a handheld camera and a drone inside the university environment to demonstrate the feasibility of the proposed approach. They further validated the method using two publicly available datasets, the power line dataset of a mountain scene (PLDM) and power line dataset of an urban scene (PLDU), published in [14]. To the best of the authors' knowledge, this approach has not been previously introduced.
The rest of the paper is organized as follows. Section II discusses the existing literature on power line detection. Section III describes the proposed approach. Section IV illustrates the experimental results on both the self-constructed and open-source datasets. Finally, Section V contains the conclusions and recommendations.

II. RELATED WORKS
Several techniques for power line detection have been presented thus far, and are discussed in the succeeding subsections. Proposed approach involves fine-tuning a pretrained model from the road lane detection problem to address the power line detection problem. In this paper, four datasets are used to validate the proposed method -from self-gathered datasets captured using (a) handheld camera and (b) drone, and from two publicly available datasets published in [14], the (c) PLDM and (d) PLDU datasets.

A. LINE-BASED METHODS
The Hough transform is the most frequently used imaging technique for detecting general lines. It has been beneficial in several real-life applications including but not limited to the following. First, the Hough transform is a handy tool for detecting tactile pavement, assisting visually impaired individuals to walk in public places [15]. Second, it is the primary method used in precision agriculture to detect crop rows and direct UAVs in selectively spraying agrochemicals in the field [16]. Third, it is significantly faster than raster scanning in detecting ellipse-enclosing characters in large-sized document images [17]. Fourth, it is capable of detecting nearby structures to map robust relevant landmarks for high-speed railway mapping [18]. Lastly, it can be used to detect and track road lanes for lane departure warning systems [19].
The Hough transform can also detect power lines for a distribution line inspection [9], [10]. This approach takes advantage of the fact that a power line is essentially a line that appears to be a straight segment in an image. Applying a filter to an image to eliminate unnecessary data is one way to improve the current Hough transform approach. For instance, the Pulse Coupled Neural Network (PCNN) filters unnecessary information from images such as grass, trees, roads, buildings, and other background noise [11]. PCNN, a single-layered, two-dimensional neural network, generates an edge map of intriguing lines, unlike the typical results of Canny and Sobel filters. In [20], the authors used the Sobel filter to obtain the edges in the image, and a combination of Hough transform and Particle Filters to detect the power lines from the edges. The number of detected points for each line in the image corresponds to the particles' weights. Another improvement technique is the creation of Region-of-Interests (ROIs) to isolate the subject matter. In [12], the image and Hough table information defines the ROIs. This approach is only reliable when the camera is neither too far from the targets or way above the targets.

B. KNOWLEDGE-BASED METHODS
With the Hough transform's limitations in a complex environment, various alternatives emerged by leveraging power line context characteristics. First, the power lines appear in groups and are parallel to one other [11], [21]. Second, power lines are connected by the pylon poles. Pylon poles, which serve as elevation proofs, must be detected to recognize power lines [9]. Third, power lines are straight parallel lines when viewed at the overhead level. Due to this auxiliary, two equidistant points of a straight line will exist at the circumference of a circle. As a result, Circle Based Search is a feasible power line detection approach [22]. Lastly, no other artifact within an overhead frame will have a high density besides power lines. This is because metallic objects reflect most electromagnetic radiation in the visual spectrum [23].
Most of the auxiliaries for detecting power lines are manually determined as shown in [24]. However, a technique proposed in [25] to optimize the procedure uses a local optimization approach to automatically acquire the context information of each type of auxiliary.
Detecting power lines can be achieved by recognizing them as objects in a way that improves auxiliary information performance [26]. The authors in [27] presented that a power line is an object if it contains the thin line structure, special material, and flat color properties. Thus, it is possible to create an object-awareness detection algorithm using these properties. The approach in [28] described the applicability of spatial correlation assistance between line and pylon. With this correlation, the technique achieved high detection rates with low false alarm rates.

C. DEEP LEARNING-BASED METHODS
The recent approach to detecting power lines involves developing deep neural networks. However, the unavailability of a sufficiently large public dataset of power lines with pixel-wise annotations is a drawback of the deep learning-based method. In [29], a model based on dilated convolutional networks was created and trained on the synthetic images of wires generated by POV-Ray, a ray-tracing engine. The LS-Net [30], a fast single-shot line segment detector, was trained using synthetic images of power lines produced by the Physically Based Rendering method. The study in [14] provided two datasets of power line images taken from urban and mountain scenes of China due to the shortage of power line datasets in the public domain. The PLDU consists of 573 images, whereas the PLDM consists of 287 images. The study in [31] also provided a dataset containing 4,000 visible and 4,000 infrared images of power lines within Turkey. The study in [32] used these visible images to train their convolutional neural network to perform binary classification to a colored image as having or not having power lines. Finally, the work in [33] provided the University of South Florida (USF) dataset having low-quality videos of thin lines taken from various urban regions in Florida and New Zealand. The dataset contains 86 videos with 10,160 wires spanning 5,576 frames. This dataset was used in [29] to evaluate their trained network.
Deep neural networks can be developed in different ways. One prominent technique is to construct the deep network from scratch. This is by selecting and stacking layers to extract features and eventually produce the desired output. Another approach is via Transfer Learning. Here the weights of a previous task's pretrained model are used to improve the generalization of another but related task. Several studies have explored the applicability of Transfer Learning for distribution line inspection using UAV images. For instance, the weights from the RetinaNet object detector [34], which was originally pretrained on ImageNet, are fine-tuned to automatically map roadside utility poles with crossarms from Google Street View images [35]. Faster R-CNN [36], an end-to-end deep learning algorithm that was also pretrained on the ImageNet dataset, was used in [37] to build a detection model that can accurately identify transmission line faults categories. Lastly, the Mask R-CNN [38], also pretrained on ImageNet, was improved to develop a segmentation algorithm that can detect power lines [39].
In this study, the authors employed Transfer Learning to develop a novel technique for detecting power distribution lines. They built on an existing road lane detection architecture. Road lanes are curves that self-driving vehicles use for navigation and control [40]. From this perspective, power lines are curves that guide UAVs in autonomous navigation during distribution line inspection. Thus, this study presents this application by fine-tuning Point Instance Network (PINet) [41]. The PINet's architecture has shared layers for feature extraction and various branches for detection and embedding to generate precise points on the lanes. The trend in Fig. 2 illustrates the refinement of the network performance as loss decreases with each epoch. The original PINet network's trained weights were able to transcend to fit a good generalization to extract features from images of the power line detection problem.  Timing diagram shows that the input image is rotated only when necessary. The triggering factor is when one of the detected power lines has less number of points than the threshold value. The comparator block secures that the image to be fed to the PINet network is in the desired layout. The resulting images with an orange border illustrate the rotated images with the same resolution as the original.

III. PROPOSED METHOD FOR DETECTING POWER DISTRIBUTION LINES
This study proposes a novel technique for detecting the power lines by refining the pretrained weights of the road lane detection model. The details of the proposed method and dataset used for training are discussed in the following subsections.

A. SYSTEM PROCESS FLOW
The end-to-end implementation of the proposed method is summarized into three major blocks, namely, the comparator for rotation block, the PINet network block, and the postprocessing block (Fig. 3). Each block has its vital function, which is discussed in the succeeding paragraphs.

1) COMPARATOR FOR ROTATION
The first block's major function is to check if the input image is in the optimal layout and to generate an image that fits the requirement of the next block. This block receives two input parameters, an input image and vector, A t , containing the number of points for every detected power line. The input image is a RGB frame with high resolution. A t is a vector with a size equal to the number of detected power lines, and t represents the iteration index. Mathematically, 1 , a 2 , . . . , a n | n is number of power lines } (1) At the start of the program (t = 0), A 0 is null. This is because there are no existing points at the start of the program's execution. The number of points is updated only at the succeeding runs. For instance, the first run generates points for each power line at the output block. If the proposed system detects three power lines at the first input image, then A 1 = {a 1 , a 2 , a 3 }. Where a 1 is the number of points from the first detected power line, a 2 is from the second detected power line, and a 3 is from the third detected power line. At the second iteration of program execution (t = 1), the comparator block receives the second input image and vector A 1 from the previous run. At the third iteration of program execution (t = 2), this block receives the third input image and vector A 2 . At the fourth iteration, it receives the fourth image and vector A 3 , so on and so forth.
The comparator block determines if the input image should be rotated by comparing the minimum number from vector A t (min(A t )) to the threshold value. In this study, a threshold value of 10 points was assigned by the authors. In other words, the input image will be rotated counter-clockwise if min(A t ) is less than 10 points. This rotated image will then be fed to the next block. Fig. 4 illustrates the timing diagram of the instances where the input image is rotated either counter-clockwise or clockwise. The image layout changes only when the number of points is below the threshold value.  The first input image in the same figure is at the original layout. The rotation is applied when the detected power lines begin to curve horizontally. The layout reverts to the original setup through a clockwise rotation when the power lines emerge again as vertical curves.
The rotational transformation is motivated by the unprecedented alteration of the slopes of the detected power lines. From the road lane detection problem, the lanes are always assumed to be vertical curves. However, the power lines can be considered as vertical or horizontal curves depending on the angle where the drone is present. Therefore, the proposed method should compare the number of points with the threshold value. Thus, as the name implies, the comparator block.

2) PINet ARCHITECTURE
PINet was originally developed to solve the road lane detection problem. It is a network that can generate points and distinguish between lane instances. It receives an input image of 512 × 256 pixels. The image is forwarded to a resizing layer and then to a feature extraction layer. A sequence of the convolution and max-pooling layers compress the size of the input data.
The features from the input data are extracted using two stacked hourglass blocks. Each hourglass block comprises down-sampling bottleneck layers, the same bottleneck layers, and up-sampling bottleneck layers. There are three output branches at the end of every hourglass block that can predict each grid's confidence, offset, and instance feature. Details on the computation of the loss function for each branch are elaborately explained in the original study [41].
In this study, the PINet architecture was not altered. However, the weights of the pretrained model were fine-tuned to solve the power line detection problem with better generalization.

3) POST-PROCESSING METHOD
The developed network's raw predictions occasionally yield false-positive points. A power line is expected to contain only a smooth curve. However, there are some outliers from the predicted points that are visually distinguishable. Fig. 5 illustrates the existing problem and the outcome after the postprocessing method. This method adopts the idea of removing outliers by eliminating the influential points across the prediction for a single power line. The governing procedure is presented below: • Step 1: Obtain every detected power line point. • Step 2: Using Ordinary Least Squares (OLS), fit a regression model for the data points of each detected power line. For n sets of data points, there are also n regression models. The data points can be represented with linear regression models because of two main reasons. First, the points are two-dimensional numerical values, the x and y coordinates. Second, a power line instance is almost always linear. The OLS model can be written in matrix notation as, where • y is the response variable (y-coordinates), • X is the explanatory variable (x-coordinates), • β β β is the unknown parameters (coefficients), • and ε ε ε represents the unobserved random variables (errors).

• Step 3: Find all influential points in each regression
model. An influential point is a point that significantly changes the parameter estimates when removed from the calculations. Fig. 6 shows the scatter plots retrieved from the actual data of a sample image. The blue line represents the least-squares regression line when an outlier is included in the analysis. The yellow line represents the least-squares regression line when the data point is tossed out. The two outliers in the scatter plot in Fig. 6a show a high influence on the blue line. On the other hand, the regression line in Fig. 6b shows a best-fit to the data points. The influence is measured using Cook's distance. The equation is expressed as, where •ŷ j is the jth fitted response value, •ŷ j(i) is the predicted jth response value with ith data point removed, • p is the number of coefficients in the regression model, and • s 2 e is the mean squared error. • Step 4: Remove the influential points in each regression model.

•
Step 5: Repeat from step 1 to step 4 for all input images.

B. DATASET
This study used a handheld camera and drone to collect images to serve as data sets for fine-tuning the network. All images are set to have a 1280 × 720 resolution. Table 1 presents the distribution of the scenes on training and test sets based on the number of power lines. The raw images captured by a handheld camera were evenly distributed. This is because the images were obtained through video streams, and the authors randomly select one image each for the training and test set for every 20 consecutive frames in the video stream. Meanwhile, the authors performed data augmentation on the images captured by drone for the training set, resulting in a data split of 80% to 20%. The data were obtained within the Mindanao State University -Iligan Institute of Technology. This is to prepare the developed system to operate accurately on the deployment. This study uses two publicly available PLDU and PLDM datasets, aside from self-gathered images, to evaluate the proposed method's performance. The distribution of images per set is listed in Table 2, as presented in the original paper [14].

IV. EXPERIMENTAL EVALUATION
The originally trained PINet model exhibits low false positives at validation on the tuSimple dataset. Therefore, the goal of this study is to fine-tune this pretrained network to perform a good generalization for the power line detection problem. The initial setup of the network training is adjusted to have a learning rate of 1e −4 , a to 1.0, and γ n to 1.0 during the first 300 epochs. For the last 300 epochs, the values of a and γ n were both changed to 1.50. The optimizer used is Adam with a learning rate of 2e −4 . The expected output has a 64 × 32 size. The training happened at Google Colaboratory, which was powered by GPUs of Compute Engine backend. The test hardware is a personal computer with NVIDIA GeForce GTX 1050 Ti graphics card. Fig. 7 shows that the validation losses exhibit decreasing trends during training of the four datasets.
Comparatively, the validation loss curve from the self-gathered image dataset using a handheld camera has the smoothest declining curve among all validation loss curves. The major reason is that the power lines in that dataset are nearly identical to the road lanes in the tuSimple dataset. They are brighter lines of almost the same thickness. However, the validation losses from PLDM and PLDU datasets exhibit abrupt spikes, displaying a nonideal but diminishing trend. It is because the power lines in the two publicly accessible datasets' images are less white and considerably thicker than the road lanes in the tuSimple dataset.

A. EVALUATION METRICS
This study adopted the evaluation metrics presented in [41], which include the accuracy, number of false positives, and number of false negatives. Accuracy means the average number of the correct points and is defined by the following   (4) where • C clip denotes the number of correct predictions on the given image clip, • S clip denotes the number of ground-truth points in the same image clip. A false positive, from a power line detection problem perspective, is an error indicating a point exists in the image clip when it should not. This results from an incorrect affirmative decision. On the other hand, a false negative is an error indicating a point does not exist in the image clip when it is supposed to exist. Moreover, a false negative is an error indicating a point does not exist in the image clip when it should exist. It results from the failure to identify the presence of a point. The two errors are defined in Equations 5 and 6 respectively. Equation 5 is also known as the probability of false discovery. While Equation 6 is also known as the miss rate.
where • F pred denotes the number of incorrect predictions, • N pred denotes the number of predictions, • M pred denotes the number of missed predictions, and • N gt denotes the number of ground-truths. In statistical analysis, the F1-measure (also known as F1score) is the harmonic mean of the precision and recall of the model's performance on the test dataset. It implies the robustness of the method in the given problem. Precision is the number of correct predictions over the total number of predictions. Alternatively, it can be computed by subtracting the probability of false discovery from one. On the other hand, VOLUME 9, 2021   Recall is the number of correct predictions over the total number of ground truths. Alternatively, it can be calculated by subtracting the miss rate from one. Thus, F-measure can be obtained by using Equation 7.
The authors conducted experimental evaluations on the proposed method. They used the images from the test set of every dataset. The authors used 475 images from the handheld camera and 120 images from the drone ( Table 1). The authors also used 50 images from the PLDM dataset and 120 images from the PLDU dataset ( Table 2). The experiments' objectives are to calculate the accuracy, number of false positives, and number of false negatives. That is performed by comparing the predictions with the ground truth. Table 3 presents the detailed evaluation results. In all datasets used by the authors, the proposed method exhibited high accuracy, low false-discovery rate, and low miss rate on the images captured by a handheld camera. Additionally,  the method also performed with acceptable accuracies on the other datasets.
The visible features of the power lines in the images are the major cause of the large disparity in performances. The input images from every dataset showed that the power lines from the image captured by the handheld camera are more visually vivid compared to the other datasets (Fig. 8). They are whiter and more distinguishable from the complex environment. Further, the PLDU dataset exhibited the second-highest accuracy. This is because the power lines in this dataset are thicker compared to the other datasets. The thickness of the power lines moderately affects the method performance. Meanwhile, the images captured by the drone present difficulties since power lines are often difficult to identify from their surroundings, particularly when they overlap with other objects of the same properties. Nevertheless, the evaluation results show that Transfer Learning from a pretrained road lane detection model to the power line detection agent is feasible. The input images from the self-gathered data using a handheld camera showed that the power lines were not in a uniform point-of-view from different scenes (Figure 9). The angles from the reference point where the authors positioned the camera may vary from one scene to another. In some scenes, the power lines are taken from the sides. While on another scene, the power lines are taken from a bird's eye view. The non-uniformity of the shots wants to simulate the possible varying views of the power lines, thus intending to train a model to predict robustly even in different pointof-views. The proposed method showed feasibility in these cases as long as the power lines are vivid and do not mix inadvertently with the background. The quality of the image still matters as it directly impacts the accuracy of the model.
The drone is positioned above the power lines during data gathering to capture the power lines on a bird's eye view. As shown in Figure 10, the input images appear as parallel vertical lines converging at the horizon. The proposed method had an acceptable result with this dataset. However, the performance may deteriorate at some point, especially when the power lines are almost untraceable because they mix with the background. This scenario causes an increase in false discovery rate and miss rate. Improving the quality of drone shots can enhance the performance of the model.
Using the PLDM and PLDU test sets, the authors compare the proposed method with the reported results from the paper that introduced the Convolutional Neural Network-based cable detection method (CNNCDM) [42]. Most of the reported results can be cited back from the paper that studied the convolutional features and structured constraints (CFSC) for power line detection [14]. Furthermore, the paper in [42] compared the performances of several methods, like Bi-directional cascade network perceptual edge detection (BDCN) [43], CFSC [14], Richer convolutional features for edge detection (RCF) [44], Holistically-nested edge detection (HED) [45], Gestalt Grouping [46], and Canny [47]. Figure 11 shows test results on some sample images from both PLDM and PLDU datasets. The sample images of the previously-mentioned methods are copied from the CNNCDM paper. It is worth noting that these methods produce resulting images of predicted edges.
Meanwhile, the proposed method of this paper predicts key points of the power lines, just like CNNCDM. Although, the proposed method can segment instances of the power lines, unlike CNNCDM. As shown in Figure 11, the predicted key points for each instance are represented with a single line through a curve fitting with linear least-squares. In this manner, the key point estimation and point instance segmentation are suitable for detecting power lines. Tables 4 and 5 compare the F1 scores of the proposed method and some existing works as evaluated on the test sets of both PLDU and PLDM. The reported results are copied from the CNNCDM paper. The proposed method may not excel on the PLDM test set, but it surpasses all existing methods on the PLDU test set. Besides, predicting key points and segmenting power lines is much significant for the distribution line inspection.

V. CONCLUSION
The authors of this paper proposed a new power line detection method that uses Transfer Learning to fine-tune a pretrained road lane detection network in order to detect power lines for distribution line inspection. The proposed method, which is based on PINet, can detect power lines from a given image and handle line orientation variations. The results from the four datasets demonstrated the feasibility of the proposed method as it achieves acceptable accuracy, low false-discovery rate, and low miss rate. The study's future work includes to deploy the method to an UAV's microcontroller, investigate other road lane detection methods as a replacement for PINet, improve latency of the postprocessing technique, and improve the drone's camera setup to obtain more vivid power lines.

ACKNOWLEDGMENT
Thanks to the Department of Science and Technology-Engineering Research and Development for Technology.