SS-IPLE: Semantic Segmentation of Electric Power Corridor Scene and Individual Power Line Extraction From UAV-Based Lidar Point Cloud

Key objects’ semantic segmentation and power line extraction from the electric power corridor point cloud are critical steps in power line inspection. However, the massive amount of point cloud data and missing power line points pose a challenge to the object extraction. To complete the extraction of power lines and other essential objects, a method called SS-IPLE is proposed, which is based on a pointwise multilayer-perceptron semantic segmentation network. The method consists of two main parts: electric power corridor semantic segmentation and individual power line extraction. In the segmentation step, SCF-Net is employed as our primary segmentation network, and the network can process large-scale point clouds. To further improve the segmentation ability of SCF-Net in the corridor, a local coding module is designed to construct the SCFL-Net. In the individual power line extraction step, individual power lines are extracted by flexible grid filtering, effectively overcoming the point-missing problem. The corridor point cloud semantic segmentation and individual power line extraction experiments are conducted in different corridors collected from suburban areas. Promising results are obtained for both semantic segmentation and individual power line extraction, with an mIou of point cloud semantic segmentation and a mean extraction rate of 96.70% and 96.56%, respectively.


I. INTRODUCTION
A RELIABLE supply of electricity is essential in modern society. As the backbone of the electrical network infrastructure, power lines are continuously exposed to the harsh natural environments, leading to potential failures and damage in electric power corridors. In addition, tree growth close to power lines can also damage electrical networks and lead to bushfires [1]. Therefore, it is necessary to inspect and maintain the power line regularly.
Traditional power line inspection methods are labor-intensive and inefficient. Currently, unmanned aerial vehicle (UAV)-based power line inspection has become a cost-effective and efficient short-distance solution [2]. UAVs provide various data types for power line inspection through remote sensing technologies (such as optical cameras, Lidar, and infrared cameras). The accuracy of power line positioning extracted from the two-dimensional (2-D) observations is limited and requires improvement [3]. Point cloud data obtained from Lidar contains accurate threedimensional (3-D) coordinate information of objects, which is essential for constructing a 3-D map for UAV power line inspection and accurately measuring the hidden danger area of power targets. Consequently, point cloud data are increasingly used in power line inspection [4].
Power line inspection includes two aspects: assessing the status of power lines and their distance from surrounding potentially threatening objects, such as vegetation and buildings. Therefore, extracting power lines and potential threat objects from point cloud data is the basis of power line inspection. Moreover, semantic segmentation methods can achieve fine classification of the electric power corridor scene. This fine-grained classification enables a more detailed understanding and analysis of the objects within the corridor scene. To achieve comprehensive and refined power line inspection, semantic segmentation of the corridors should be an essential part of the power line extraction process.
In previous studies, the extraction of power lines involves two independent steps: the extraction of candidate power line points and refinement, where individual power lines are extracted from the candidate points [5]. In this study, we adopt a similar approach but replace the candidate power line point extraction process with semantic segmentation of the corridor. For the extraction of the candidate points, the previous related methods can be broadly classified as supervised and unsupervised according to whether the label is used.
Unsupervised extraction methods: In these methods, the point cloud is first divided into grids, and the grid is used as the unit for the candidate power line point extraction. Zhang et al. [6] extracted the candidate points by neighboring grids' height similarity estimation and linear-feature-based clustering. However, a significant number of ground points and vegetation points impact computational efficiency and extraction accuracy. To eliminate this problem, Jung et al. [7] removed the ground surface and adjacent objects before extracting power line points and extracted the candidate points by calculating the similarity of adjacent grids and their internal features. Shi et al. [8] projected the point cloud onto the XY plane's cell to extract the power line points and pylon points based on the height histogram distribution characteristics of the cell. The above-mentioned grid-based methods are susceptible to the size of the grid and high tree point noise.
Supervised extraction methods: Most of the previous methods used traditional classifiers, such as Random Forest, Boost, and support vector machine (SVM) to classify the corridor point cloud and get the candidate power line points. Kim and Sohn [9] constructed 21 features from the sphere domain of points and classified a corridor into power lines, buildings, pylons, ground, and vegetation based on Random Forest. Bo et al. [10] added intensity and echo features based on Kim's method. However, both approaches require constructing too many local features, which results in redundant information. Liu et al. [11] selected the top five 2-D and 3-D features with the best discrimination and combined them into a robust classifier through the AdaBoost algorithm. SVM has also been applied to electric power corridor classification as a classical classifier [12]. These methods have achieved impressive segmentation results in experiments, but building broadly applicable features is still complicated.
Deep learning (DL) has shown good performance in classification [13] and semantic segmentation [14]. DL-based point cloud semantic segmentation methods can be divided into four categories: multiview-based, voxel-based, pointwise-based, and graph-based methods. PointNet [15] aggregates per-point features by max-pooling and causes local information to be missing. Therefore, many methods are proposed to make improvements, such as PointNet++ [16] and PointWeb [17]. These methods cannot directly deal with larger point clouds due to expensive computation and the incapability of capturing structures. To address these problems, RandLA-Net [18] is proposed to solve these two problems by random sampling and expanding the single-point receptive field. However, its ability of local context information extraction is not satisfactory. To address this issue, SCF-Net [19] enhances the ability of RandLA-Net to capture local context information without losing too much computational efficiency. Meanwhile, MSIDA-Net [20] encodes local information in multiple coordinate systems to enhance the local feature expression ability. Although many DL-based semantic segmentation methods have been developed, few are applied in the corridor point cloud. Li et al. [5] proposed a segmentation method based on a graph convolution network, whereas Guan et al. [21] converted the raw point cloud to cuboids before being fed into the PointNet, achieving high classification accuracy.
For the refinement stage, we review some related works as follows. Some extraction methods are based on the traditional plane line detection algorithm, such as the Hough transform algorithm [22]. On the basis of this algorithm, Guan et al. [23] separated power lines into linear clusters in the XY plane and then applied a Euclidean distance clustering method to cluster the individual power lines in 3-D space. Lehtomaki et al. [24] used the RANSAC method to detect linear clusters and cluster the individual power lines according to the angle and distance threshold in 3-D space. These methods have an excellent extraction effect but still need to be adapted to address missing power line points and overlapping on the XY plane.
Other methods mainly extract the individual power lines by clustering and regional growth in 3-D space. In terms of clustering, Peng et al. [25] proposed an individual power line extraction method based on DBSCAN clustering. Similarly, Mao et al. [26] used the distance between power line points and the overall fitting equation of all points as features and applied the DBSCAN clustering algorithm in the feature space to extract individual power lines. Other clustering methods are also applied in the extraction process, Chen et al. [27] constructed a spherical neighborhood for each point, and the individual power lines are separated by conditional Euclidean clustering with constraints of linear features in the spherical neighborhood. In terms of regional growth, Shi et al. [28] selected the lowest elevation point of the power line points as the seed point and extracted individual power lines by a regional growth algorithm with spatial feature constraints. The above-mentioned methods work well to solve overlapping problems, but the missing power line point problem is still a challenge for individual power line extraction.
To sum up, this study aims to propose an automatic power line extraction method based on semantic segmentation of electric power corridors. Our main contributions are as follows.
1) A power line extraction algorithm based on pointwise multilayer-perceptron is proposed. Different from the previous extraction algorithms that only extract power lines, the proposed method can precisely implement semantic segmentation of multiple objects in the electric power corridor scene. 2) In the corridor semantic segmentation stage, according to the characteristics of the local structure of the corridor point cloud, we propose a local coding module (LCM) that can effectively and fully aggregate a single point's local information to enhance the network's segmentation ability. 3) To address the issue of missing power line points, an individual power line extraction method based on a flexible grid is designed. This method is robust against missing power line points and can be applied to various types of power lines.

II. PROPOSED METHODOLOGY
In this section, we present the power line extraction method named SS-IPLE. The extraction process comprises two steps: electric power corridor semantic segmentation and individual  power line extraction. In the first step, the corridor point cloud with only XYZ coordinate information is input and processed by the local coding module (LCM). Subsequently, it is fed into SCF-Net to classify into four categories: ground, vegetation, pylons, and power lines, as shown in Fig. 1(a). In the second step, to accurately extract the individual power lines between two pylons, a two-stage individual power line extraction method is proposed, including span power line extraction and separation of individual power lines, as shown in Fig. 1(b). The details of these two steps are presented below.

A. Electric Power Corridor Semantic Segmentation
SCF-Net can effectively learn context features for large-scale point clouds. Here, SCF-Net is used as the basic semantic segmentation network. To better capture the local structural information of corridors, the LCM is integrated into the SCF-Net to enhance its ability to capture local structural information. This combined segmentation network is named SCFL-Net, as shown in Fig. 2(a). The LCM uses the local interior point distribution and point elevation statistical characteristics to construct local features. A shared MLP is used to map the features to the same dimensional space. In this section, we will introduce the LCM in detail.

1) Local Distribution Characteristic:
The neighboring point distribution characteristics mainly represent the spatial relationship between a single point and its neighboring points. We express it as the distance between a point and the center of the mass point of the local neighborhood. Generally, if the neighboring points are evenly distributed, the center of mass will be close to the point. Define a point p i = (x i , y i , z i ) ∈ P , where P is the set of all points, the number of points in the local neighborhood is k, and a set of neighborhood points Q i = {p j } , j = 01, 2 . . . k can be obtained by KNN search. The distribution characteristics can be defined as Ld i , which is given as follows: where "||·||" is the L2 norm.

2) Local Elevation Standard Deviation:
The local elevation standard deviation is an index that describes the degree of elevation fluctuation in a point cloud. It can effectively quantify the difference in elevation variation between points. A large elevation fluctuation means the local region has a high standard deviation, such as vegetation and pylons. Conversely, slight elevation fluctuation means a lower standard deviation, such as ground and power lines. The expression for elevation standard deviation is given as follows: where z j is the elevation value of p j . Finally, the above-mentioned two features are concatenated with the single-point coordinates, and the local feature information can be mapped to the same feature vector space by using a shared MLP where "⊕" is the concatenation operator. (x i , y i , z i ) are the coordinates of p i . In summary, the structure of LCM is shown in Fig. 2(b). It takes all the 3-D point coordinates in the corridor as input and generates an eight-dimensional feature vector that contains local information for each point as output.

B. Individual Power Line Extraction
Extracting individual power lines from the semantic segmentation results can be divided into two steps: separation of power lines between adjacent pylons and separation of individual power lines. The separation of power lines between adjacent pylons can also be called span power line extraction.

1) Span Power Line Extraction:
Based on the semantic segmentation results, all the points belonging to pylons and power lines can be extracted. The points corresponding to a pylon can be obtained by using a clustering method. To extract the span power lines accurately and completely, we first project the power lines and the corresponding pylon points onto the X-Y plane, as depicted in Fig. 3(b). The green line in the enlarged region represents the ideal axis of symmetry of the pylon, which effectively distinguishes the power line points in the span from those belonging to others. Here, we calculate the axis of symmetry of the pylons as follows.
First, a segment of the pylon body is taken vertically from top to bottom to represent the pylon. Assuming that the axis of symmetry passes through the center point of the pylon, we calculate the average of the pylon's points to obtain the center point (x 0 , y 0 ) as shown in Fig. 3(c). The axis of symmetry equation of the pylon can then be expressed as follows: where α * is the angle between the axis of symmetry and the X-axis.
On the other hand, the equation of all lines passing through the center point can be expressed as follows: where α is a variable that denotes the angle between a line and the X-axis. Second, to determine the equation, we need to find α * in α. This can be achieved by minimizing the variance of distances between all pylon's points and the line denoted by line α (x). The distance between the i th point of the pylon and line α (x) is represented as dis α i . The variance of distances corresponding to line α (x) can be expressed as follows: where dis α is the average distance of all pylon's points from line α (x), and N is the total number of points in the pylon. Fig. 3(d) shows the functional relationship between α and Var α . The minimum of Var α corresponds to the angle α * . To search for α * , the golden section search way [29] is utilized within the range of α ∈ [−90 • , 90 • ].
2) Separation of Individual Power Lines: In span power lines, each power line does not overlap and is stratified in the elevation direction. Based on this, an individual power line separation algorithm that utilizes flexible grid filtering is proposed. First, the span power lines are rotated parallel to the X-axis to reduce computation, and the discrete points are filtered by radius filtering. The algorithm flowchart is illustrated in Fig. 4. The separation module takes the span power lines' points as input and cyclically separates a single power line until all points are separated. The separation is composed of five steps: Step 1: Plane grid points are generated uniformly according to the defined grid resolution res (m). The Z-axis value of each grid point is initialized below the lowest elevation value of the span power line points. Next, the power line points are projected onto the grid surface. Within the adjacent rectangular region of a grid point, the power line points that are projected within this region are designated as the search points set for the grid point. As shown in Fig. 5.
Step 2: The upward motion of grid points at a constant speed is simulated by iteratively increasing the Z-axis value for each grid point. In each iteration, the Z-axis value of the point is increased by (res * rate) (m), where the rate is a ratio.
Step 3: Motion cessation is determined for each grid point based on the following conditions: the Z-axis distance between a grid point and the nearest power line point in the corresponding search points set is less than (res * rate) (m), or the adjacent grid points have already stopped. Grid points that stop moving due to the former condition are considered elevation reference grid points. The iteration is terminated when all grid points cease motion.
Step 4: The Z-axis distance between the power line point and the nearest elevation reference grid point is calculated. The power line points with a Z-axis distance less than the threshold h(m) are selected as the separated single-layer power line points.
Step 5: The extracted single-layer power line is rotated 90°a long its length. Repeat steps 1-4 to separate a single power line.
To account for the varying section sizes of different layers of power lines, an automatic adjustment of the threshold h (m) is necessary during the cyclic separation of individual power lines. As shown in Fig. 6, the power line points are projected onto the X-Z plane, and the power line segments are vertically cut around the lowest height point. Subsequently, the threshold h (m) is adaptively calculated based on the centralized distribution characteristics of the Z-axis values of different layers within the segment.
To further illustrate the implementation process of the algorithm in the separation module, we provide a detailed description of the primary process, as listed in Algorithm 1.

A. Dataset
The power corridor dataset was collected by DJI RTK300 UAV Lidar in Nanning, China. The raw point cloud contains the 3-D coordinate (XYZ) information of points, with a density of about 580 points per square meter. The raw point cloud contains three independent electric power corridors: S 0 , S 1 , and S 2 . The total length of all corridors is about 3500 m, with a total of 27.4 million points. These corridors include different voltage classes of pylons and various types of power lines. The raw point cloud is divided into a training set and a test set using Cloud Compare software (https://www.cloudcompare.org/). The point clouds are categorized into ground, vegetation, pylons, and power lines. The number of points for each category label is shown in Fig. 7. Fig. 8 provides more details about the training and test set division for each corridor. The lengths of the test regions in S 0 , S 1 , and S 2 are 527, 350, and 404 m, respectively, and include different span areas T. Table I presents the detailed parameters of the power lines in each testing span area. The power line lengths and densities varied significantly across different span areas, which can better test the individual power line separation algorithm's robustness.  In this study, the disconnection rate index from [5] is employed to quantify the extent of missing power line points. The disconnection rate is defined as follows: Disconnection rate = Disconnection length Total length Notably, in S 2 /T 0 , the individual power line disconnection rate reaches nearly 36% due to obstructing vegetation, posing a challenge to individual power line extraction, as shown in Fig. 8(c).

B. Semantic Segmentation on Electric Power Corridor
To evaluate the performance of SCFL-Net in the corridor semantic segmentation, we compare our method with several existing DL-based semantic segmentation algorithms, i.e., PointNet++, RandLA-Net, SCF-Net, BAAF [30], and MSIDA-Net. We apply the same training strategy to all networks to  / /H is a set of max z-value for each grid point, h is the max z-value 5 generating function.
if exist δ = 1 in N eighbor grid points then 13    We use mIou, OA, and mRec to quantify and evaluate the segmentation effect of our model. OA evaluates the accuracy of point classification. mRec evaluates the recall rate of truth value, and mIoU evaluates the semantic segmentation accuracy of each category where N is the total sample, TP i is the true-positive sample of class i, FP i is the false-positive sample of class i, FN i is the false-negative sample of class i. n class is the number of classes.
As listed in Table II, our approach shows a significant advantage over PointNet++. Although the OA of our method only improves by 1.64%, mIoU and mRec increase by 24.42% and 11.25%, respectively. Compared with the recent advanced segmentation methods, our approach achieves the highest segmentation performance with an mIoU of 96.04%. In particular, our approach outperforms SCF-Net with an improvement of  1.3% in mIoU, 0.14% in OA, and 1.23% in mRec, indicating that our LCM significantly enhances the segmentation capability of the SCF network. Furthermore, our method achieves the best segmentation of ground, vegetation, and pylon, with IoU reaching 96.76%, 96.05%, and 95.57%, respectively. The critical power line IoU is also above 95%. From the segmentation effect image analysis in Fig. 9, the majority of the misclassified points are located at the junction of the pylon and power lines. This can be attributed to the presence of structural components, such as insulators in the pylon at the connection point, which are similar to the power lines. The segmentation performance of most comparison methods is not ideal, but SCFL-Net has a good segmentation effect here. This further validates the effectiveness of our LCM in aggregating local distribution features.
The LCM employs a local neighborhood of k points to capture local distribution features. To determine the optimal value of k,   Fig. 10. With the increase in neighborhood points, local information is gradually enriched. Both OA and mIoU have been significantly improved. However, beyond a certain point, increasing the number of local points did not yield any further improvement in segmentation performance. The increase in local points cannot provide more local information, and redundant information interference appears, reducing the segmentation performance. Therefore, we choose k = 16 to provide the best semantic segmentation performance.

C. Efficiency of SCFL-Net
In this section, we systematically evaluate the overall efficiency of our SCFL-Net using the corridor data. To ensure a fair comparison, we conduct experiments on several networks [18], [19], [20] under identical conditions. The number of points put into each network is 40 960 points for one inference. The total time required to complete the inference for all corridors is compared. Table III quantitatively shows the total time of different networks. In addition, the number of parameters for each network is reported to reflect their memory consumption. SCFL-Net exhibits similar efficiency and memory consumption as SCF-Net. However, SCFL-Net is a combination of SCF-Net and LCM, and the proposed LCM only introduces less than 1k parameters compared to SCF-Net, resulting in an improvement in the performance of approximately 1.3%.

D. Ablation Study on LCM
In order to further demonstrate the contribution of Local distribution characteristic (Ld)and Local elevation standard  Table IV, we first remove the Lesd. The performance reduction from the first row to the second row demonstrates that the Lesd can effectively improve the segmentation performance of the scene. Then, the Ld continues to be removed, and the mIou is decreased by 0.8%, which demonstrates the effectiveness of Ld.

E. Span Power Line Extraction and the Axis of Symmetry Equation Search Results
To test the accuracy of the span power line extraction method. The span power line points are manually extracted from the power corridor segmentation results as the correct labels. Note that in the individual power line extraction stage, experiments are conducted on an Intel(R) Core (TM) I9-9900K CPU at a 3.60-GH processor. The algorithms are implemented in Python. Here, precision and recall are used as the evaluation metrics. Table V shows the experimental results for different power line regions. Our method's average precision and average recall are 99.95% and 100%, respectively. It demonstrates that our method can effectively extract the span power lines from the segmentation results.
To further demonstrate the versatility of our search method across different types of pylons, we conduct experiments on four pylons with varying voltage classes and structures within the dataset. We select a segment that spans 6 m from the top to the bottom of the pylon to search for the axis of the symmetry equation. Fig. 11 illustrates the results of our search method and demonstrates that our search algorithms have achieved satisfactory results for all pylons.

F. Separation of Individual Power Line Results
To evaluate the effectiveness of the separation module, we compare it with two methods. The first method is to manually separate a single power line from the span using the Cloud Compare software. The second method is to separate a single power line based on the DBSCAN method. Our separation module algorithm parameters are set to {res = 1.0, rate = 0.6} through the preliminary parameter experiment. In the evaluation  criteria, the line point extraction rate (LPE) index is adopted to measure the separation effect of a single power line, and LPE is defined as follows: where N denotes the number of individual power lines in a span area. Num * i represents the number of points on ith individual power line in the original scenario. Num i represents the number of correctly extracted points on ith individual power line. Table VI shows the comparison of the effects of each separation method. The power lines in S 0 /T 0 and S 0 /T 1 are relatively intact and connected. Therefore, the LPE of the three methods is close. However, in S 1 /T 0 and S 2 /T 0 , some individual power lines are broken to varying degrees, as shown in Fig. 12  In terms of efficiency, the time consumption of our algorithm is heavily influenced by the scale of power line points. For span power lines with a scale below 0.37 million, our algorithm does not exhibit a speed advantage. However, for span power lines with a scale of 0.85 million, our method exhibits a significant time consumption advantage. The primary reason is that the majority of computations in our method are concentrated on grid points instead of the vast number of power line points.

G. Robustness Test
To verify the robustness of our individual power line separation algorithm under different missing power line point cases.
We conduct experiments on S 1 /T 0 , which consists of different types of power lines, including both single conductors and bundled conductors. Two scenes are generated to simulate disconnections in the power lines: disconnections in the middle (Case1), and disconnections at the edge (Case2). In addition, scenes with disconnection rates ranging from 10% to 15% are created for both Case1 and Case2. The individual power lines in different colors shown in Fig. 13 are used as labels.

H. Parameter Sensitivity Analysis on Individual Power Line Extraction
There are two essential parameters res and rate in individual power line extraction. In the experiment, one parameter is kept constant while the other varies within the optimization space to assess their sensitivity. After a number of test experiments, we preliminary found a favorable combination of parameters {res = 1.0, rate = 0.6}. The optimization space for res is set as {0.5, 1, 1.5, 2, 2.5}, and for rate, it is set as {0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0}.
As shown in Fig. 14(a), while keeping rate = 0.6 and varying res within the range of [0.5, 2], LPE for power lines within four spans remains high. However, at res = 2.5, there is a significant drop in S 2 /T 0 performance, whereas the performance in other regions remains stable. Hence, lower values of res can provide greater robustness against the missing points problem. In terms of rate, when res is fixed at 1.0 and rate is less than 0.3, the performance across all regions is poor. When rate >0.3, the performance remains stable, as shown in Fig. 14(b). Based on these results, the parameter combination can maintain the consistently high accuracy of LPE for power lines within four spans, while minimizing the negative impact of the missing point issue.

IV. CONCLUSION
In this study, we have proposed an effective method for automatic semantic segmentation and individual power line extraction from electric power corridor point clouds for power line inspection. In the semantic segmentation stage, our method uses a specially designed LCM to improve the corridor semantic segmentation ability of SCF-NET. In the individual power line extraction stage, we use the symmetry axis equation of the pylon and the flexible grid to extract individual power lines. Our individual power line extraction method has shown a speed advantage when processing massive power line points. Experimental results show that the proposed methods achieve high accuracy in both stages. Specifically, the SCF-Net with LCM achieved a segmentation mIoU of 96.4%. In the extraction experiment, the LPE of all four spans is above 96.56%. The robustness test indicates the robustness of our method to missing power line point situation.
However, we acknowledge that some drawbacks still need to be addressed, such as the reliance on large datasets for training the segmentation model and the sensitivity of the flexible grid filter method to clutter points. Future work will focus on building a more generalized dataset and exploring instance-aware segmentation networks for end-to-end individual power line extraction. He is currently a Professor with the Faculty of Geographical Science, Beijing Normal University, Beijing, China. His research interests include remote sensing image processing, three-dimensional urban reconstruction, and spatial object recognition.
Xingwen Huang received the B.S. degree in electrical engineering from the Hefei University of Technology, Hefei, China, in 2019. He is currently working toward the postgraduate degree in pattern recognition and deep learning at the Department of Electrical Engineering, University of Guangxi, Nanning, China.
His research interests include intelligent electric power inspection and point cloud semantic segmentation.
Jianchuang Qin received the bachelor's degree in automation from the Wuhan University of Technology, Wuhan, China, in 2021. He is currently working toward the M.Sc. degree in artificial intelligence at the Intelligent Robot Research Center, School of Electrical Engineering, Guangxi University, Guangxi, China.
His research interests include mobile laser scanning and point cloud instance segmentation.