Long-Term Monitoring for Track Slab in High-Speed Rail via Vision Sensing

Track slab deformation has become a challenging issue in high-speed rail (HSR) operation in recent years. This article proposed a novel approach for track slab deformation monitoring based on computer vision techniques. The basic principle of visual measurement of track slab displacement is first introduced. Then the detailed process of slab displacement calculation from the on-site images is presented, including region of interest (ROI) extraction, determination of the target edge, and displacement calculation. In this process, considering the actual operation environment of in-service HSR lines, an improved Canny algorithm, which can adaptively extract the location information of the target is proposed and employed in the image processing. Based on the modular design method, an online monitoring system for the displacement of the track slab is established. The devised system is installed on an in-service HSR line for long-term slab deformation monitoring. The performance of the proposed system is verified by one-year monitoring data of slab displacement. The measurement results of the proposed system are compared with an existing linear variable differential transformer (LVDT) system, demonstrating that it can accurately report track displacement with lower cost and easier instrumentation. This research is expected to provide insights to the railway maintenance-of-way department for better management and maintenance of slab deformation, especially under high temperature.


I. INTRODUCTION
With the increase of the service time of the high-speed rail (HSR) slab track infrastructure, the deterioration of track geometry caused by slab deformation gradually appears [1]. Typical structural defects associated with the slab deformation include inter-layer de-bonding and slab upwarping [2]- [4] (Fig. 1), which can reduce the track serviceability and compromise the operation safety [5]. Previous studies reveal that slab deformation is mainly caused by the significant difference of the inter-layer coordination in the displacement of the slab track, especially in the summer [6]. Without timely detection, slab deformation as well as the slab defects can be worsened under the large wheel load of the high-speed train, and reduce the structure safety and serviceability [7]. Therefore, it is necessary to carry out condition inspection for slab tracks to detect the potential slab deformation at an early stage.
The associate editor coordinating the review of this manuscript and approving it for publication was Szidónia Lefkovits .
The most common measure to inspect track structure condition is the regular inspection conducted by running the track geometry vehicles [8]. However, the defects are difficult to be reflected in the track geometry data until they develop to a very serious degree because the variation of the track geometry data can be affected by multiple factors and the feature of track slab defects can hardly to be extracted. As an attempt to detect the defects at earlier stages, the maintenance-ofway department always arranges auxiliary manual inspection, but the efficiency and cost of manual inspection can be a growing concern as the maintenance time is commonly at night [9]. In this regard, long-term monitoring for the structural conditions can be a good way to detect the initiation and development of potential defects and schedule maintenance works accordingly.
Deformation monitoring for track structure is usually achieved with linear variable differential transformer (LVDT)-based system [10]- [13], which can measure structural displacement with transducers, mounted on the track and external data acquisition instrument. However, as it needs fixed installation on the structure, the in-situ installation can cost a long time and a lot of manpower. Besides, the instrumentation of the LVDT-based system may compromise the integrity of the track structure and cause stress concentration. Another drawback is that an external power supply is needed on-site [14], [15]. An alternative is employing accelerometers and obtaining track displacement with double integral. But the problem is the difficulty of identifying the trend term in integral and the uncertainty of acceleration data during long-term monitoring can also affect the measurement accuracy [16], [17].
To overcome the drawbacks of the existing sensing systems, the non-contact non-destructive testing method has become one of the effective ways [18]. The typical non-contact method includes Global Positioning System (GPS), laser Doppler vibrometer (LDV), and computer vision [19]. Due to the positioning error of GPS, its measurement accuracy is only between 5 and 10 mm [17], [20], which limits its application in slab track, since the slab displacement is generally no more than 2 mm. LDV can be more accurate but it is challenged with the high price, energy consumption, and is harmful to health [21], [22]. With the recognition of this, this article proposes a computer vision-based slab displacement monitoring approach. Through long-term slab displacement monitoring, the temporal variation of slab deformation as well as the dominant factors behind the variation can be revealed. Using the long-term monitoring data, the slab maintenance can be scheduled more rational to ensure operational safety and reduce the maintenance cost of HSR.
The rest of this article is organized as follows: Section 2 briefly introduces the principle of computer vision and gives the image processing algorithm for track displacement measurement; Section 3 presents the development of the on-line slab displacement monitoring system and the application of the proposed system on in-service HSR line with performance validation is described in Section 4; Section 5 gives the brief conclusion.

A. BASIC PRINCIPLE
In computer vision-based displacement measurement, with the images of the object to be measured, the feature information of the object can be obtained through image processing. Then the displacement can be calculated using the relationship between the spatial position of the object and the image information. As light is the media between the target and measurement system, the physical characteristics of the target remain unchanged. The computer vision-based measurement method is featured with being contactless, real-time, and continuous shooting. The main advantages that are favorable in HSR online monitoring include antiinterference, low cost, and high precision. Fig. 2 illustrates the principle of measuring the displacement of track slab, where point P(X,Y,Z) is the manually marked point fixed on the track, which is transferred to point p(u,v) in the image plane with machine learning approaches.
To obtained the coordinate of point P on the track, the coordinate system of the image plane is transformed into that of track, which is expressed as: where Z C is the projection length on the optic axis between the manually marked point P and the light center of the camera, and M is the parameter matrix, which is given by where the first matrix is the internal parameter matrix with camera distortion; F x and F v are the equivalent focal length on axis x and y, respectively; C x and C v are the coordinates of focal axis and image focus of the image plate on axis x and y, respectively. The second matrix is the external parameter matrix, where R and T are the 3 × 3 rotation matrix and the 3 × 1 translation matrix, respectively. In this article, the internal and external parameters are calibrated using the calibration technique [30]. Unit conversion is required between the marking center coordinate (unit: pixel) of the measurement model with computer vision and actual track displacement (millimeter). If two calibration plates are fixed on the track slab with a preset distance between circle centers with coordinates (a, b, c) and (A, B, C), then the initial vertical distance is d = Bb. Thus the movement of circle i at time j can be represented with d ij (i = 1,, n) where the displacement changes of the pixel is P i = d ijd i1 , thereby the displacement of the marking point i at time j is given by: where k i is the ratio between slab displacement and imaging pixel and it can be calculated with the initial conditions of circle center distance P i and actual displacement D i . Then the movement of the n circle centers is averaged to get the track slab displacement S.

B. DIGITAL IMAGE PROCESSING WITH IMPROVED CANNY ALGORITHM
According to the basic principle of computer vision, the key to accurately measure the displacement of the track slab is to determine the center position of calibration circles on the target. This is a challenging task under the on-site harsh environment of railway track because the rainwater and the lack of light can affect the accuracy of circle center determination. To ensure both the speed and accuracy of circle center determination, the algorithm of digital imaging processing is adopted. The calculation process is illustrated in Fig. 3. After acquiring the images of the track slab, the region of interest (ROI) will be extracted for displacement measurement. Fig. 4 shows the ROI area in the original image. It is seen that the area of ROI in the image is small, so extracting the ROI from the original image can effectively narrow the scope of image processing and analysis, and improve the computation efficiency. The next step is to locate the edge of the calibration circles, which is the fundamental problem for the determination of the circle center of the target. Because the camera angle of the monitoring system is relatively fixed, the position change is limited, which means that the displacement of the scale is also small. In such a case, the Canny algorithm [31] is commonly adopted to determine the edge of the target. The flowchart of the Canny algorithm is illustrated in Fig. 5. In traditional Canny algorithm, Gaussian filter (GF) is employed to smooth the original image, which is expressed as where σ is the Gaussian filter parameter, which is inversely proportional to edge locating accuracy and proportional to the signal-to-noise ratio (SNR). So the value of σ is the dominant factor of the smoothing effect. But in the traditional Canny algorithm, the value of σ needs to be set artificially, which lowers the calculation efficiency of image processing as well as the degree of image smoothness. VOLUME 8, 2020 To solve this problem, this article uses the adaptive median filters (AMF) [32] instead of the GF algorithm to smooth the original image. The basic idea of AMF is to use a small average weighting template to perform iterative convolution with the original image [32], and update the weighting coefficients of each pixel adaptively in each iteration. The iterative operation of AMF sharpens the edge of the image in the process of noise suppression, which facilitates the subsequent process of edge detection. Define the input image as f (x, y), the calculation steps of one iteration are as follows: Step 1: determine the image gradient components G x (x, y) and G v (x, y) by Eq. (5) and Eq. (6) Step 2: determine the template coefficient by Eq. (7) w Step 3: take the weighted average for image after nth iteration f (n) (x, y) using Eq. (8) where the parameter k in Eq. (4) need to be preset before calculation and the value of k controls the preservation of the edge information.
To compare the efficiency of GF and AMF, the two methods are employed for image processing. 10% salt-and-pepper noise has been added to the original image. The comparison result is shown in Fig. 6, which indicates that AMF outperforms GF with better precision. To further address the problem of adaptive imaging partition and avoid the manually preset of threshold values, an iterative block partition algorithm based on Otsu's method [33] is proposed. This algorithm is based on the gray histogram and the least square method. As a statistical segmentation algorithm, it chooses an optimum threshold, which maximizes the inter-class variance of the foreground and background. Assume that the image size is M×N, the number of total gray levels is L and the number of pixels at gray level i is f i , then the probability of the occurrence of gray level i is where i = 0, 1, . . . , L -1, subjected to L−1 i=0 = 1. The pixels in the image are divided into two classes by a gray level threshold k, namely, background C 0 and foreground C 1 , in which the gray level range of C 0 is [0, k] whereas that of C 1 is [k+ 1, L -1]. So the probabilities of background C 0 and foreground C 1 are expressed as (10) and (11) respectively.
So the average gray value of C 0 and C 1 can be obtained by Eq. (12) and (13), respectively.
and the average gray value of the whole image is Therefore the inter-class variance between C 0 and C 1 can be expressed as According to Otsu's method, maximizing the inter-class variance δ2 can perform the automatic image thresholding. This article defines the optimum threshold obtained by Otsu's method as the high threshold and set the low threshold as half of the high threshold. With a combination of Canny operator and Otsu's method, the optimum threshold can be chosen adaptively, overcoming the drawback of setting the high and low threshold artificially.
To validate the improved Canny algorithm as proposed above, we take the manually marked points collected on-site as the original image and results of the traditional and improved Canny algorithm are compared as in Fig. 7. It is seen that the improved Canny algorithm outperforms the traditional Canny algorithm, especially under the condition of varying strength of light. Based on the crude extraction of the circle edge with the improved Canny algorithm, precise confirmation (sub-pixel positioning) can be realized with Gaussian fitting, as shown in Fig. 8. The left panel of Fig. 8 shows the edge information of the target circle obtained with the improved Canny algorithm. The circle center is then determined with the edge information, as shown in the right panel for Fig. 8, in which the dotted line in the image represents the pixel size of each integer image, and the gradient function is denoted as F, so that the coordinate origin coincides with the circle edge. Then we generate 2N + 1 sampling points along the gradient direction (i.e. x-axis) through linear interpolation, i.e. F(-N )... F(0)... F(N ). By fitting these sampling points, the sub-pixel edge coordinates corresponding to the extreme points of Gaussian fitting function can be obtained. After fitting with these points F(-N )... F(0)... F(N ), the extreme point with Gaussian fitting function can be obtained, which represents the edge coordinate of sub-pixel. Thus the displacement of track slab can be measured.

III. DEVELOPMENT OF THE ON-LINE MONITORING SYSTEM A. SYSTEM CONFIGURATION
To satisfy the needs of on-site slab displacement monitoring, the actual operating environment for the monitoring system should be fully considered. Namely, the monitoring system should be capable to take high-quality images, have a high transmission rate, and use the solar power supply to maintain operation. Meanwhile, because there is a need to install the monitoring system at multiple sites, the system should be cost-effective. Therefore, this article adopts the modular design method in the development of the monitoring system according to the functional requirements. The system consists of image acquisition module, wireless communication module, power module, main control system, and marking points. Besides, a specific fixture is designed to ensure the stability and reliability of the system, especially during the passage of the trains. Fig. 9 shows the schematic of the devised online system.

B. DESIGN OF THE MODULES
The detailed description of the design for different modules is as follows:

1) IMAGE ACQUISITION MODULE
For image acquisition module, there are two commonly used sensors for the field environment: charge-coupled devices (CCD) and complementary metal-oxidesemiconductor (CMOS). In this study, CMOS sensors with VOLUME 8, 2020 high integration and good compatibility are selected, which consume almost no energy when not collecting data. The imaging module that encapsulates CMOS sensors employs OV4689 photosensitive devices with the resolution of four megapixels and 2,560 × 1,440 pixels for the device and imaging, respectively.

2) WIRELESS COMMUNICATION MODULE
Regarding wireless communication, the low energyconsuming module with GPRS, built up as GPRS SIM800C, is adopted for real-time transmission of monitoring data. Reliable data transfer, compact appearance, and low price characterize this module. The monitoring data are transmitted to control center every 30 minutes. In case of data loss in transmission, the monitoring data can be stored on-site in a storage unit which has the capacity of three-month data for a ten-camera system.

3) SOLAR POWER MODULE
Faced with no exterior power supply on-site, the solar power module is developed with monocrystalline silicon solar panels. To address the impact of the cloudy and rainy climate in the areas of many HSR lines on the solar power supply, the storage battery is added with 7500Ah capacity sustaining for 15 days. In addition, due to the poor lighting at night, an LED light source is adopted to provide proper illumination by setting a voltage-conditioning module.

4) CONTROL UNIT
The control unit is developed based on the Amlink-Temolin platform, using four ARM Cortex-A53 processors, integrated dual-channel SRAM and EMMC Flash, as well as UART, SPI, IIC, SDcard, USB Host, USB Device, WiFi, Bluetooth, GPS, 2G, 3G, 4G, FM, and other communication interfaces, with the power consumption of less than 5mA.

5) MARKING POINTS
The design of marking points should take full consideration of the target parameters of imaging collection. If the marking point is too small, the number of pixels in the image will be insufficient, and the center of the mark point cannot be accurately extracted; if the marking point is too large, the size of target plate can be too big and it is difficult to be installed and fixed on the track slab and CA mortar layer. In this regard, a single marking point with a diameter of 3 mm is selected according to the actual situation on-site. To reduce the measurement error caused by lens distortion and other factors, two kinds of artificial marking points of 9 marking circles are designed and mounted on the track slab.
Considering that the markers can be affected by rainwater on rainy days which results in incomplete markers, this article combines the Hough transform and the least square method to extract the center of the incomplete circles: firstly, the center and radius of the incomplete circle are roughly extracted by Hough transform, so that the edge of the incomplete circle can be extracted and the coordinates of all points on the edge can be obtained; then, with the center and radius of the incomplete circle, we can judge whether the edge points are on the boundary of the fitting circle; finally, the center coordinates of the circle after fitting are calculated by the least square method.

C. INTEGRATION
Regarding the image processing, considering both the computational accuracy and efficiency, in AMF, the number of iteration is set as 5. Table 1 summarizes the specifications of the monitoring system, with a total cost of around USD 1,000. Fig. 10 shows the physical photos of the system components. The monitoring data collected from the above system is exchanged and analyzed with Ali cloud.

IV. IN-SITU INSTRUMENTATION AND PERFORMANCE VALIDATION
To validate the proposed displacement measurement method and the performance of the monitoring system, a trial system is installed on a section of Shanghai-Hangzhou HSR line. The track type of this section is CRTSII, a longitudinally coupled prefabricated slab track, which is laid on the subgrade. The monitoring section is located at 121 • 17'E, 30 • 1'N. The HSR line orientation is from southwest to northeast. The operation speed is 300 km/h. Fig. 11 shows the actual environmental conditions and Fig. 12 illustrates the system installation.   Sensors of the monitoring system with circular markers are installed on the track slab and the CA mortar layer. The camera is fixed 60 cm away from the track slab with anchor bolt on the concrete roadbed, which can be regarded as a rigid connection. Adjusting LED light strength and targeting COMS sensors directly to marking points can regard slab displacement regarded as the average vertical downward displacement of marking points. To further validate the accuracy of the measurement result, a comparative test is conducted.  The results of the proposed system will be compared with those of an LVDT system, which is installed at the neighboring slab, as shown in Fig. 13. Compared to the LVDT system, the proposed monitoring system only needs to be installed on the roadbed, as shown in Fig. 12, and the volume of equipment is smaller than LVDT equipment.
As indicated in previous research, the temperature is the dominant factor of the variation of the slab displacement [24]. Therefore, we investigate the displacement monitoring data of both systes under different temperature conditions. Figs. 14 to 17 shows the time series of track displacement data of the four months of January, April, July, and October with significantly different ambient temperature (averaged as 4.36m, 15.43m, 28.23m and 14.18m by weather station) with their distribution.
Data collected from the proposed system and LVDT system present a similar trend, amplitude value, and distribution characteristics, and they all change periodically with temperature. The difference of amplitude, especially in April and July, can be explained as the two monitoring systems are installed on different slabs. Thus it can be concluded that the proposed computer vision can effectively capture the changes of track displacement with varying temperatures. To further clarify the correlation between data of temperature and track displacement, canonical correlation analysis (CCA) [34] results of different measuring points are listed in Table 2. It is found that the correlation coefficients are always larger than 0.7, which means the displacement and temperature are highly correlated. Besides, the higher the ambient temperature, the higher the accuracy of the proposed monitoring system in displacement measurement.   Regarding the displacement at different positions on the track slab, it is observed that at the median and end of the track slab, the amplitude and trend of track displacement are similar, i.e. present similar variation pattern with the ambient temperature. In addition, the difference between the displacement at the slab end and median is small mainly  because of the large structure stiffness of the large track structure which causes only slight deformation within the range of 3 m. Besides, it is found that track displacement increases rapidly in April compared to that in January and comes to a peak in July. This can be explained as that higher temperature accelerates the internal stress of the slab, which has the potential to cause track defects, such as that shown in Fig. 1. Thus it is concluded that track maintenance in the summer should be emphasized to control slab deformation and inter-layer de-bonding. Large-scaled maintenance such as to strengthen the CA mortar layer and to replace track slab is recommended for low ambient temperature.

V. CONCLUSION
Structural defects can be detrimental to HSR operation especially for the lines with slab track, so timely inspection of track structure condition is fundamental to ensure the operational safety and reduce the maintenance cost of rail infrastructure. This study proposes a track deformation monitoring method for slab track based on computer vision. The principle of computer vision-based track displacement measurement is first introduced followed by the description of the process of displacement calculation with the digital image processing method. In displacement calculation, this article proposes an improved Canny algorithm in which the detection of the target edge can be conducted automatically and the accuracy of displacement measurement can be improved.
Considering the actual harsh environment of the in-service HSR line, the on-line monitoring system is established with a modular design. The devised system includes image acquisition module, wireless communication module, power module, main control system, and marking points. With in-situ monitoring data, the performance of the proposed method is compared with that of the LVDT system in the neighboring slab. Results show that the proposed online monitoring system can accurately capture track displacement and it has the advantages of lower cost and easier installation compared with the LVDT system. In addition, through the in-situ test, the track slab deformation features under high temperatures are revealed which indicates that the slab end deserves more attention of displacement monitoring than the slab median. This research can provide insights to railway authority for better management of track slab displacement under high temperature and further to properly schedule the maintenance work.